Embedding models

Semantic and hybrid retrieval embed with BAAI/bge-small-en-v1.5 by default: in-process, keyless, English-only. The model is configurable per catalog (ADR 0012).

Declare it once, at construction, with the embedding option next to method. The same model embeds the corpus at register and the query at search, so the two can never land in different vector spaces. Construction only validates the spec; nothing loads until the first embedding.

The four sources

• HuggingFace repo: { huggingface: "BAAI/bge-base-en-v1.5" }. Runs in-process via Candle, like the default. BERT-family sentence-transformers models (bge, e5, gte, MiniLM).
• Local directory: a bare path string, or { local: "/opt/models/bge" }. The same in-process engine for an air-gapped or private checkpoint.
• Ollama: { ollama: "nomic-embed-text" }. Calls the local Ollama server at http://localhost:11434/v1/embeddings; a 404 hints ollama pull <model>.
• OpenAI-compatible endpoint: { url, model, apiKeyEnv }. Any /embeddings API: OpenAI, TEI, vLLM, a remote Ollama. This is the route for non-BERT and multilingual models the in-process engine cannot load, and the only source that can leave the machine or need an API key.

The source is always named by its key, never guessed: a bare string that looks like a repo id or a URL is rejected with a pointer to the right object form.

from ratel_ai import ToolCatalog

# A bigger HuggingFace model, still in-process and keyless.
catalog = ToolCatalog(
    method="hybrid",
    embedding={"huggingface": "BAAI/bge-base-en-v1.5", "download": True},
)

# An OpenAI-compatible endpoint; the key is read from the env at call time.
remote = ToolCatalog(
    method="semantic",
    embedding={
        "url": "https://api.openai.com/v1/embeddings",
        "model": "text-embedding-3-small",
        "api_key_env": "OPENAI_API_KEY",
    },
)

import { ToolCatalog } from "@ratel-ai/sdk";

// A bigger HuggingFace model, still in-process and keyless.
const catalog = new ToolCatalog({
  method: "hybrid",
  embedding: { huggingface: "BAAI/bge-base-en-v1.5", download: true },
});

// An OpenAI-compatible endpoint; the key is read from the env at call time.
const remote = new ToolCatalog({
  method: "semantic",
  embedding: {
    url: "https://api.openai.com/v1/embeddings",
    model: "text-embedding-3-small",
    apiKeyEnv: "OPENAI_API_KEY",
  },
});

SkillCatalog takes the same embedding option, and so do the registries underneath. The tool and skill caches are independent, so the two catalogs can run different models.

In-process models

• Cache-only by default. Ratel auto-downloads only the built-in default; a configured HuggingFace model must already be in the HuggingFace cache (huggingface-cli download <repo>), or you opt in with download: true. A missing model errors with that hint (EmbedderError, code NotCached); register never silently pulls gigabytes. When a cold download does happen it emits an embedder_download trace event with the actual byte size.
• revision pins a git revision (default main) for reproducible vectors.
• Pooling is auto-detected from the repo's 1_Pooling/config.json; pooling: "cls" | "mean" overrides it. A model with no pooling metadata warns and assumes mean.
• Asymmetric models (e5 and friends) take queryPrefix / docPrefix (query_prefix / doc_prefix in Python); each side is prefixed automatically.
• A non-BERT model fails to load with a typed error that points at the endpoint route.

Endpoints

• apiKeyEnv names the env var holding the bearer key, read at call time; the secret never lands in code or serialized config. A named-but-unset var is an immediate error, not a downstream 401. Omit it for keyless servers.
• Document batches are chunked (up to 64 inputs per request), keep their order, and commit only after every chunk succeeds.
• Every vector from any source is L2-normalized at the boundary, so an endpoint returning un-normalized vectors cannot skew cosine ranking.
• The embedding cache is in-process only: every process start re-embeds the corpus. Cheap for a local model; over a remote endpoint it is real latency and cost, so pass whole batches to register and prefer a long-lived process.

Changing the model

There is no rebuild call in the SDKs: construct a new catalog with the new spec and re-register the corpus.

The cache stamps the model identity and vector width that built it. A vector of another width is a hard DimensionMismatchError; a different model identity is a ModelMismatch error. Mixing vector spaces never degrades into silently wrong rankings. Every embedding failure (load, download, endpoint, auth, dimension) is a typed EmbedderError with a stable code, surfaced from await register(...) or await searchAsync(...) / await search_async(...).

The exact types are in the reference: EmbeddingSpec and EmbeddingModelConfig (TypeScript), the config dicts (Python).

Next steps