Embedding models

Semantic and hybrid retrieval embed with BAAI/bge-small-en-v1.5 by default: in-process, keyless, English-only. The model is configurable per catalog (ADR 0012).

Declare it once, at construction, with the embedding option next to method — ratel({ method, embedding }) in TypeScript, the catalog constructor in Python. The configured embedding spec defines the vector space used for both corpus and query embeddings. For semantic and hybrid catalogs without an artifact, the corpus is embedded during register. An experimental build-time artifact can preload covered corpus vectors instead.

Construction only validates the spec. Model resolution, loading, endpoint calls, and artifact compatibility checks happen later during dense preparation or search.

The four sources

• HuggingFace repo: { huggingface: "BAAI/bge-base-en-v1.5" }. Runs in-process via Candle, like the default. BERT-family sentence-transformers models (bge, e5, gte, MiniLM).
• Local directory: a bare path string, or { local: "/opt/models/bge" }. The same in-process engine for an air-gapped or private checkpoint.
• Ollama: { ollama: "nomic-embed-text" }. Calls the local Ollama server at http://localhost:11434/v1/embeddings; a 404 hints ollama pull <model>.
• OpenAI-compatible endpoint: { url, model, apiKeyEnv }. Any /embeddings API: OpenAI, TEI, vLLM, a remote Ollama. This is the route for non-BERT and multilingual models the in-process engine cannot load, and the only embedding source that can send input text to a remote inference service or need an API key.

The source is always named by its key, never guessed: a bare string that looks like a repo id or a URL is rejected with a pointer to the right object form.

from ratel_ai import ToolCatalog

# A bigger HuggingFace model, still in-process and keyless.
catalog = ToolCatalog(
    method="hybrid",
    embedding={"huggingface": "BAAI/bge-base-en-v1.5", "download": True},
)

# An OpenAI-compatible endpoint; the key is read from the env at call time.
remote = ToolCatalog(
    method="semantic",
    embedding={
        "url": "https://api.openai.com/v1/embeddings",
        "model": "text-embedding-3-small",
        "api_key_env": "OPENAI_API_KEY",
    },
)

import { ratel } from "@ratel-ai/sdk";

// A bigger HuggingFace model, still in-process and keyless.
const r = ratel({
  method: "hybrid",
  embedding: { huggingface: "BAAI/bge-base-en-v1.5", download: true },
});

// An OpenAI-compatible endpoint; the key is read from the env at call time.
const remote = ratel({
  method: "semantic",
  embedding: {
    url: "https://api.openai.com/v1/embeddings",
    model: "text-embedding-3-small",
    apiKeyEnv: "OPENAI_API_KEY",
  },
});

In TypeScript, ratel({ embedding }) forwards the spec to both internal catalogs; the standalone ToolCatalog and SkillCatalog take the same option, and so do the registries underneath. The tool and skill caches are independent, so two catalogs can run different models.

In-process models

• Cache-only by default. Ratel auto-downloads only the built-in default; a configured HuggingFace model must already be in the HuggingFace cache (huggingface-cli download <repo>), or you opt in with download: true. A missing model errors with that hint (EmbedderError, code NotCached); register never silently pulls gigabytes. When a cold download does happen it emits an embedder_download trace event with the actual byte size.
• revision pins a git revision (default main) for reproducible vectors.
• Pooling is auto-detected from the repo's 1_Pooling/config.json; pooling: "cls" | "mean" overrides it. A model with no pooling metadata warns and assumes mean.
• Asymmetric models (e5 and friends) take queryPrefix / docPrefix (query_prefix / doc_prefix in Python); each side is prefixed automatically.
• A non-BERT model fails to load with a typed error that points at the endpoint route.

Endpoints

• apiKeyEnv / api_key_env names the env var holding the bearer key, read at call time; the secret never lands in code or serialized config. A named-but-unset var is an immediate error, not a downstream 401. Omit it for keyless servers.
• Document batches are chunked (up to 64 inputs per request), keep their order, and commit only after every chunk succeeds.
• Every vector from any source is L2-normalized at the boundary, so an endpoint returning un-normalized vectors cannot skew cosine ranking.
• For semantic and hybrid catalogs, the dense cache is prepared during registration. Without an embedding artifact, each fresh process embeds the registered corpus through the configured backend. Stable catalogs can instead configure an experimental build-time embedding artifact to reuse covered corpus/document vectors. Over a remote endpoint, corpus embedding adds real latency and cost when those vectors are not reused, so pass whole batches to register and prefer a long-lived process.

Build-time embedding artifacts (experimental)

If the searchable metadata for your Tools or Skills is known at build or deploy time, its embeddings can be built once and stored in an artifact. The artifact stores precomputed vectors for matching catalog entries; it does not replace Tool or Skill registration, executors, or the runtime catalog itself.

At runtime, register the same catalog entries as usual and configure the artifact alongside them. Ratel warms covered entries from those vectors on register (and skills replaceAll / replace_all) instead of running corpus/document embedding inference again.

Artifacts only replace corpus/document inference for covered entries. They do not eliminate query embeddings or all runtime embedding-backend work: Ratel still validates artifact compatibility, Local/HuggingFace models may still initialize or load, and an endpoint-backed configuration may make a compatibility request when needed. Semantic and hybrid searches continue to embed each query with the configured backend.

from ratel_ai import (
    ExecutableTool,
    Skill,
    SkillCatalog,
    ToolCatalog,
    experimental_build_embedding_artifact,
)

embedding = {"huggingface": "BAAI/bge-base-en-v1.5", "download": True}

tool = ExecutableTool(
    id="read_file",
    name="Read file",
    description="Read a text file.",
    execute=lambda _args: {"contents": "..."},
)

skill = Skill(
    id="file-review",
    name="File review",
    description="Review and summarize a text file.",
)

await experimental_build_embedding_artifact(
    output="./catalog.ratel-embeddings",
    embedding=embedding,
    tools=[tool],
    skills=[skill],
)

artifact = {"path": "./catalog.ratel-embeddings"}

tools = ToolCatalog(
    method="hybrid",
    embedding=embedding,
    experimental_embedding_artifact=artifact,
)

skills = SkillCatalog(
    method="hybrid",
    embedding=embedding,
    experimental_embedding_artifact=artifact,
)

await tools.register(tool)
await skills.register(skill)

import {
  experimentalBuildEmbeddingArtifact,
  ratel,
  type ExecutableTool,
  type Skill,
} from "@ratel-ai/sdk";

const embedding = { huggingface: "BAAI/bge-base-en-v1.5", download: true };

const tool: ExecutableTool = {
  id: "read_file",
  name: "Read file",
  description: "Read a text file.",
  inputSchema: {},
  outputSchema: {},
  execute: async () => ({ contents: "..." }),
};

const skill: Skill = {
  id: "file-review",
  name: "File review",
  description: "Review and summarize a text file.",
};

await experimentalBuildEmbeddingArtifact({
  output: "./catalog.ratel-embeddings",
  embedding,
  tools: [tool],
  skills: [skill],
});

const r = ratel({
  method: "hybrid",
  embedding,
  experimentalEmbeddingArtifact: {
    path: "./catalog.ratel-embeddings",
  },
});

await r.tools.register(tool);
await r.skills.register(skill);

• Configure path or bytes at runtime (exactly one).
• Default miss policy is "error" (onMiss / on_miss).
• "embed" reuses covered entries and embeds only missing or invalidated ones.
• Runtime warm never updates or writes through to the artifact.
• One mixed artifact can contain both Tool and Skill entries. In TypeScript, ratel() forwards the configured artifact to both catalogs; in Python, the same artifact can be configured on both ToolCatalog and SkillCatalog.
• With default "error", every non-empty Tool or Skill corpus registered with that artifact must be fully covered.
• Artifact/model incompatibility fails closed; rebuild the artifact after changing the embedding model.

Changing the model

To use a different embedding model, construct a new catalog — or a new ratel() core in TypeScript — with the new embedding spec and register the corpus again. If you use a build-time embedding artifact, build a new artifact with that same embedding spec before using it with the new catalog.

Ratel validates the model identity and vector dimensions associated with dense embeddings and artifacts. Incompatible vector spaces fail instead of being mixed into the same search index.

Embedding-backend failures such as model loading, download, endpoint, authentication, and dimension errors surface as EmbedderError (with DimensionMismatchError for dimension mismatches). They can surface during awaited dense preparation or while a semantic/hybrid search embeds its query.

Failures while warming a configured embedding artifact surface as ArtifactWarmError from awaited registration or replacement operations such as register(...), replaceAll(...), and replace_all(...).

For embedding configuration types, see EmbeddingSpec and EmbeddingModelConfig (TypeScript) and the config dicts (Python).

Next steps