Semantic & hybrid search

BM25 misses the query that shares no words with the right definition. "Summarize this contract" will not surface extract_document_text unless someone wrote the overlap in.

Two opt-in methods close that gap (ADR 0011). The default engine and the text they all rank is covered in Keyword search.

The two methods

• "semantic" embeds the query with a local model and cosine-ranks it against pre-built per-tool vectors.
• "hybrid" runs both the BM25 and semantic arms, each to a depth of at least 100, and fuses them with Reciprocal Rank Fusion (k = 60, no reranker).

RRF fuses rank positions, not raw scores. BM25's unbounded scores and cosine's [-1, 1] never need reconciling — which also means scores are not comparable across methods. Treat each as the engine's raw score for that query.

All three methods rank the same indexed projection; embeddings are computed over that text, not raw JSON.

Enable it per catalog or per call

The method is a string — "bm25" (default), "semantic", or "hybrid" — chosen at catalog construction or overridden on a single search. The per-call value wins.

from ratel_ai import ToolCatalog

# Per catalog: every search defaults to hybrid.
# register() embeds each new tool eagerly; a model-load
# failure raises at register (fail-fast).
catalog = ToolCatalog(method="hybrid")

# Per call: override the catalog default for one search.
hits = catalog.search("summarize this contract", 5, method="semantic")

import { ToolCatalog } from "@ratel-ai/sdk";

// Per catalog: every search defaults to hybrid.
// register() embeds each new tool eagerly; a model-load
// failure throws at register (fail-fast).
const catalog = new ToolCatalog({ method: "hybrid" });

// Per call: override the catalog default for one search.
// method is the 4th argument, after origin.
const hits = catalog.search("summarize this contract", 5, "direct", "semantic");

SkillCatalog takes the same method option and per-call override; skills rank in their own corpus with the same engines.

One consumer never overrides: search_capabilities always searches with the catalog's construction-time default, and Ratel Local inherits it the same way. To change the engine your agent uses, change the catalog.

Build embeddings first

Semantic and hybrid rank against a pre-built embedding cache. A search only ever embeds the query, never the corpus.

A catalog constructed as semantic or hybrid keeps the cache current automatically: every register embeds the new tool. A BM25-default catalog that wants per-call semantic or hybrid searches must build the cache explicitly:

from ratel_ai import ToolCatalog

catalog = ToolCatalog()      # method defaults to "bm25"
# ...register tools...
catalog.build_embeddings()   # embed anything not yet embedded
hits = catalog.search("summarize this contract", 5, method="hybrid")

import { ToolCatalog } from "@ratel-ai/sdk";

const catalog = new ToolCatalog(); // method defaults to "bm25"
// ...register tools...
catalog.buildEmbeddings();         // embed anything not yet embedded
const hits = catalog.search("summarize this contract", 5, "direct", "hybrid");

The cache is incremental and keyed by id. Adding or re-registering one tool costs one embedding, not N; a call with nothing left to embed is a no-op. The per-language cache walk-throughs — including MCP-ingested tools and the skill corpus — live on Register tools (TypeScript, Python).

Re-registering a tool invalidates its cached vector. On a BM25-default catalog, run build_embeddings again before the next semantic search (an eager catalog rebuilds at register). Both methods are synchronous and fail if the model cannot load: TypeScript throws an Error, Python raises RuntimeError.

The local embedding model

Opting in loads BAAI/bge-small-en-v1.5 (384 dimensions) in the SDK's own process:

• Pure-Rust Candle inference, CPU-only. No service, no API key, no GPU.
• Pinned HuggingFace revision, so embeddings are reproducible across machines and over time.
• Downloaded on first use into the shared HuggingFace cache (~/.cache/huggingface), then loaded offline. Nothing downloads at install time or inside a search.
• Roughly 130 MB resident while loaded.
• HF_HOME moves the cache; HF_ENDPOINT points the download at a mirror for proxied or air-gapped environments.
• A cold load slower than 5 s emits an embedder_load trace event with status slow — a hint the machine may be underpowered for local inference. RATEL_EMBED_SLOW_MS overrides the threshold.

Choosing a method

• Stay on BM25 (the default) until you see missed recall. When it misses, the first fix is usually better definitions, not a different engine.
• Pick semantic when your queries genuinely paraphrase: query and tool share no vocabulary, and rewording the definitions cannot close the gap.
• Pick hybrid to keep BM25's exact-identifier precision and add semantic's paraphrase recall. If you opt in at all, hybrid is usually the safer choice.
• Semantic and hybrid pay the model's costs: a one-time download, resident memory, embedding work at register time, and possible load failures. BM25 pays none of that.

Next steps