Keyword search

Every registered tool and skill definition competes for the model's attention. Stack up a few dozen and each turn opens with the model reading definitions it will never call, paying in tokens and accuracy.

Ratel keeps the catalog outside the prompt and ranks it per query. The default engine is BM25 over a schema-aware text projection of each definition: it runs in your process, deterministically — no model, no vector DB, no service.

Two ways to consume the ranking

Both SDKs expose one retrieval engine through two patterns, and most agents run both at once:

• Pre-filter (top-K). Before each model call, catalog.search(query, top_k) returns the few tools most relevant to the user's message. Only those enter the tool list.
• Dynamic capability search. The agent carries search_capabilities and invoke_tool and reaches the rest of the catalog itself when the pre-filtered set is not enough. See Progressive disclosure.

Whichever path a query arrives through, the ranking below is the same.

BM25 over a schema-aware projection

BM25 is the lexical algorithm behind most search engines. Ratel runs it in-process: the index is rebuilt from the registered definitions at each search call, and an empty catalog simply returns no hits.

The scorer is tuned for short documents like tool definitions (k1 = 0.9, b = 0.4; fixed, not a knob). Ties break by tool id, so the same query over the same catalog returns the same hits in the same order on any machine.

What gets indexed

BM25 does not rank raw JSON. Each tool is flattened into a schema-aware text projection (ADR 0004):

1. The tool name, whole and identifier-split: search_files also indexes "search files". Hyphens are not split, so prefer snake_case or camelCase over kebab-case ids.
2. The description, verbatim.
3. Every property of the input and output schemas: the property name (whole and split), its description, and each string value of its enum, recursing through nested objects and array items. Non-string enum values are skipped.

For each skill, the projection is the name, each tag, and the description. The body is excluded by design — a 15 KB runbook would drown the description's term weights.

Structural JSON Schema tokens (type, required, braces) are never indexed. The emitted text is tokenized at index and query time (lowercased, stemmed, stop words removed), so matching is not verbatim.

The consequence is the point: your vocabulary is the recall mechanism. The projection is a contract — every retrieval method ranks exactly this text, and changing the flattening is treated as a breaking change.

Write definitions the index can find

Every definition serves two readers. The index decides whether it surfaces; the model reads the description to decide when to call and the schema to decide how.

• Say what it does and when to use it. One sentence for the what, then Use when .... The "when" clause is what separates a tool from its sibling in a shared result set.
• Use the caller's vocabulary. Stemming matches "refunded" to "refund"; nothing maps "monetary adjustments" to "money back". Add distinct trigger phrasings, not repetitions — term-frequency scoring saturates fast.
• Avoid near-duplicate descriptions. Two tools described in the same words split the score and the model's confidence. Merge them, or rewrite each "when to use" until they no longer overlap.
• Name parameters and enum values like they will be searched. arg1 is invisible to the index; a status enum of ["pending", "shipped", "refunded"] adds query terms and constrains the call.
• Keep schemas tight for the model. Structure is not indexed, but it stops malformed calls: type every property, list required, set additionalProperties: false.
• Tag your skills. The body is not indexed, so tags and the description are where an intent prompt has to land. Fold in category labels ("billing") and raw task phrases ("issue a refund").

Anti-patterns:

• Paraphrase chains. "Searches files. Finds files. Locates files." Saturated terms, still no "when to use".
• Marketing tone. No caller types "seamless" or "powerful". Concrete verbs and domain nouns only.
• One-line anemia. "Refund tool." Nothing to match, nothing for the model to decide with.
• Structural noise. Restating the schema in prose, or documenting internals (transport, timeouts, id formats).

Spot-check with catalog.search using realistic queries — the user's phrasing, not the tool's name. The intended tool should top each phrasing you expect to serve.

from ratel_ai import ToolCatalog

catalog = ToolCatalog()
# ...register the exact tools your agent registers in production.

for q in ["refund a customer order", "customer wants money back"]:
    hits = catalog.search(q, top_k=5)
    print(q, "->", [(h.tool_id, round(h.score, 2)) for h in hits])

import { ToolCatalog } from "@ratel-ai/sdk";

const catalog = new ToolCatalog();
// ...register the exact tools your agent registers in production.

for (const q of ["refund a customer order", "customer wants money back"]) {
  const hits = catalog.search(q, 5);
  console.log(q, "->", hits.map((h) => `${h.toolId}: ${h.score.toFixed(2)}`));
}

SkillCatalog.search works the same way for skills (hits carry skillId in TypeScript, skill_id in Python).

When keyword search misses

BM25 misses the query that shares no words with the right definition: "summarize this contract" will not surface extract_document_text unless the overlap is written in. The first fix is better definitions.

When paraphrase queries keep missing, opt into Semantic & hybrid search.

Next steps