Telemetry

Ratel emits telemetry on two deliberately separate streams (ADR 0007):

• Local trace events, owned by the Rust core. Structured events (search, invoke_start, upstream_error, ...) recorded into a sink you configure; this stream feeds the offline inspector, Ratel Local statusline and savings reporting, and future rerankers. Its reliability contract is query-log semantics — best-effort, sampleable, lossy on backpressure: losing an event is acceptable, corrupting a catalog is not.
• OpenTelemetry spans. Remote telemetry is OpenTelemetry: gen_ai.* spans per a pinned semconv baseline plus a ratel.* overlay, exported as stock OTLP (http/protobuf + Bearer auth). This is the stream for Ratel Cloud, Langfuse, or your own collector.

Both are off by default: the local sink is noop until you pass one, and spans are no-ops until an OpenTelemetry provider exists. Pick your path:

• Local visibility, no network → configure a trace sink.
• No OpenTelemetry in your stack yet → call configure_telemetry / configureTelemetry once at startup and spans flow to Ratel Cloud.
• Already running OpenTelemetry (Langfuse, the Vercel AI SDK, your own collector) → skip configureTelemetry; your provider already receives the spans. Add the Ratel span processor to dual-export the Ratel cut.

Local trace sinks

Pass trace to the ToolCatalog (or SkillCatalog) constructor and the core records every search, invoke, capability-tool, upstream-MCP, and auth event into the sink.

from ratel_ai import ToolCatalog, TraceSinkConfig

catalog = ToolCatalog(
    trace=TraceSinkConfig(kind="jsonl", session_id="session-1", path="/tmp/ratel.jsonl"),
)
# Every catalog.invoke, capability-tool call, and register_mcp_server call now
# appends one JSON line per event to /tmp/ratel.jsonl.

import { ToolCatalog } from "@ratel-ai/sdk";

const catalog = new ToolCatalog({
  trace: { kind: "jsonl", sessionId: "session-1", path: "/tmp/ratel.jsonl" },
});
// Every catalog.invoke, capability-tool call, and registerMcpServer call now appends one JSON line per event to /tmp/ratel.jsonl.

Every recorded event lands wrapped in an envelope: { "v": 1, "ts": <unix epoch ms>, "session_id": "...", "type": "search", ...event fields }, with the event type tagged in snake_case.

• Draining the memory sink. catalog.drain_trace_events() (Python) / catalog.drainTraceEvents() (TypeScript) returns the buffered events and removes them. It returns [] unless the active sink is memory; the jsonl and noop sinks have no drain, read the file instead.
• JSONL file behavior. The sink creates missing parent directories, opens the file append-only with mode 0600 on Unix, and flushes after every line. Writes are best-effort: a serialization or write failure is swallowed rather than crashing the agent loop.
• Origin tagging. Searches the model makes through search_capabilities are tagged origin: "agent"; direct catalog.search(query, k) calls default to "direct", overridable with a third argument.

Event types

The event set is defined once, in the Rust core, so every SDK emits the same shapes. The full type vocabulary:

A single agent call often records several of these: search_capabilities produces both a core-level search (or skill_search) and a gateway_search, and invoke_tool produces invoke_start / invoke_end (or invoke_error) plus gateway_invoke / gateway_error, with upstream_invoke / upstream_error on top for proxied MCP tools. The four auth_* events come only from Ratel Local OAuth upstreams, which embeds the SDK and records into the same stream; the SDKs never emit them.

The OpenTelemetry span vocabulary

Independently of the local sink, the SDK emits OpenTelemetry spans around the same funnel: gen_ai.* attributes pinned to semconv v1.42.0, plus the Ratel-owned ratel.* overlay. The local event and the span are two independent channels around the same call.

Tool invocations deliberately reuse execute_tool, the standard gen_ai.operation.name value, instead of a bespoke ratel.invoke span, so a generic OTel backend already understands them. Errors use the standard span status ERROR plus the exception event, not a bespoke attribute. The SDK does not emit the LLM-call spans (chat <model>): those come from your LLM instrumentation, and Ratel's spans join the same trace next to them.

No-op until a provider

Instrumentation is free until you opt in, in both languages:

• TypeScript: the SDK's only OpenTelemetry dependency is @opentelemetry/api, never the OTel SDK. Until a provider is registered, every span is a no-op NonRecordingSpan, and the local trace stream is untouched.
• Python: the base ratel-ai install has zero dependencies. If opentelemetry or ratel_ai_telemetry is missing, every telemetry helper is a straight pass-through, zero overhead, no spans. With them installed, spans go to whatever provider is registered.

Greenfield: ship spans to Ratel Cloud

No OpenTelemetry in your stack yet? One call at startup registers a global tracer provider with a batch OTLP exporter pointed at Ratel Cloud.

pip install 'ratel-ai[otlp]'

import os
from ratel_ai import configure_telemetry

handle = configure_telemetry(
    endpoint=os.environ.get("RATEL_URL", "https://cloud.ratel.sh/v1/traces"),
    api_key=os.environ["RATEL_API_KEY"],
)
# ... run your agent: every ratel.* / execute_tool span now exports ...
handle.shutdown()  # flush the exporter on exit (handle.force_flush() mid-run)

npm install @ratel-ai/telemetry-otlp

import { configureTelemetry } from "@ratel-ai/sdk";

const handle = await configureTelemetry({
  endpoint: process.env.RATEL_URL ?? "https://cloud.ratel.sh/v1/traces",
  apiKey: process.env.RATEL_API_KEY,
});
// ... run your agent: every ratel.* / execute_tool span now exports ...
await handle.shutdown(); // flushes and stops the exporter

Endpoint and auth resolve identically in both languages:

• The endpoint option wins; otherwise the RATEL_URL env var. Either way it is the full OTLP traces URL including /v1/traces, for example https://cloud.ratel.sh/v1/traces. With neither set, the call fails before anything registers.
• The API key is sent as Authorization: Bearer <key> on top of any extra headers. When omitted it falls back to the RATEL_API_KEY env var (unless you already set an Authorization header explicitly).
• service_name / serviceName defaults to "ratel".

Runnable examples live in the repo: examples/telemetry-ts and examples/telemetry-python emit a sample ratel.search + execute_tool trace to a console exporter, no collector or API key needed, and switch to a real OTLP export when RATEL_URL is set.

Already on OpenTelemetry: dual-export

OpenTelemetry's coexistence model is one provider, many span processors. If Langfuse, the Vercel AI SDK, or your own collector already owns the provider, the SDK's spans are already flowing to it, so skip configureTelemetry and add the Ratel span processor to send a copy to Ratel Cloud:

import os

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from ratel_ai_telemetry.otlp import ratel_span_processor

provider = trace.get_tracer_provider()  # the provider your existing setup registered
if not isinstance(provider, TracerProvider):
    raise RuntimeError("register your OpenTelemetry SDK provider before adding processors")
provider.add_span_processor(
    ratel_span_processor(
        endpoint=os.environ.get("RATEL_URL", "https://cloud.ratel.sh/v1/traces"),
        api_key=os.environ["RATEL_API_KEY"],
    )
)

import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { ratelSpanProcessor } from "@ratel-ai/telemetry-otlp";

const provider = new NodeTracerProvider({
  spanProcessors: [
    new LangfuseSpanProcessor(),                                // Langfuse keeps every span
    ratelSpanProcessor({                                        // Ratel takes gen_ai.*/ratel.* only
      endpoint: process.env.RATEL_URL ?? "https://cloud.ratel.sh/v1/traces",
      apiKey: process.env.RATEL_API_KEY,
    }),
  ],
});
provider.register();

The processor batches spans to the Ratel OTLP exporter and forwards only those passing its filter:

• The default filter forwards a span if its name starts with ratel. or any attribute key starts with gen_ai. or ratel., so the framework's ai.* wrapper noise stays out.
• Pass spanFilter: () => true (TypeScript) / span_filter=lambda _s: True (Python) to forward everything. Do that, or tail-sample, when full-trace fidelity matters — per-span filtering can orphan the AI SDK's ai.* wrapper span from its gen_ai.* child.
• The exporter carries no resource; your provider keeps ownership of service.name.

Content capture

Message and tool content never leaves the process by default. Set the mode in code — configure_telemetry(capture_content=...) / configureTelemetry({ captureContent }) — or through the ecosystem instrumentation flag OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, the fallback when nothing is set in code; parsing is trimmed and case-insensitive.

The gated payloads in the SDK are ratel.search.query on ratel.search and gen_ai.tool.call.arguments / gen_ai.tool.call.result on execute_tool. They appear only when the mode is a span mode (SPAN_ONLY or SPAN_AND_EVENT). EVENT_ONLY does not put content on spans, and since the 0.4.0 SDKs emit no content-bearing span events, it captures nothing from the SDK today: the gen_ai.client.inference.operation.details event is the convention for message content on your LLM instrumentation's spans, not something the Ratel SDK emits.

Next steps