/deepeval-otel

Endpoint and Exporter

Where to export OpenTelemetry traces and how to authenticate so they land in Confident AI's Observatory.

Endpoints

Confident AI exposes one OTLP/HTTP traces endpoint per region. There are exactly two.

When configuring an OTLP/HTTP span exporter directly, the endpoint value must include the /v1/traces suffix. When configuring through the standard OTEL_EXPORTER_OTLP_ENDPOINT environment variable, supply only the base endpoint — the SDK appends /v1/traces itself.

Choosing the Endpoint by API Key Region

Confident AI API keys are region-prefixed. Pick the endpoint from the prefix of the project's CONFIDENT_API_KEY:

Only confident_eu_… keys use the EU endpoint. When in doubt, ask the user for project region or use the default endpoint.

Authentication

Every request must carry the Confident AI API key in an HTTP header:

x-confident-api-key: <CONFIDENT_API_KEY>

Set this as a header on the OTLP exporter. Read the key from the CONFIDENT_API_KEY environment variable — never hardcode it into source.

Transport: HTTP Only

Confident AI's OTLP endpoint accepts OTLP/HTTP only — never gRPC.

• Python: use the HTTP exporter OTLPSpanExporter from opentelemetry.exporter.otlp.proto.http.trace_exporter (package opentelemetry-exporter-otlp-proto-http). Do not use the opentelemetry.exporter.otlp.proto.grpc variant.
• OpenTelemetry Collector: use the otlphttp exporter, not otlp (gRPC).
• Other SDKs: choose the OTLP/HTTP exporter (proto-http, HttpProtobuf, or the language's equivalent).

Standard OTel Environment Variables

The exporter honors standard OpenTelemetry environment variables, so the endpoint and headers can be configured without code changes:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.confident-ai.com"
export OTEL_EXPORTER_OTLP_HEADERS="x-confident-api-key=<CONFIDENT_API_KEY>"

With env-var configuration the base endpoint is given; the SDK appends /v1/traces automatically.

Exporter Wiring (Python)

The minimal path:

1. Create a TracerProvider.
2. Attach a BatchSpanProcessor wrapping an OTLP/HTTP OTLPSpanExporter pointed at <endpoint>/v1/traces with the x-confident-api-key header.
3. Register the provider as the global tracer provider.
4. Get a tracer, start spans, and set confident.* attributes on them.

import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

api_key = os.environ["CONFIDENT_API_KEY"]
endpoint = (
    "https://eu.otel.confident-ai.com"
    if api_key.startswith("confident_eu_")
    else "https://otel.confident-ai.com"
)

provider = TracerProvider()
provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint=f"{endpoint}/v1/traces",
            headers={"x-confident-api-key": api_key},
        )
    )
)
trace.set_tracer_provider(provider)
tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("my-llm-app") as span:
    span.set_attribute("confident.span.type", "agent")
    span.set_attribute("confident.trace.name", "my-llm-app")
    # ... see trace-attributes.md and span-attributes.md

See templates/confident_otel_setup.py for the full runnable version.

Other Languages

The wiring shape is identical in any OpenTelemetry SDK — only the class and package names differ. In every language:

1. Construct an OTLP/HTTP span exporter.
2. Set its endpoint/url to <region endpoint>/v1/traces.
3. Add the x-confident-api-key header.
4. Register it with a batch span processor on the tracer provider.

Then emit spans as usual and set the confident.* attributes documented in references/trace-attributes.md and references/span-attributes.md. The attribute keys are the entire contract — they are the same regardless of language.

Export Only AI Spans (Isolate From Other Instrumentation)

Most real applications run more OpenTelemetry instrumentation than just the AI code. Auto-instrumentation libraries and APM agents (Datadog, New Relic, Grafana, OpenTelemetry auto-instrumentation, etc.) emit spans for HTTP requests, database queries, cache calls, outbound network calls, and framework internals. This happens in any runtime — Node.js, Python, Java, Go, and others — not just one.

If the Confident AI exporter shares a tracer provider or processor pipeline with that instrumentation, all of those unrelated spans get shipped to Confident AI's Observatory, where they bury the AI trace in non-AI noise. Confident AI's Observatory is for AI behavior; only AI spans belong there.

Rule: the Confident AI export pipeline must carry AI spans only. Use one of two approaches.

Approach 1 — Dedicated pipeline (preferred when feasible)

Register the Confident AI exporter on a tracer provider / processor used only by the AI instrumentation, separate from the global provider that auto-instrumentation and APM agents feed. Create AI spans with that dedicated provider's tracer. Non-AI spans never reach the Confident AI exporter because they were never on its pipeline.

Approach 2 — Filter the pipeline

When AI spans and other spans unavoidably share a provider (common when an AI framework emits its spans onto the global provider), wrap the Confident AI-bound processor or exporter in a filter that forwards only AI spans and drops everything else.

Identify an AI span by any of:

• it has a confident.span.type attribute set; or
• it carries gen_ai.* semantic-convention attributes; or
• its span name matches a known AI-framework prefix (for example, the Vercel AI SDK emits spans named ai.*).

Implement the filter as either:

• a span processor that no-ops onStart/onEnd for non-AI spans, so only AI spans are handed to the underlying Confident AI processor; or
• an exporter wrapper that removes non-AI spans from each batch before calling the real OTLP exporter.

A working reference implementation is the deepeval TypeScript SDK's DeepEvalBatchFilterProcessor (a name-prefix span-processor filter) and DeepEvalExporterWrapper (an exporter wrapper) in deepeval.ts/src/integrations/ai-sdk/index.ts. Mirror that shape in whatever language and SDK the app uses.

Caveat — preserve span nesting when filtering. Dropping an intermediate non-AI span can orphan its AI child spans (their parentSpanId now points at a span that was never exported). When filtering, re-parent orphaned AI spans onto the nearest exported ancestor, or strip the dangling parent reference so the child becomes a clean root. The DeepEvalExporterWrapper above does exactly this for root spans.