前面六课讲的都是 Langfuse 的原生摄取——你用它自己的 SDK、按它自己的事件 schema 上报。可现实里,很多团队早已用 OpenTelemetry(业界可观测性标准)埋好了点。这一课讲 Langfuse 怎么张开双臂接住 OTLP:把五花八门的 OpenTelemetry span 映射成自己的 observation 模型。关键洞察是——OTel 摄取本质是一个万能适配器:前端把各种约定翻译过来,翻译完汇入和原生完全相同的合并/算钱/落盘管道。
gen_ai.* 约定、
有的是 Vercel AI SDK 的 ai.*、有的是 OpenLLMetry 的 llm.*。万能适配器每个标准都认:它挨个去试「这个字段在你那套约定里叫什么」,
一旦认出来,就把电流接进同一个插座(Langfuse 的 observation 模型)。插座后面的电路(合并、算成本、写库)只有一套——
适配器只负责「把各种插头转成统一接口」,转完之后,谁来的都一样走。
OTLP 数据从 POST /api/public/otel/v1/traces 进来。这个端点和第 12 课的原生入口性格一致——也是「鉴权 + 解析 + 入队,立即返回」,
只是解析的是 OpenTelemetry 的 span 格式(支持 protobuf 和 JSON 两种编码、支持 gzip)。解析出 resourceSpans 后,走的还是第 14 课那套S3 存本体 + 队列递指针:
OtelIngestionQueue(+次队列)→ worker 用 OtelIngestionProcessor 把 span 翻译成原生 ingestion 事件,再走第 15–17 课同一条合并/算钱/落盘管道。源码:web/.../otel/v1/traces/index.ts、OtelIngestionProcessor.ts、worker/.../otelIngestionQueue.ts。packages/shared/src/server/otel/ObservationTypeMapper.ts;值为示例):你用任意 OpenTelemetry SDK 发出的 span,其 gen_ai.* 语义属性被逐字段映射成 observation——gen_ai.request.model→model、gen_ai.usage.*→usageDetails、operation.name→type。于是不改埋点也能接入 Langfuse。// 端点:按 content-type 解析 OTLP(protobuf 或 JSON),取出 resourceSpans const resourceSpans = contentType.includes("x-protobuf") ? ExportTraceServiceRequest.decode(body).resourceSpans : JSON.parse(body).resourceSpans; return await processor.publishToOtelIngestionQueue(resourceSpans); // publishToOtelIngestionQueue:本体落 S3,指针入专用队列(同第 14 课模式) const fileKey = `…otel/${projectId}/…/${randomUUID()}.json`; await getS3EventStorageClient(bucket).uploadJson(fileKey, resourceSpans); OtelIngestionQueue.getInstance({ shardingKey: `${projectId}-${fileKey}` }) .add(QueueJobs.OtelIngestionJob, { payload: { data: { fileKey } } });
映射的核心难点在于:同一个语义(比如「这次调用用了哪个模型」),不同的埋点库会写进不同的 span 属性键。OtelIngestionProcessor 的办法是
为每个字段准备一份优先级列表,从上往下试,第一个有值的就采用。以提取模型名为例:
extractModelName 依次尝试 Langfuse 原生 → OTel gen_ai.* → Vercel ai.* → OpenLLMetry llm.* → 通用 model,第一个命中的即为模型名。usage/input/output 同理。提取出的 usage 进 provided_usage_details,再走第 16 课。源码:OtelIngestionProcessor.ts:2232-2257。| Langfuse 字段 | 依次尝试的 span 属性键(节选) | 来自哪套约定 |
|---|---|---|
| 模型名 | gen_ai.response.model · ai.model.id · gen_ai.request.model · llm.response.model · llm.model_name · model | OTel GenAI · Vercel · OpenLLMetry · 通用 |
| usage(token) | Langfuse observation.usage_details · gen_ai 的 token 计数 · Genkit 输出… | 原生 · OTel · Genkit |
| input / output | span 属性 + span 事件(gen_ai.user.message、gen_ai.choice) | OTel GenAI 语义约定 |
| observation 类型 | observationTypeMapper 据属性判定 GENERATION / SPAN / … | 映射器规则 |
举个具体的:一段用 OpenLLMetry 埋点的 OpenAI 调用,会产生一个带这些属性的 span——gen_ai.request.model="gpt-4o"、
若干 token 计数属性,外加两条 span 事件 gen_ai.user.message(用户输入)和 gen_ai.choice(模型输出)。适配器一一对上:
模型名从 gen_ai.request.model 取到、usage 从 token 属性取到、input/output 从那两条事件取到、类型被判成 GENERATION。
于是这个原本「OTel 味」的 span,就变成了一条结构完整、和你用 Langfuse SDK 上报别无二致的 generation observation。
特别值得一提的是 input/output:它们常常不在 span 的属性里,而藏在 span 的事件流中——适配器必须连 span events 也一并翻译,才能把整段对话完整还原出来。
worker 端的 OtelIngestionProcessor.processToIngestionEvents 把一批 span 翻译成 IngestionEventType[]——
正是第 13 课那套原生事件格式。接着它把结果一分为二:observation 类直接交给 IngestionService.mergeAndWrite(第 15 课),
trace 等非 observation 再走一遍 processEventBatch(第 12 课)。从这里开始,OTel 来的数据和原生 SDK 来的数据再无分别:
同样的合并、同样的算 token/成本、同样的批量落盘。
你用 Langfuse SDK,直接说它的母语——事件就是它定义的 schema,字段精确、无需猜测。这是「精确通道」。
你用 OpenTelemetry,需要翻译——适配器从异构约定里尽力推断字段。灵活但有语义损耗,是「兼容通道」。两者最终汇入同一后端。
把 worker 端 OTel 任务的处理拆开,就能看清「翻译完汇入主管道」这件事是怎么发生的:
worker 从 OtelIngestionQueue 拿到任务,照 fileKey 去 S3 下载这批 span 本体。
把 S3 里的 resourceSpans 解析成结构化的 span 列表,准备喂给适配器。
processToIngestionEvents(spans) 用优先级属性键把每个 span 映射成原生 ingestion 事件(第 13 课格式)。
按实体类型一分为二:observation 直接交 IngestionService,trace 走 processEventBatch。
从这步起,OTel 与原生共用第 15–17 课:合并、算 token/成本、ClickhouseWriter 批量落盘。
正因为映射是「尽力推断」,OTel 通道天然存在语义落差:某些埋点库没写、或用了适配器还不认的属性键,对应字段就可能缺失。 这也是为什么模型名要试六七个键、usage 还要去翻 span 事件——适配器认得越多,落差越小。但无论怎么补,OTel 的「推断」终究不如原生的「明说」精确—— 这是拥抱开放标准必然付出的代价。好在这套适配器是持续演进的:每当社区冒出一种新的埋点约定,只要往优先级列表里再加几个属性键, 老数据不受影响、新数据立刻被认得。这种「加键即兼容」的扩展方式,让 Langfuse 能跟着 OpenTelemetry GenAI 生态一起成长,而不必每次都大改一遍映射逻辑。
/api/public/otel/v1/traces),接收 OpenTelemetry span(protobuf/JSON、支持 gzip),让你无需改埋点即可接入。OtelIngestionQueue(含主/次)。OtelIngestionProcessor 是万能适配器:每个字段备一份优先级属性键列表,试遍 OTel gen_ai.*、Vercel ai.*、OpenLLMetry llm.* 等约定,第一个有值的即采用。IngestionService、trace 入 processEventBatch——之后与原生完全同路。The last six lessons covered Langfuse's native ingestion — you use its own SDK, report in its own event schema. But in reality, many teams already instrument with OpenTelemetry (the industry observability standard). This lesson covers how Langfuse opens its arms to OTLP: mapping the motley assortment of OpenTelemetry spans onto its own observation model. The key insight — OTel ingestion is essentially a universal adapter: the front end translates the various conventions, and once translated, it flows into the exact same merge / cost / persist pipeline as native.
gen_ai.* convention, some Vercel AI SDK's ai.*, some OpenLLMetry's llm.*. The universal adapter
recognizes them all: it tries, one by one, "what's this field called in your convention", and once it recognizes it, routes the current
into the same socket (Langfuse's observation model). The circuitry behind the socket (merge, cost, write) is just one —
the adapter only "turns various plugs into one interface"; after that, everyone flows the same way.
OTLP data arrives at POST /api/public/otel/v1/traces. This endpoint shares Lesson 12's native-entry character — also "auth +
parse + enqueue, return immediately" — only it parses the OpenTelemetry span format (both protobuf and JSON encodings, plus gzip). After parsing out
resourceSpans, it follows the same S3-stores-body + queue-passes-pointer pattern as Lesson 14:
OtelIngestionQueue (+ secondary) → the worker uses OtelIngestionProcessor to translate spans into native ingestion events, then runs the same Lesson 15–17 merge/cost/persist pipeline. Source: web/.../otel/v1/traces/index.ts, OtelIngestionProcessor.ts, worker/.../otelIngestionQueue.ts.packages/shared/src/server/otel/ObservationTypeMapper.ts; values illustrative): a span from any OpenTelemetry SDK has its gen_ai.* semantic attributes mapped field-by-field into an observation — gen_ai.request.model→model, gen_ai.usage.*→usageDetails, operation.name→type. So you can feed Langfuse without changing your instrumentation.// endpoint: parse OTLP by content-type (protobuf or JSON), extract resourceSpans const resourceSpans = contentType.includes("x-protobuf") ? ExportTraceServiceRequest.decode(body).resourceSpans : JSON.parse(body).resourceSpans; return await processor.publishToOtelIngestionQueue(resourceSpans); // publishToOtelIngestionQueue: body to S3, pointer to the dedicated queue (Lesson 14 pattern) const fileKey = `…otel/${projectId}/…/${randomUUID()}.json`; await getS3EventStorageClient(bucket).uploadJson(fileKey, resourceSpans); OtelIngestionQueue.getInstance({ shardingKey: `${projectId}-${fileKey}` }) .add(QueueJobs.OtelIngestionJob, { payload: { data: { fileKey } } });
The core difficulty of mapping: the same semantic (say "which model did this call use") gets written into different span attribute keys
by different instrumentation libraries. OtelIngestionProcessor's approach is to keep a priority list per field and try top
to bottom, taking the first one with a value. Take extracting the model name:
extractModelName tries in turn Langfuse-native → OTel gen_ai.* → Vercel ai.* → OpenLLMetry llm.* → generic model, taking the first hit as the model name. usage/input/output likewise. The extracted usage goes into provided_usage_details, then Lesson 16. Source: OtelIngestionProcessor.ts:2232-2257.| Langfuse field | span attribute keys tried in order (excerpt) | from which convention |
|---|---|---|
| model name | gen_ai.response.model · ai.model.id · gen_ai.request.model · llm.response.model · llm.model_name · model | OTel GenAI · Vercel · OpenLLMetry · generic |
| usage (tokens) | Langfuse observation.usage_details · gen_ai token counts · Genkit output… | native · OTel · Genkit |
| input / output | span attributes + span events (gen_ai.user.message, gen_ai.choice) | OTel GenAI semantic conventions |
| observation type | observationTypeMapper decides GENERATION / SPAN / … by attributes | mapper rules |
Concretely: an OpenAI call instrumented with OpenLLMetry produces a span with attributes like gen_ai.request.model="gpt-4o", several token
counts, plus two span events gen_ai.user.message (the user input) and gen_ai.choice (the model output). The adapter
matches them one by one: model name from gen_ai.request.model, usage from the token attributes, input/output from those two events, type judged
as GENERATION. So that originally "OTel-flavored" span becomes a fully-structured generation observation, indistinguishable
from one reported via the Langfuse SDK. Note input/output especially: they often aren't in the span's attributes but hide in the span's event
stream — the adapter must translate span events too to fully reconstruct the conversation.
On the worker, OtelIngestionProcessor.processToIngestionEvents translates a batch of spans into IngestionEventType[] —
exactly Lesson 13's native event format. It then splits the result in two: observation kinds go straight to
IngestionService.mergeAndWrite (Lesson 15), trace and other non-observations run through processEventBatch (Lesson
12). From here on, OTel-sourced data and native-SDK data are indistinguishable: the same merge, the same token/cost calculation, the same
batched persistence.
You use the Langfuse SDK and speak its mother tongue directly — events are exactly its schema, fields precise, no guessing. The "precise channel".
You use OpenTelemetry and need translation — the adapter infers fields from heterogeneous conventions. Flexible but with semantic loss, the "compatibility channel". Both converge into one backend.
Unpack the worker's OTel job and you see exactly how "translate then converge into the main pipeline" happens:
The worker pulls a job from OtelIngestionQueue and downloads this batch of span bodies from S3 by fileKey.
Parse the S3 resourceSpans into a structured span list, ready to feed the adapter.
processToIngestionEvents(spans) maps each span into a native ingestion event (Lesson 13's format) via priority attribute keys.
By entity type: observation straight to IngestionService, trace via processEventBatch.
From here OTel shares Lessons 15–17 with native: merge, token/cost, ClickhouseWriter batched persistence.
Precisely because the mapping is "best-effort inference", the OTel channel inherently has a semantic gap: if a library didn't write a key, or used one the adapter doesn't yet recognize, that field may be missing. That's why the model name tries six or seven keys and usage even digs into span events — the more the adapter recognizes, the smaller the gap. But however much it backfills, OTel's "inference" is never as precise as native's "explicit statement" — the price of embracing an open standard. Happily the adapter keeps evolving: whenever the community spawns a new convention, just add a few attribute keys to the priority list — old data unaffected, new data instantly recognized. This "add a key, gain compatibility" extensibility lets Langfuse grow with the OpenTelemetry GenAI ecosystem without rewriting the mapping each time.
/api/public/otel/v1/traces) accepting OpenTelemetry spans (protobuf/JSON, gzip), letting you onboard without changing instrumentation.OtelIngestionQueue (with primary/secondary).OtelIngestionProcessor is a universal adapter: a priority attribute-key list per field, trying OTel gen_ai.*, Vercel ai.*, OpenLLMetry llm.* and more, taking the first present.IngestionService, traces to processEventBatch — thereafter identical to native.