🪢 Langfuse 图解教程Langfuse Visual Guide 第三部分 · 摄取链路Part 3 · The Ingestion Path 18 / 55
第三部分 · 摄取链路Part 3 · The Ingestion Path

OpenTelemetry 摄取OpenTelemetry ingestion

🧭 读前Before you start
先读 第12课 的原生摄取 API,本课讲 OpenTelemetry 这条并行入口。
Read Lesson 12 on the native ingestion API; this lesson covers OpenTelemetry as a parallel entry point.

前面六课讲的都是 Langfuse 的原生摄取——你用它自己的 SDK、按它自己的事件 schema 上报。可现实里,很多团队早已用 OpenTelemetry(业界可观测性标准)埋好了点。这一课讲 Langfuse 怎么张开双臂接住 OTLP:把五花八门的 OpenTelemetry span 映射成自己的 observation 模型。关键洞察是——OTel 摄取本质是一个万能适配器:前端把各种约定翻译过来,翻译完汇入和原生完全相同的合并/算钱/落盘管道。

🔌 生活类比
把 OTel 端点想成一个万能转换插头。你的设备(应用)来自世界各地,插头标准各不相同——有的是 OpenTelemetry 的 gen_ai.* 约定、 有的是 Vercel AI SDK 的 ai.*、有的是 OpenLLMetry 的 llm.*。万能适配器每个标准都认:它挨个去试「这个字段在你那套约定里叫什么」, 一旦认出来,就把电流接进同一个插座(Langfuse 的 observation 模型)。插座后面的电路(合并、算成本、写库)只有一套—— 适配器只负责「把各种插头转成统一接口」,转完之后,谁来的都一样走。

OTLP 的一条路:解析 → 落 S3 → 专用队列 → 适配器

OTLP 数据从 POST /api/public/otel/v1/traces 进来。这个端点和第 12 课的原生入口性格一致——也是「鉴权 + 解析 + 入队,立即返回」, 只是解析的是 OpenTelemetry 的 span 格式(支持 protobuf 和 JSON 两种编码、支持 gzip)。解析出 resourceSpans 后,走的还是第 14 课那套S3 存本体 + 队列递指针:

OTLP 摄取:前端适配,后端归一 应用 + OTel SDKPOST otel/v1/traces 端点解析protobuf/JSON·gzip→ resourceSpans S3:span 本体otel/<proj>/…json OtelIngestionQueue指针 fileKey(+ 次队列) OtelIngestionProcessorspan→ingestion 事件 同一管道第 15–17 课合并·算钱·写库 OTel 特异的只有「映射」这一段 ▲;其后与原生摄取共用一套后端 processToIngestionEvents → IngestionEventType[](第 13 课的事件格式)observation → IngestionService.mergeAndWrite;trace → processEventBatch
边缘适配、核心归一:端点解析 OTLP(protobuf/JSON)→ span 本体落 S3、指针入 OtelIngestionQueue(+次队列)→ worker 用 OtelIngestionProcessor 把 span 翻译成原生 ingestion 事件,再走第 15–17 课同一条合并/算钱/落盘管道。源码:web/.../otel/v1/traces/index.ts、OtelIngestionProcessor.ts、worker/.../otelIngestionQueue.ts。
示例:一个 OTLP span 映射成 observation OTLP span(标准遥测) name: "chat completion" gen_ai.operation.name: "chat" gen_ai.request.model: "gpt-4o" gen_ai.usage.input_tokens: 512 gen_ai.usage.output_tokens: 88 startTimeUnixNano: … endTimeUnixNano: … 任意 OTel SDK 都能发 Langfuse observation type: GENERATION ← operation.name name: "chat completion" model: "gpt-4o" ← request.model usageDetails: {input:512, output:88} startTime/endTime ← UnixNano 无需改 SDK,OTel 直接进来
OTel 是第二条入口(属性名对齐 packages/shared/src/server/otel/ObservationTypeMapper.ts;值为示例):你用任意 OpenTelemetry SDK 发出的 span,其 gen_ai.* 语义属性被逐字段映射成 observation——gen_ai.request.model→model、gen_ai.usage.*→usageDetails、operation.name→type。于是不改埋点也能接入 Langfuse。
web/src/pages/api/public/otel/v1/traces/index.ts & OtelIngestionProcessor.ts端点+发布
// 端点:按 content-type 解析 OTLP(protobuf 或 JSON),取出 resourceSpans
const resourceSpans = contentType.includes("x-protobuf")
  ? ExportTraceServiceRequest.decode(body).resourceSpans
  : JSON.parse(body).resourceSpans;
return await processor.publishToOtelIngestionQueue(resourceSpans);

// publishToOtelIngestionQueue:本体落 S3,指针入专用队列(同第 14 课模式)
const fileKey = `…otel/${projectId}/…/${randomUUID()}.json`;
await getS3EventStorageClient(bucket).uploadJson(fileKey, resourceSpans);
OtelIngestionQueue.getInstance({ shardingKey: `${projectId}-${fileKey}` })
  .add(QueueJobs.OtelIngestionJob, { payload: { data: { fileKey } } });

万能适配器:一个字段,试遍各种约定

映射的核心难点在于:同一个语义(比如「这次调用用了哪个模型」),不同的埋点库会写进不同的 span 属性键。OtelIngestionProcessor 的办法是 为每个字段准备一份优先级列表,从上往下试,第一个有值的就采用。以提取模型名为例:

extractModelName:按优先级试遍各家约定 ① langfuse.observation.model(原生) ② gen_ai.response.model(OTel 标准) ③ ai.model.id(Vercel AI SDK) ④ gen_ai.request.model ⑤ llm.response.model / llm.model_name ⑥ model(兜底通用键) ▶ 第一个有值的→ observation.model ▶ 进 provided_*交第 16 课匹配价格/算钱 usage、input/output、observation 类型……每个字段都有这样一份「试遍各家」的优先级列表
优先级回退,认遍各家约定:extractModelName 依次尝试 Langfuse 原生 → OTel gen_ai.* → Vercel ai.* → OpenLLMetry llm.* → 通用 model,第一个命中的即为模型名。usage/input/output 同理。提取出的 usage 进 provided_usage_details,再走第 16 课。源码:OtelIngestionProcessor.ts:2232-2257。
Langfuse 字段依次尝试的 span 属性键(节选)来自哪套约定
模型名gen_ai.response.model · ai.model.id · gen_ai.request.model · llm.response.model · llm.model_name · modelOTel GenAI · Vercel · OpenLLMetry · 通用
usage(token)Langfuse observation.usage_details · gen_ai 的 token 计数 · Genkit 输出…原生 · OTel · Genkit
input / outputspan 属性 + span 事件(gen_ai.user.message、gen_ai.choice)OTel GenAI 语义约定
observation 类型observationTypeMapper 据属性判定 GENERATION / SPAN / …映射器规则

举个具体的:一段用 OpenLLMetry 埋点的 OpenAI 调用,会产生一个带这些属性的 span——gen_ai.request.model="gpt-4o"、 若干 token 计数属性,外加两条 span 事件 gen_ai.user.message(用户输入)和 gen_ai.choice(模型输出)。适配器一一对上: 模型名从 gen_ai.request.model 取到、usage 从 token 属性取到、input/output 从那两条事件取到、类型被判成 GENERATION。 于是这个原本「OTel 味」的 span,就变成了一条结构完整、和你用 Langfuse SDK 上报别无二致的 generation observation。 特别值得一提的是 input/output:它们常常不在 span 的属性里,而藏在 span 的事件流中——适配器必须连 span events 也一并翻译,才能把整段对话完整还原出来。

映射完,殊途同归

worker 端的 OtelIngestionProcessor.processToIngestionEvents 把一批 span 翻译成 IngestionEventType[]—— 正是第 13 课那套原生事件格式。接着它把结果一分为二:observation 类直接交给 IngestionService.mergeAndWrite(第 15 课), trace 等非 observation 再走一遍 processEventBatch(第 12 课)。从这里开始,OTel 来的数据和原生 SDK 来的数据再无分别: 同样的合并、同样的算 token/成本、同样的批量落盘。

原生 SDK 摄取 精确通道(L12–13) OTel span 兼容通道(本课) 适配器翻译 processToIngestionEvents IngestionEventType[] L13 原生事件格式 殊途同归 · 同一后端 mergeAndWrite · 算 token/成本 ClickhouseWriter 批写(L15–17) 从这一步起,OTel 与原生 SDK 再无分别——同样合并、同样算成本、同样批量落盘

🟦 原生摄取(第 12–13 课)

你用 Langfuse SDK,直接说它的母语——事件就是它定义的 schema,字段精确、无需猜测。这是「精确通道」。

🔌 OTel 摄取(本课)

你用 OpenTelemetry,需要翻译——适配器从异构约定里尽力推断字段。灵活但有语义损耗,是「兼容通道」。两者最终汇入同一后端。

把 worker 端 OTel 任务的处理拆开,就能看清「翻译完汇入主管道」这件事是怎么发生的:

1

取下 OTel 任务

worker 从 OtelIngestionQueue 拿到任务,照 fileKey 去 S3 下载这批 span 本体。

2

解析 span

把 S3 里的 resourceSpans 解析成结构化的 span 列表,准备喂给适配器。

3

适配器翻译

processToIngestionEvents(spans) 用优先级属性键把每个 span 映射成原生 ingestion 事件(第 13 课格式)。

4

分流

按实体类型一分为二:observation 直接交 IngestionService,trace 走 processEventBatch。

5

汇入主管道

从这步起,OTel 与原生共用第 15–17 课:合并、算 token/成本、ClickhouseWriter 批量落盘。

正因为映射是「尽力推断」,OTel 通道天然存在语义落差:某些埋点库没写、或用了适配器还不认的属性键,对应字段就可能缺失。 这也是为什么模型名要试六七个键、usage 还要去翻 span 事件——适配器认得越多,落差越小。但无论怎么补,OTel 的「推断」终究不如原生的「明说」精确—— 这是拥抱开放标准必然付出的代价。好在这套适配器是持续演进的:每当社区冒出一种新的埋点约定,只要往优先级列表里再加几个属性键, 老数据不受影响、新数据立刻被认得。这种「加键即兼容」的扩展方式,让 Langfuse 能跟着 OpenTelemetry GenAI 生态一起成长,而不必每次都大改一遍映射逻辑。

🎯 设计取舍
既然原生通道更精确,为什么还要费力支持 OpenTelemetry? 因为要去用户所在的地方接住他们。OpenTelemetry 是可观测性的事实标准, 无数团队的应用早已用它埋好了点。如果 Langfuse 只认自己的 SDK,就等于要求用户为了它重新埋一遍点——这个门槛会劝退一大批人。 支持 OTLP,意味着「你不用改一行埋点,把 OTel 数据指过来就能用」。代价确实存在:一个要不断追各家约定、满是优先级回退的适配器, 以及推断带来的语义损耗。但这笔账算得过来——因为适配器只在最前端,翻译完就汇入和原生共享的核心(合并、算钱、落盘只有一套)。 把脏活、特例都收敛到边缘的一层适配器,让核心保持纯净统一——这正是「适配器模式」最经典的用法。
🎯 本课要点
  • Langfuse 提供 OTLP 端点(/api/public/otel/v1/traces),接收 OpenTelemetry span(protobuf/JSON、支持 gzip),让你无需改埋点即可接入。
  • OTLP 走和第 14 课相同的 S3 + 队列模式:span 本体落 S3,指针入专用 OtelIngestionQueue(含主/次)。
  • OtelIngestionProcessor 是万能适配器:每个字段备一份优先级属性键列表,试遍 OTel gen_ai.*、Vercel ai.*、OpenLLMetry llm.* 等约定,第一个有值的即采用。
  • 映射产出原生 ingestion 事件(第 13 课格式),observation 入 IngestionService、trace 入 processEventBatch——之后与原生完全同路。
  • 取舍:OTel 是「兼容通道」,靠推断、有语义落差;原生是「精确通道」。把适配特例收敛到边缘,核心保持统一——经典适配器模式。

🧪 自测 · 想一想为什么这么设计

1. OTLP 数据从 /otel/v1/traces 进来后,OtelIngestionProcessor 把 span 翻译成什么?之后又如何处理?
  1. 翻译成原生的 IngestionEventType[](第 13 课事件格式):observation 交 IngestionService.mergeAndWrite,trace 走 processEventBatch——从此与原生 SDK 数据完全同路(合并/算钱/落盘共用一套)
  2. 直接写进一张专门的 otel 表,不经过合并
  3. 转成 SQL 语句执行
  4. 原样存进 S3 就结束了
看答案与解析 点击展开
答案:A。OTel 摄取本质是“边缘适配、核心归一”。OtelIngestionProcessor.processToIngestionEvents 把 span 映射成原生事件格式,按实体类型分流后汇入第 15–17 课同一条管道。OTel 特异的只有最前端的映射,后端与原生完全共享。
2. 同一个“模型名”,OpenTelemetry 写在 gen_ai.response.model、Vercel AI SDK 写在 ai.model.id、OpenLLMetry 写在 llm.model_name。适配器怎么应对这种混乱?
  1. 随机选一个键
  2. 要求所有用户统一改用一种约定
  3. 为每个字段准备一份优先级属性键列表,从上往下试,第一个有值的就采用;新约定出现只需往列表里加几个键,老数据不受影响、新数据立刻被认得
  4. 只认 OpenTelemetry 官方约定,其他一律丢弃
看答案与解析 点击展开
答案:C。extractModelName 依次尝试 langfuse 原生 → gen_ai.response.model → ai.model.id → gen_ai.request.model → llm.* → 通用 model,第一个命中即用。usage、input/output 同理。这种“加键即兼容”的优先级回退,让 Langfuse 不必强迫用户改约定,也能跟着生态演进。
3. 原生摄取比 OTel 更精确,为什么 Langfuse 还要支持 OpenTelemetry?这个取舍的本质是什么?
  1. 纯粹为了营销噱头
  2. 因为 OTel 比原生更快
  3. 为了“去用户所在的地方接住他们”:OTel 是事实标准,无数应用早已埋好点;支持 OTLP 意味着用户不用改一行埋点就能接入。代价是一个满是回退的适配器+推断的语义损耗,但适配器只在最前端,翻完即汇入共享核心
  4. 因为原生 SDK 即将废弃
看答案与解析 点击展开
答案:C。这是经典的适配器模式:把脏活、特例(追各家约定、处理语义落差)收敛到边缘的一层适配器,让核心(合并/算钱/落盘)保持纯净统一。降低接入门槛换来更广的用户覆盖,而复杂度被限制在最前端一层,不污染后端。
💭 发散思考(没有标准答案,动手或动脑想想)
  • “把适配各种外部格式的脏活收敛到边缘一层适配器,核心只认一种统一格式”——这条原则你在自己的系统里能用上吗?如果要同时支持多种输入协议/数据源,你会在哪里画那条“适配器 vs 核心”的边界?画错了会怎样?

The last six lessons covered Langfuse's native ingestion — you use its own SDK, report in its own event schema. But in reality, many teams already instrument with OpenTelemetry (the industry observability standard). This lesson covers how Langfuse opens its arms to OTLP: mapping the motley assortment of OpenTelemetry spans onto its own observation model. The key insight — OTel ingestion is essentially a universal adapter: the front end translates the various conventions, and once translated, it flows into the exact same merge / cost / persist pipeline as native.

🔌 Analogy
Think of the OTel endpoint as a universal travel adapter. Your devices (apps) come from all over, with different plug standards — some OpenTelemetry's gen_ai.* convention, some Vercel AI SDK's ai.*, some OpenLLMetry's llm.*. The universal adapter recognizes them all: it tries, one by one, "what's this field called in your convention", and once it recognizes it, routes the current into the same socket (Langfuse's observation model). The circuitry behind the socket (merge, cost, write) is just one — the adapter only "turns various plugs into one interface"; after that, everyone flows the same way.

OTLP's path: parse → land in S3 → dedicated queue → adapter

OTLP data arrives at POST /api/public/otel/v1/traces. This endpoint shares Lesson 12's native-entry character — also "auth + parse + enqueue, return immediately" — only it parses the OpenTelemetry span format (both protobuf and JSON encodings, plus gzip). After parsing out resourceSpans, it follows the same S3-stores-body + queue-passes-pointer pattern as Lesson 14:

OTLP ingestion: adapt at the edge, unify at the core app + OTel SDKPOST otel/v1/traces endpoint parseprotobuf/JSON·gzip→ resourceSpans S3: span bodyotel/<proj>/…json OtelIngestionQueuepointer fileKey (+ secondary) OtelIngestionProcessorspan→ingestion event same pipelineLessons 15–17merge·cost·write Only the "mapping" stretch ▲ is OTel-specific; everything after is shared with native processToIngestionEvents → IngestionEventType[] (Lesson 13's event format)observation → IngestionService.mergeAndWrite; trace → processEventBatch
Adapt at the edge, unify at the core: the endpoint parses OTLP (protobuf/JSON) → span body to S3, pointer to OtelIngestionQueue (+ secondary) → the worker uses OtelIngestionProcessor to translate spans into native ingestion events, then runs the same Lesson 15–17 merge/cost/persist pipeline. Source: web/.../otel/v1/traces/index.ts, OtelIngestionProcessor.ts, worker/.../otelIngestionQueue.ts.
Example: an OTLP span mapped to an observation OTLP span (standard telemetry) name: "chat completion" gen_ai.operation.name: "chat" gen_ai.request.model: "gpt-4o" gen_ai.usage.input_tokens: 512 gen_ai.usage.output_tokens: 88 startTimeUnixNano: … endTimeUnixNano: … any OTel SDK can send Langfuse observation type: GENERATION ← operation.name name: "chat completion" model: "gpt-4o" ← request.model usageDetails: {input:512, output:88} startTime/endTime ← UnixNano no SDK change; OTel comes straight in
OTel is a second entry point (attribute names per packages/shared/src/server/otel/ObservationTypeMapper.ts; values illustrative): a span from any OpenTelemetry SDK has its gen_ai.* semantic attributes mapped field-by-field into an observation — gen_ai.request.model→model, gen_ai.usage.*→usageDetails, operation.name→type. So you can feed Langfuse without changing your instrumentation.
web/src/pages/api/public/otel/v1/traces/index.ts & OtelIngestionProcessor.tsendpoint+publish
// endpoint: parse OTLP by content-type (protobuf or JSON), extract resourceSpans
const resourceSpans = contentType.includes("x-protobuf")
  ? ExportTraceServiceRequest.decode(body).resourceSpans
  : JSON.parse(body).resourceSpans;
return await processor.publishToOtelIngestionQueue(resourceSpans);

// publishToOtelIngestionQueue: body to S3, pointer to the dedicated queue (Lesson 14 pattern)
const fileKey = `…otel/${projectId}/…/${randomUUID()}.json`;
await getS3EventStorageClient(bucket).uploadJson(fileKey, resourceSpans);
OtelIngestionQueue.getInstance({ shardingKey: `${projectId}-${fileKey}` })
  .add(QueueJobs.OtelIngestionJob, { payload: { data: { fileKey } } });

The universal adapter: one field, try every convention

The core difficulty of mapping: the same semantic (say "which model did this call use") gets written into different span attribute keys by different instrumentation libraries. OtelIngestionProcessor's approach is to keep a priority list per field and try top to bottom, taking the first one with a value. Take extracting the model name:

extractModelName: try each convention in priority order ① langfuse.observation.model (native) ② gen_ai.response.model (OTel standard) ③ ai.model.id (Vercel AI SDK) ④ gen_ai.request.model ⑤ llm.response.model / llm.model_name ⑥ model (generic fallback) ▶ first one present→ observation.model ▶ into provided_*handed to Lesson 16price/cost usage, input/output, observation type… each field has such a "try everyone" priority list
Priority fallback, recognizing every convention: extractModelName tries in turn Langfuse-native → OTel gen_ai.* → Vercel ai.* → OpenLLMetry llm.* → generic model, taking the first hit as the model name. usage/input/output likewise. The extracted usage goes into provided_usage_details, then Lesson 16. Source: OtelIngestionProcessor.ts:2232-2257.
Langfuse fieldspan attribute keys tried in order (excerpt)from which convention
model namegen_ai.response.model · ai.model.id · gen_ai.request.model · llm.response.model · llm.model_name · modelOTel GenAI · Vercel · OpenLLMetry · generic
usage (tokens)Langfuse observation.usage_details · gen_ai token counts · Genkit output…native · OTel · Genkit
input / outputspan attributes + span events (gen_ai.user.message, gen_ai.choice)OTel GenAI semantic conventions
observation typeobservationTypeMapper decides GENERATION / SPAN / … by attributesmapper rules

Concretely: an OpenAI call instrumented with OpenLLMetry produces a span with attributes like gen_ai.request.model="gpt-4o", several token counts, plus two span events gen_ai.user.message (the user input) and gen_ai.choice (the model output). The adapter matches them one by one: model name from gen_ai.request.model, usage from the token attributes, input/output from those two events, type judged as GENERATION. So that originally "OTel-flavored" span becomes a fully-structured generation observation, indistinguishable from one reported via the Langfuse SDK. Note input/output especially: they often aren't in the span's attributes but hide in the span's event stream — the adapter must translate span events too to fully reconstruct the conversation.

Once mapped, all roads converge

On the worker, OtelIngestionProcessor.processToIngestionEvents translates a batch of spans into IngestionEventType[] — exactly Lesson 13's native event format. It then splits the result in two: observation kinds go straight to IngestionService.mergeAndWrite (Lesson 15), trace and other non-observations run through processEventBatch (Lesson 12). From here on, OTel-sourced data and native-SDK data are indistinguishable: the same merge, the same token/cost calculation, the same batched persistence.

native SDK ingestion precise channel (L12–13) OTel span compat channel (here) adapter translates processToIngestionEvents IngestionEventType[] L13 native event format converge · one backend mergeAndWrite · token/cost ClickhouseWriter batch (L15–17) from here on, OTel and native SDK are indistinguishable — same merge, same cost, same batched persistence

🟦 native ingestion (L12–13)

You use the Langfuse SDK and speak its mother tongue directly — events are exactly its schema, fields precise, no guessing. The "precise channel".

🔌 OTel ingestion (this lesson)

You use OpenTelemetry and need translation — the adapter infers fields from heterogeneous conventions. Flexible but with semantic loss, the "compatibility channel". Both converge into one backend.

Unpack the worker's OTel job and you see exactly how "translate then converge into the main pipeline" happens:

1

take the OTel job

The worker pulls a job from OtelIngestionQueue and downloads this batch of span bodies from S3 by fileKey.

2

parse spans

Parse the S3 resourceSpans into a structured span list, ready to feed the adapter.

3

adapter translates

processToIngestionEvents(spans) maps each span into a native ingestion event (Lesson 13's format) via priority attribute keys.

4

split

By entity type: observation straight to IngestionService, trace via processEventBatch.

5

converge into the main pipeline

From here OTel shares Lessons 15–17 with native: merge, token/cost, ClickhouseWriter batched persistence.

Precisely because the mapping is "best-effort inference", the OTel channel inherently has a semantic gap: if a library didn't write a key, or used one the adapter doesn't yet recognize, that field may be missing. That's why the model name tries six or seven keys and usage even digs into span events — the more the adapter recognizes, the smaller the gap. But however much it backfills, OTel's "inference" is never as precise as native's "explicit statement" — the price of embracing an open standard. Happily the adapter keeps evolving: whenever the community spawns a new convention, just add a few attribute keys to the priority list — old data unaffected, new data instantly recognized. This "add a key, gain compatibility" extensibility lets Langfuse grow with the OpenTelemetry GenAI ecosystem without rewriting the mapping each time.

🎯 Design trade-off
If the native channel is more precise, why bother supporting OpenTelemetry? Because you meet users where they are. OpenTelemetry is the de facto observability standard; countless teams' apps are already instrumented with it. If Langfuse only spoke its own SDK, it would demand users re-instrument just for it — a barrier that turns many away. Supporting OTLP means "you change not one line of instrumentation, just point your OTel data over and it works". The cost is real: an adapter that must keep chasing each convention, full of priority fallbacks, plus the semantic loss of inference. But the math works out — because the adapter lives only at the very front, and after translating it converges into the shared core (merge, cost, write is just one). Confine the grunt work and special cases to one adapter layer at the edge, keep the core pure and unified — the most classic use of the "adapter pattern".
🎯 Key points
  • Langfuse offers an OTLP endpoint (/api/public/otel/v1/traces) accepting OpenTelemetry spans (protobuf/JSON, gzip), letting you onboard without changing instrumentation.
  • OTLP uses the same S3 + queue pattern as Lesson 14: span body to S3, pointer to the dedicated OtelIngestionQueue (with primary/secondary).
  • OtelIngestionProcessor is a universal adapter: a priority attribute-key list per field, trying OTel gen_ai.*, Vercel ai.*, OpenLLMetry llm.* and more, taking the first present.
  • Mapping yields native ingestion events (Lesson 13 format); observations to IngestionService, traces to processEventBatch — thereafter identical to native.
  • Tradeoff: OTel is the "compatibility channel", inferred and with a semantic gap; native is the "precise channel". Confine adapter special-cases to the edge, keep the core unified — the classic adapter pattern.

🧪 Self-test - think about the design

1. After OTLP data enters at /otel/v1/traces, what does OtelIngestionProcessor translate spans into, and what happens next?
  1. into native IngestionEventType[] (Lesson 13's format): observations go to IngestionService.mergeAndWrite, traces to processEventBatch — thereafter identical to native-SDK data (one shared merge/cost/write)
  2. straight into a dedicated otel table, skipping merge
  3. into SQL statements to execute
  4. just stored verbatim in S3 and done
Show answer & explanation click to expand
Answer: A. OTel ingestion is essentially 'adapt at the edge, unify at the core'. OtelIngestionProcessor.processToIngestionEvents maps spans into the native event format, splits by entity type, and converges into the same Lesson 15–17 pipeline. Only the front-end mapping is OTel-specific; the backend is fully shared with native.
2. The same 'model name' is written to gen_ai.response.model by OpenTelemetry, ai.model.id by Vercel AI SDK, llm.model_name by OpenLLMetry. How does the adapter cope with this mess?
  1. picks a key at random
  2. requires all users to standardize on one convention
  3. it keeps a priority list of attribute keys per field, tries top to bottom, takes the first present; a new convention just needs a few keys added to the list — old data unaffected, new data instantly recognized
  4. only recognizes the official OpenTelemetry convention, discarding others
Show answer & explanation click to expand
Answer: C. extractModelName tries in turn langfuse-native → gen_ai.response.model → ai.model.id → gen_ai.request.model → llm.* → generic model, using the first hit. usage and input/output likewise. This 'add a key, gain compatibility' priority fallback lets Langfuse avoid forcing users to change conventions while evolving with the ecosystem.
3. Native ingestion is more precise than OTel — so why does Langfuse still support OpenTelemetry, and what's the essence of the trade-off?
  1. purely a marketing gimmick
  2. because OTel is faster than native
  3. to 'meet users where they are': OTel is the de facto standard and countless apps are already instrumented; supporting OTLP means users onboard without changing a line of instrumentation. The cost is a fallback-laden adapter + inference's semantic loss, but the adapter is only at the front, converging into the shared core after translation
  4. because the native SDK is being deprecated
Show answer & explanation click to expand
Answer: C. A classic adapter pattern: confine the grunt work and special cases (chasing conventions, handling the semantic gap) to one edge adapter, keeping the core (merge/cost/write) pure and unified. Lowering the onboarding barrier buys broader user coverage, while complexity is confined to the front layer without polluting the backend.
💭 Open questions (no single right answer - just think or try)
  • 'Confine the grunt work of adapting various external formats to one edge adapter, and let the core know only one unified format' — can you apply this in your own systems? If you must support multiple input protocols/data sources, where would you draw the 'adapter vs core' boundary? What goes wrong if you draw it wrong?