🪢 Langfuse 图解教程Langfuse Visual Guide 第十部分 · 平台与运维Part 10 · Platform & Operations 51 / 55
第十部分 · 平台与运维Part 10 · Platform & Operations

自我可观测与配置Self-observability & config

🧭 读前Before you start
先读 第18课 的 OTel 摄取,本课讲 Langfuse 如何自我观测与配置。
Read Lesson 18 on OTel ingestion; this lesson covers how Langfuse observes and configures itself.

Langfuse 是一个帮你观测 LLM 应用的工具。那它自己出问题时,靠什么观测自己?答案很「自洽」:它用同一套可观测性手段观测自身——OpenTelemetry(OTel)的链路追踪 + 结构化日志 + 指标。这一课讲两件让平台「对运维者透明」的事:① 自我可观测——你早就见过的 instrumentAsync(第30课eval、第33课告警、第46课集成、第48课登录都用它)其实是个 OTel span 包装器,把每个关键操作都变成一条可追踪的 span;而它的日志更妙,每一行都自动带上当前的 trace_id/span_id,让「日志」和「链路」一键互跳。② 配置即校验——所有环境变量都过一遍 Zod 校验,配错了启动时就当场报错,而不是跑着跑着神秘崩溃。 一句话:成熟的平台不仅能被观测,还吃自己的狗粮(dogfooding);不仅能跑,还在跑之前检查好自己的每一项配置。

📋 生活类比
想象一位专给别人体检的医生。最有说服力的,是他自己也定期体检、且用的是同一套仪器——这就是 dogfooding:观测工具观测自己。Langfuse 给每个内部操作都贴上一个「计时手环」(OTel span:何时开始、耗时多久、有没有出错),出问题时能精确还原「慢在哪一步」。 更贴心的是它的「病历」(日志):每条日志都自动盖上当次就诊的编号(trace_id/span_id)。于是你在一堆日志里看到一条报错,顺着这个编号就能跳到对应的完整链路看清前因后果;反过来从链路也能找到相关日志——两本账自动对得上号。 而配置校验,像医院开诊前的设备自检:开机时先把所有仪器参数(环境变量)逐一核对,缺一个、错一个就当场亮红灯不开诊,绝不带病上岗、看到一半才发现某台机器没插电。

自我可观测:instrumentAsync 这把「计时手环」

翻回前面几课,你会发现一个反复出现的名字:instrumentAsync。第30课 eval 处理、第33课告警状态机、第46课集成的逐项目任务、第48课登录的 session 回调——全都用它包了一层。现在揭晓它是什么:一个 OpenTelemetry span 包装器。你把一段异步逻辑交给它,它就:开一条 span(记下名字、开始时刻、span 类型)、运行你的逻辑、自动捕获异常(出错就 traceException 记到 span 上再抛出)、最后 span.end() 收尾。于是每个被包裹的操作,都在 OTel 里留下一条「何时开始、耗时多久、成功还是抛错」的可追踪记录。这正是 Langfuse 观测自己的方式——和它帮你观测 LLM 应用,是同一套 trace/span 思路。

把每个关键操作变成一条可追踪的 span instrumentAsync(ctx, fn) ① 开 span(名字·开始时刻·kind) ② 运行你的逻辑 await fn(span) ③ 出错 traceException → ④ span.end() 处处复用eval 处理(第30课)告警状态机(第33课)集成逐项目(第46课)登录 session 回调(第48课)同一个包装器,全局一致 同模块还提供getCurrentSpan / getTracertraceExceptionrecordGauge / IncrementrecordHistogram(指标)trace + metrics 一站齐 startNewTrace 开新根 trace,或从 traceContext 续上游链路(分布式追踪跨服务串起来) Langfuse 观测自己,用的就是它帮你观测 LLM 应用的同一套 OTel trace/span——dogfooding 它甚至能反向「摄取」OTel:OtelIngestionProcessor 让你的 OTel 数据直接进 Langfuse(两头都说 OTel)
instrumentAsync 包装器:instrumentation/index.ts:54 instrumentAsync(startActiveSpan 开 span、startNewTrace 开根 trace 或从 traceContext 续、catch 内 traceException 再抛、span.end())。同模块::140 getCurrentSpan、:142 traceException、:263 getTracer、:328/343/354 recordGauge/Increment/Histogram。OTel 摄取见 otel/OtelIngestionProcessor.ts。
示例:Langfuse 给自己埋的一条 OTel span worker · 埋点代码 instrumentAsync( { name: "blob-export-table", spanKind: INTERNAL }, async (span) => { span.setAttribute("blob.table", t); span.setAttribute("blob.fileType",f); span.setAttribute("blob.compressed",c); … do the export … }) 产出的 OTel span blob-export-table1.2s kind=INTERNAL · 自身也是一条 trace blob.tableobservations blob.projectIdproj_8f3a blob.fileTypeCSV blob.compressedtrue → OTLP exporter → 你的 APM(Datadog / Grafana / …) 同一套 OTel:对外摄取用户的 trace,对内也给自己埋点——「吃自己的狗粮」
Langfuse 用「它卖给你的那套可观测」来观测自己(取自 worker 的 instrumentAsync;值为示例):一句 instrumentAsync({ name:"blob-export-table", spanKind:INTERNAL }, span => …) 就把一次内部导出包成一条 OTel span,用 span.setAttribute 挂上 blob.table/blob.fileType/blob.compressed 等属性,再经 OTLP exporter 发往你的 APM(Datadog/Grafana……)。于是平台自己的延迟、错误、队列深度都有 trace 可查。这就是「吃自己的狗粮」:对外摄取用户 trace 的同一套 OpenTelemetry,对内也用来给自己埋点。
追踪instrumentAsync(span 包装器)
把异步逻辑包成一条 OTel span:自动计时、自动 catch 异常记到 span、自动结束。startNewTrace 开新根、或从上游 traceContext 续——跨服务的分布式追踪就这么串起来。前面四课用的都是它。
指标recordGauge / Increment / Histogram
trace 之外还有 metrics:用 recordIncrement 计数(如「处理了多少条」)、recordGauge 记瞬时值(如队列深度)、recordHistogram 记分布(如耗时 P95)。同一个 instrumentation 模块一站提供。
双向既发也收 OTel
Langfuse 不只用 OTel 观测自己(发出 span),还能接收 OTel:OtelIngestionProcessor 让你把应用的 OTel 数据直接喂进 Langfuse。两头都说 OTel 这门通用语言——既是工具的自洽,也是生态的开放。

日志的点睛:每行自动盖上 trace_id

Langfuse 的日志用 winston,按环境变量出 text(开发,人读友好)或 json(生产,机器可解析)两种格式,级别由 LANGFUSE_LOG_LEVEL 控制。但真正的点睛之笔是一个叫 tracingFormat 的自定义格式器:每写一行日志,它都去问一句「当前有没有活跃的 OTel span?」——有就把这条 span 的 trace_id 和 span_id 自动塞进这行日志(还顺带塞一份 Datadog 风格的 dd.trace_id)。

这一下,「日志」和「链路追踪」这两本账就自动对上了号。排查线上问题时,这是天大的便利:你在海量日志里发现一条报错 → 顺着它带的 trace_id → 一键跳到那次请求的完整链路,看清它走了哪些服务、每步耗时、错在第几跳;反过来,盯着一条慢链路,也能精准捞出它沿途打的所有日志。没有这层关联,日志和链路就是两座孤岛,排查全靠手工对时间戳——又慢又易错。一行不起眼的「自动注入」,省下的是无数个深夜的对账之苦。

日志 ⇄ 链路:自动对上号,一键互跳 winston loggertext(开发·人读)/ json(生产·机读)级别 LANGFUSE_LOG_LEVELtracingFormat 格式器每行:问「有活跃 span 吗?」 注入 trace 标识trace_id · span_id+ dd.trace_id(Datadog 风格)getCurrentSpan() 拿当前 span 日志 → 链路见报错日志,顺 trace_id 跳完整链路 链路 → 日志盯一条慢链路,捞出沿途所有日志 没有这层关联,日志和链路是两座孤岛,排查靠手工对时间戳——又慢又易错 一行「自动注入」,换来无数次「从这条错日志,直达那次请求全貌」的顺滑排查 这正是可观测性的「三支柱」协同:traces + logs(+ metrics)互相打通才好用
日志关联 trace:logger.ts winston,tracingFormat(:6 起)每行调 getCurrentSpan(),有 span 就注入 trace_id/span_id 与 dd.trace_id/dd.span_id;格式 textLoggerFormat(:31)或 jsonLoggerFormat 由 LANGFUSE_LOG_FORMAT 选、级别由 LANGFUSE_LOG_LEVEL 定。
packages/shared/src/server/logger.ts日志自动注入 trace 标识
// 自定义格式器:每写一行日志,自动盖上当前 span 的 trace/span id
const tracingFormat = () => winston.format((info) => {
  const span = getCurrentSpan();                       // 当前活跃的 OTel span
  if (span) {
    const { spanId, traceId } = span.spanContext();
    info["trace_id"] = traceId;  info["span_id"] = spanId;  // 注入!
    info["dd.trace_id"] = …;                          // 顺带 Datadog 风格
  }
  return info;
})();

// 格式按 env 选 text(开发) / json(生产);级别按 env
const format = env.LANGFUSE_LOG_FORMAT === "text" ? textLoggerFormat : jsonLoggerFormat;
export const logger = winston.createLogger({ level: env.LANGFUSE_LOG_LEVEL, format,
                                          transports: [new winston.transports.Console()] });

🔗 traces(链路)

一次请求/任务走过的完整路径与各步耗时。由 instrumentAsync 的 span 拼成——回答「慢在哪一步、错在第几跳」。

📝 logs(日志)

离散的事件记录(含报错)。winston 输出,且每行自动带 trace_id——回答「具体发生了什么」,并能跳回对应链路。

📊 metrics(指标)

聚合的数值趋势:计数、瞬时值、分布。recordIncrement/Gauge/Histogram 上报——回答「整体健康吗、量级多大、P95 几何」。

配置即校验:让错配在启动时就暴露

一个平台有几十上百个环境变量(数据库地址、Redis、密钥、各种开关)。如果哪个填错了、漏填了,最糟的情况是程序照样启动,跑到半路才神秘崩溃——而且报的错往往离真正的根因十万八千里。Langfuse 的对策是把所有 env 都过一遍 Zod 校验:共享层用 EnvSchema = z.object({...}) 声明每个变量的类型与约束(如 REDIS_PORT 用 z.coerce.number() 把字符串转数字、REDIS_TLS_ENABLED 只能是 "true"|"false");web 层用 @t3-oss/env-nextjs 的 createEnv 把 env 分成 server(仅服务端,如 DATABASE_URL)和 client(可进浏览器,须 NEXT_PUBLIC_ 前缀)两类。

这带来两个硬好处:① fail-fast(快速失败)——配置不合法,启动那一刻就当场报清晰的错(「DATABASE_URL 不是合法 URL」),而不是带病运行、半夜崩给你看。② server/client 边界——createEnv 强制区分「只能在服务端用的密钥」和「能打进前端包的公开配置」,从机制上防止你手滑把数据库密码泄露进浏览器。配置不再是「一堆 process.env.XXX 散落各处、类型全靠猜」,而是一份声明式、类型安全、边界清晰的契约。

配置即校验:启动时过 Zod,错配当场暴露 原始 env process.env · string|undefined Zod 校验 EnvSchema = z.object + createEnv(server/client) ✗ 不合法 → fail-fast 启动报错 ✓ 类型安全 z.coerce / z.enum server | client 边界(防密钥泄露)
没有校验(裸 process.env)Langfuse 的 Zod 校验
错配可能跑到半路才崩fail-fast:启动时当场报清晰的错
process.env.X 类型永远是 string|undefined类型安全:z.coerce 转数字、z.enum 限取值
密钥可能手滑打进前端包server/client 边界:createEnv 强制隔离
有哪些配置、什么含义,全靠翻代码声明式契约:一份 schema 即文档

把「可观测」和「配置校验」放在同一课,是因为它们朝着同一个方向使劲:让系统对运维者「透明、可预期、早暴露问题」。可观测让你在运行时看清系统在干什么、慢在哪、错在哪;配置校验让你在启动时就堵住一整类「配错了」的低级故障。一个管运行期、一个管启动期,合起来就是「这个平台好不好运维」的底色。

🎯 设计取舍
Langfuse 自己就是个可观测性工具,为什么还要费力用 OTel 观测自己?这种 dogfooding 值得吗? 太值了,而且不只是「显得自洽」。第一,它是最严苛的自测:当你每天用自己的产品扛自己的生产负载,产品的毛病你会第一个、最痛地感受到——这种「自己人先吃苦」的反馈,比任何用户调研都快、都真。第二,技术上极其顺滑:Langfuse 本就讲 OTel 这门语言(它能摄取你的 OTel 数据),于是观测自己时复用同一套 trace/span 概念,几乎零额外认知成本——发出去的 span 和收进来的 span 是同构的。第三,它逼着把可观测做成「随手可用」:正因为内部到处要用,instrumentAsync 才被打磨成一个包一下就行的薄包装,而不是每处手写一堆 OTel 样板。一个工具最好的广告,是它自己离不开它。

为什么要花力气把 env 做成 Zod 校验 + server/client 分离,直接读 process.env.XXX 不是更省事? 因为「省事」省的是写的时候,赔的是出事的时候。裸读 process.env 有三宗罪:① 类型永远是 string | undefined,你以为是数字的端口其实是字符串,比较、运算处处埋雷;② 错配不会立刻报错——少配一个 DATABASE_URL,应用照启动,直到第一次查库才崩,而那时的错误信息离根因很远,排查能耗掉半天;③ 最危险的是边界失守:一个本该只在服务端用的密钥,手滑加了 NEXT_PUBLIC_ 或被打进客户端 bundle,就泄露给了所有访问者。Zod + createEnv 一次性解决:启动时校验(fail-fast)、自动类型推导(类型安全)、强制 server/client 隔离(防泄露)。把「配置」当成需要校验的「输入」来对待,而不是一堆可以随便读的全局字符串——这是把低级故障挡在生产之外的关键一步。
🎯 本课要点
  • 自我可观测(dogfooding):Langfuse 用它帮你观测 LLM 应用的同一套 OTel 观测自己。instrumentAsync(instrumentation/index.ts:54)是个 span 包装器——开 span、跑逻辑、自动 traceException、span.end,第30/33/46/48 课都在用。
  • trace + metrics 一站齐:同模块还有 getCurrentSpan/traceException/getTracer 和 recordGauge/Increment/Histogram(指标)。还能反向摄取 OTel(OtelIngestionProcessor)——两头都说 OTel。
  • 日志自动关联 trace:winston 的 tracingFormat 每行日志都注入当前 span 的 trace_id/span_id(含 dd.* Datadog 风格)。于是日志↔链路一键互跳——排查线上问题的天大便利。格式 text/json、级别由 env 定。
  • 配置即校验:所有 env 过 Zod(共享 EnvSchema = z.object、web createEnv 分 server/client)。好处:fail-fast(启动即报错而非半路崩)、类型安全(z.coerce/z.enum)、server/client 边界(防密钥泄露进前端)。
  • 同一个方向:可观测管「运行时透明」、配置校验管「启动时堵错」,合起来决定「这平台好不好运维」。把配置当「需要校验的输入」而非「随便读的全局字符串」。

🧪 自测 · 想一想为什么这么设计

1. Langfuse 本身就是一个 LLM 可观测性平台,但它内部还到处用 OpenTelemetry(instrumentAsync 等)观测自己。关于这种「dogfooding(吃自己的狗粮)」,下面哪种说法最准确?
  1. 为了让代码看起来更复杂、更专业
  2. 只是为了营销,技术上没有实际意义
  3. 因为 OTel 是唯一能用的可观测方案
  4. 它既是最严苛的自测(每天用自己产品扛自己负载,毛病第一个最痛地暴露),技术上又极顺滑(本就讲OTel这门语言、能摄取OTel,观测自己复用同一套trace/span概念近零额外成本),还逼着把instrumentAsync打磨成包一下就行的薄包装
看答案与解析 点击展开
答案:D。dogfooding 在 Langfuse 这里有三重价值。其一,最真实的自测:当团队每天用自己的产品扛自己的生产流量,产品的卡顿、缺陷、难用之处,开发者会第一个、最切肤地感受到,反馈比任何外部调研都快都真。其二,技术自洽:Langfuse 本就讲 OTel 这门通用语言(它能摄取你应用的 OTel 数据),所以观测自己时复用同一套 trace/span 概念,发出去的 span 和收进来的 span 是同构的,几乎没有额外认知负担。其三,它逼着把可观测能力做得「随手可用」——正因为内部处处要用,instrumentAsync 才被打磨成「包一下就行」的薄包装,而非每处手写 OTel 样板。一个工具最好的背书,是它自己离不开它。
2. Langfuse 的 winston 日志里有个 tracingFormat 格式器:每写一行日志,就把当前活跃 OTel span 的 trace_id/span_id 自动注入这行。这个设计解决了什么排查痛点?
  1. 让日志文件更小
  2. 让「日志」和「链路追踪」自动对上号:从一条报错日志顺trace_id一键跳到该请求的完整链路(看清走了哪些服务、每步耗时、错在第几跳),反向也能从慢链路捞出沿途所有日志——否则两者是孤岛,排查只能手工对时间戳
  3. 给日志加密
  4. 让日志写入更快
看答案与解析 点击展开
答案:B。可观测性常说「三支柱」:traces(链路)、logs(日志)、metrics(指标)。但三者若彼此孤立,价值大打折扣——最典型的痛点就是:你在海量日志里看到一条报错,却不知道它属于哪次请求、那次请求前后发生了什么;或者你盯着一条慢链路,却捞不出它沿途打了哪些日志。tracingFormat 用一行「自动注入 trace_id/span_id」把这两座岛连了起来:日志带着链路 id,于是日志↔链路可以一键互跳,前因后果一目了然。没有它,排查就退化成「手工对齐时间戳」,又慢又易错。这就是「让可观测三支柱协同」的关键一招。
3. Langfuse 不直接到处读 process.env.XXX,而是把所有环境变量过一遍 Zod 校验(共享 EnvSchema、web 用 createEnv 分 server/client)。这样做的好处不包括下面哪一项?
  1. fail-fast:配置不合法时启动当场就报清晰的错,而非带病运行到半路神秘崩溃
  2. server/client 边界:createEnv 强制隔离仅服务端密钥与可进前端的公开配置,防止密钥泄露进浏览器包
  3. 类型安全:z.coerce 把端口字符串转数字、z.enum 限定取值,不再人人都是 string|undefined
  4. 让程序运行时占用更少的内存
看答案与解析 点击展开
答案:D。Zod 校验 env 的三大好处是 fail-fast、类型安全、server/client 边界,但「省内存」不在其列——校验本身在启动时跑一次,对运行时内存占用没有实质影响,选它为「好处」是不对的。逐一看真正的好处:① fail-fast——错配在启动那一刻就当场报清晰的错(如 DATABASE_URL 非法),而不是跑到第一次用才崩、错误信息还离根因很远;② 类型安全——z.coerce/z.enum 把「永远是 string|undefined」的裸 env 变成有确定类型和取值约束的值;③ server/client 边界——createEnv 从机制上区分「只能服务端用的密钥」与「能打进前端的公开配置」,防止手滑把数据库密码泄露给所有访问者。核心理念:把配置当成「需要校验的输入」,而非「随便读的全局字符串」。
💭 发散思考(没有标准答案,动手或动脑想想)
  • 这一课把「可观测性」(运行时透明:trace/log/metric 三支柱协同)和「配置校验」(启动时堵错:Zod fail-fast)放在一起,因为它们都服务于「让系统对运维者透明、可预期、早暴露问题」。请回想你运维或开发过的系统:你是怎么把 trace、log、metric 关联起来的(还是它们各自为政)?你吃过「配置错了却跑到半路才崩」的亏吗?如果让你给一个新服务定「最低限度的可运维性」清单,你会列哪几条?

Langfuse is a tool that helps you observe LLM applications. So when it itself has a problem, how does it observe itself? The answer is fittingly self-consistent: it uses the same observability means to observe itself — OpenTelemetry (OTel) distributed tracing + structured logging + metrics. This lesson covers two things that make the platform "transparent to operators": ① self-observability — the instrumentAsync you've long seen (Lesson 30 eval, Lesson 33 alerts, Lesson 46 integrations, Lesson 48 login all use it) is actually an OTel span wrapper turning every key operation into a traceable span; and its logging is cleverer still, auto-stamping each line with the current trace_id/span_id, so "logs" and "traces" jump to each other in one click. ② config as validation — all environment variables pass through Zod validation, so a misconfiguration errors out at boot, on the spot, rather than crashing mysteriously mid-run. In a sentence: a mature platform is not only observable but eats its own dog food (dogfooding); not only runs, but checks every one of its configs before running.

📋 Analogy
Picture a doctor who gives others checkups. The most convincing one also gets regular checkups himself, using the same instruments — that's dogfooding: the observability tool observing itself. Langfuse puts a "timing wristband" on every internal operation (an OTel span: when it started, how long it took, whether it errored), so on a problem it can precisely reconstruct "which step was slow." Even more thoughtful is its "medical record" (logs): each log line is auto-stamped with the visit number (trace_id/span_id). So when you spot one error among a pile of logs, follow that number to jump to the full corresponding trace and see the whole cause-and-effect; conversely, from a trace you can find related logs — the two ledgers auto-reconcile. And config validation is like the hospital's pre-opening equipment self-check: at startup it cross-checks every instrument's parameters (env vars) one by one, and if one is missing or wrong, it flashes red and doesn't open — never operating impaired, never discovering halfway that a machine wasn't plugged in.

Self-observability: the "timing wristband" instrumentAsync

Flip back through earlier lessons and you'll see a recurring name: instrumentAsync. Lesson 30's eval processing, Lesson 33's alert state machine, Lesson 46's per-project integration jobs, Lesson 48's login session callback — all wrap a layer with it. Now the reveal: it's an OpenTelemetry span wrapper. Hand it a piece of async logic and it: opens a span (records name, start time, span kind), runs your logic, auto-captures exceptions (on error, traceException records it on the span then rethrows), and finally span.end() wraps up. So every wrapped operation leaves a traceable record in OTel of "when it started, how long it took, success or threw." This is exactly how Langfuse observes itself — the same trace/span thinking it uses to observe your LLM apps for you.

Turn every key operation into a traceable span instrumentAsync(ctx, fn) ① open span (name·start·kind) ② run your logic await fn(span) ③ on error traceException → ④ span.end() reused everywhereeval processing (L30)alert state machine (L33)per-project integration (L46)login session callback (L48)same wrapper, globally consistent same module also offersgetCurrentSpan / getTracertraceExceptionrecordGauge / IncrementrecordHistogram (metrics)trace + metrics in one place startNewTrace opens a new root trace, or continues the upstream chain from traceContext (distributed tracing stitched across services) Langfuse observes itself with the very same OTel trace/span it uses to observe your LLM apps — dogfooding It can even "ingest" OTel in reverse: OtelIngestionProcessor lets your OTel data flow straight into Langfuse (both ends speak OTel)
The instrumentAsync wrapper: instrumentation/index.ts:54 instrumentAsync (startActiveSpan opens a span, startNewTrace opens a root trace or continues from traceContext, catch calls traceException then rethrows, span.end()). Same module: :140 getCurrentSpan, :142 traceException, :263 getTracer, :328/343/354 recordGauge/Increment/Histogram. OTel ingestion at otel/OtelIngestionProcessor.ts.
Example: an OTel span Langfuse emits about itself worker · instrumentation instrumentAsync( { name: "blob-export-table", spanKind: INTERNAL }, async (span) => { span.setAttribute("blob.table", t); span.setAttribute("blob.fileType",f); span.setAttribute("blob.compressed",c); … do the export … }) the OTel span produced blob-export-table1.2s kind=INTERNAL · itself a trace blob.tableobservations blob.projectIdproj_8f3a blob.fileTypeCSV blob.compressedtrue → OTLP exporter → your APM (Datadog / Grafana / …) the same OTel: it ingests users' traces outward and instruments itself inward — eating its own dog food
Langfuse observes itself with the same observability it sells you (from the worker's instrumentAsync; values illustrative): one instrumentAsync({ name:"blob-export-table", spanKind:INTERNAL }, span => …) wraps an internal export into an OTel span, attaches attributes like blob.table/blob.fileType/blob.compressed via span.setAttribute, and ships it through an OTLP exporter to your APM (Datadog/Grafana…). So the platform's own latency, errors and queue depth are all traceable. That's eating its own dog food: the same OpenTelemetry it ingests users' traces with, it also uses to instrument itself.
traceinstrumentAsync (span wrapper)
Wraps async logic into an OTel span: auto-timing, auto-catch exceptions onto the span, auto-end. startNewTrace opens a new root, or continues from the upstream traceContext — that's how cross-service distributed tracing is stitched. The previous four lessons all use it.
metricsrecordGauge / Increment / Histogram
Beyond traces there are metrics: recordIncrement for counts (e.g. "how many processed"), recordGauge for instantaneous values (e.g. queue depth), recordHistogram for distributions (e.g. P95 latency). The same instrumentation module offers them all.
two-wayemits and receives OTel
Langfuse not only uses OTel to observe itself (emitting spans), it also receives OTel: OtelIngestionProcessor lets you feed your app's OTel data straight into Langfuse. Both ends speak the universal OTel language — the tool's self-consistency and the ecosystem's openness.

The logging masterstroke: each line auto-stamped with trace_id

Langfuse's logging uses winston, emitting text (dev, human-friendly) or json (prod, machine-parseable) by env var, with level controlled by LANGFUSE_LOG_LEVEL. But the real masterstroke is a custom formatter called tracingFormat: every time it writes a log line, it asks "is there an active OTel span right now?" — if so it auto-stuffs that span's trace_id and span_id into the line (plus a Datadog-style dd.trace_id for good measure).

With that, "logs" and "distributed tracing" auto-reconcile. When debugging production issues, this is a huge convenience: you find one error among a sea of logs → follow its trace_id → jump in one click to that request's full trace, seeing which services it traversed, each step's latency, which hop errored; conversely, eyeing a slow trace, you can precisely scoop up all the logs it emitted along the way. Without this correlation, logs and traces are two isolated islands and debugging means manually matching timestamps — slow and error-prone. One inconspicuous line of "auto-injection" saves countless late-night reconciliation headaches.

Logs ⇄ traces: auto-reconciled, one-click jump winston loggertext(dev·human) / json(prod·machine)level LANGFUSE_LOG_LEVELtracingFormat formattereach line: "any active span?" inject trace idstrace_id · span_id+ dd.trace_id (Datadog style)getCurrentSpan() gets the current span log → tracesee an error log, follow trace_id to the full trace trace → logeye a slow trace, scoop all its logs Without this correlation, logs and traces are isolated islands; debugging means manually matching timestamps — slow and error-prone One line of "auto-injection" buys countless smooth "from this error log, straight to that request's full picture" debug sessions This is the "three pillars" of observability working together: traces + logs (+ metrics) are useful only when wired to each other
Logs correlate with traces: logger.ts winston, tracingFormat (from :6) on each line calls getCurrentSpan(), and if there's a span injects trace_id/span_id and dd.trace_id/dd.span_id; the format textLoggerFormat (:31) or jsonLoggerFormat is chosen by LANGFUSE_LOG_FORMAT, level by LANGFUSE_LOG_LEVEL.
packages/shared/src/server/logger.tslogs auto-inject trace ids
// custom formatter: each log line auto-stamped with the current span's trace/span id
const tracingFormat = () => winston.format((info) => {
  const span = getCurrentSpan();                       // the current active OTel span
  if (span) {
    const { spanId, traceId } = span.spanContext();
    info["trace_id"] = traceId;  info["span_id"] = spanId;  // inject!
    info["dd.trace_id"] = …;                          // plus Datadog style
  }
  return info;
})();

// format chosen by env: text(dev) / json(prod); level by env
const format = env.LANGFUSE_LOG_FORMAT === "text" ? textLoggerFormat : jsonLoggerFormat;
export const logger = winston.createLogger({ level: env.LANGFUSE_LOG_LEVEL, format,
                                          transports: [new winston.transports.Console()] });

🔗 traces

The full path a request/job traverses with each step's latency. Stitched from instrumentAsync's spans — answers "which step was slow, which hop errored."

📝 logs

Discrete event records (incl. errors). Emitted by winston, each line auto-carrying trace_id — answers "what specifically happened," and can jump back to the corresponding trace.

📊 metrics

Aggregated numeric trends: counts, instantaneous values, distributions. Reported via recordIncrement/Gauge/Histogram — answers "is it healthy overall, what's the volume, what's the P95."

Config as validation: surface misconfig at boot

A platform has dozens to hundreds of environment variables (DB address, Redis, secrets, various toggles). If one is wrong or missing, the worst case is the program starts anyway and crashes mysteriously mid-run — and the reported error is often light-years from the true root cause. Langfuse's countermeasure is to run all env through Zod validation: the shared layer uses EnvSchema = z.object({...}) to declare each variable's type and constraints (e.g. REDIS_PORT uses z.coerce.number() to turn a string into a number, REDIS_TLS_ENABLED can only be "true"|"false"); the web layer uses @t3-oss/env-nextjs's createEnv to split env into server (server-only, like DATABASE_URL) and client (may reach the browser, must have a NEXT_PUBLIC_ prefix).

This brings two hard wins: ① fail-fast — invalid config errors clearly the moment of startup ("DATABASE_URL is not a valid URL"), rather than running impaired and crashing on you at midnight. ② server/client boundary — createEnv forces a distinction between "secrets usable only server-side" and "public config that can ship in the front-end bundle," mechanically preventing you from fat-fingering a DB password into the browser. Config is no longer "a pile of process.env.XXX scattered everywhere, types all guesswork" but a declarative, type-safe, boundary-clear contract.

config as validation: pass Zod at boot, surface misconfig on the spot raw env process.env · string|undefined Zod validation EnvSchema = z.object + createEnv(server/client) ✗ invalid → fail-fast boot error ✓ type-safe z.coerce / z.enum server | client boundary (no leaks)
No validation (raw process.env)Langfuse's Zod validation
misconfig may crash mid-runfail-fast: errors clearly at startup
process.env.X is always string|undefinedtype-safe: z.coerce to number, z.enum to limit values
a secret may be fat-fingered into the front-end bundleserver/client boundary: createEnv enforces isolation
what configs exist and mean is buried in codedeclarative contract: one schema is the doc

Putting "observability" and "config validation" in one lesson is because they push in the same direction: making the system "transparent, predictable, early-surfacing problems" to operators. Observability lets you see at runtime what the system is doing, where it's slow, where it errored; config validation plugs at startup a whole class of "misconfigured" low-level failures. One owns the run phase, one the boot phase, and together they're the baseline of "how operable is this platform."

🎯 Design trade-off
Langfuse is itself an observability tool — why bother using OTel to observe itself? Is this dogfooding worth it? Very much so, and not just for "looking self-consistent." First, it's the most rigorous self-test: when you run your own product carrying your own production load every day, you feel the product's flaws first and most painfully — this "eat your own bitterness first" feedback is faster and truer than any user research. Second, it's technically seamless: Langfuse already speaks the OTel language (it can ingest your OTel data), so observing itself reuses the same trace/span concepts with near-zero extra cognitive cost — the spans it emits and the spans it ingests are isomorphic. Third, it forces observability to be "ready at hand": precisely because it's used internally everywhere, instrumentAsync got polished into a thin wrap-once wrapper rather than hand-written OTel boilerplate at every site. A tool's best advertisement is that it can't live without itself.

Why bother making env into Zod validation + server/client separation — isn't reading process.env.XXX directly easier? Because "easier" saves you when writing and costs you when things break. Raw process.env has three sins: ① the type is always string | undefined, so a port you think is a number is actually a string, planting landmines in comparisons and arithmetic; ② misconfig doesn't error immediately — miss a DATABASE_URL and the app starts anyway, crashing only on the first DB query, by which point the error message is far from the root cause and debugging eats half a day; ③ most dangerous is boundary breach: a secret meant only for the server, fat-fingered with NEXT_PUBLIC_ or bundled into the client, leaks to every visitor. Zod + createEnv solves all at once: validate at startup (fail-fast), auto type inference (type-safe), enforce server/client isolation (anti-leak). Treating "config" as "input" that needs validation, rather than a pile of freely-readable global strings, is the key step to keeping low-level failures out of production.
🎯 Key points
  • Self-observability (dogfooding): Langfuse observes itself with the same OTel it uses to observe your LLM apps. instrumentAsync (instrumentation/index.ts:54) is a span wrapper — open span, run logic, auto-traceException, span.end; used by Lessons 30/33/46/48.
  • Trace + metrics in one place: the same module also has getCurrentSpan/traceException/getTracer and recordGauge/Increment/Histogram (metrics). It can also ingest OTel in reverse (OtelIngestionProcessor) — both ends speak OTel.
  • Logs auto-correlate with traces: winston's tracingFormat injects the current span's trace_id/span_id (incl. dd.* Datadog style) into every log line. So logs↔traces jump in one click — a huge convenience for production debugging. Format text/json, level by env.
  • Config as validation: all env passes through Zod (shared EnvSchema = z.object, web createEnv splitting server/client). Wins: fail-fast (error at startup not crash mid-run), type-safe (z.coerce/z.enum), server/client boundary (prevent secrets leaking to the front end).
  • One direction: observability owns "runtime transparency," config validation owns "boot-time error-plugging," together deciding "how operable is this platform." Treat config as "input needing validation," not "freely-readable global strings."

🧪 Self-test - think about the design

1. Langfuse is itself an LLM observability platform, yet internally it uses OpenTelemetry (instrumentAsync etc.) to observe itself everywhere. Which statement about this 'dogfooding' is most accurate?
  1. to make the code look more complex and professional
  2. purely for marketing, with no real technical meaning
  3. because OTel is the only available observability option
  4. it's both the most rigorous self-test (running its own product under its own load daily, flaws surface first and most painfully) and technically seamless (it already speaks OTel and can ingest OTel, so observing itself reuses the same trace/span concepts at near-zero extra cost), and it forces instrumentAsync to be polished into a thin wrap-once wrapper
Show answer & explanation click to expand
Answer: D. Dogfooding has triple value at Langfuse. First, the truest self-test: when the team runs its own product under its own production traffic daily, the product's jank, defects, and rough edges are felt first and most acutely by developers—feedback faster and truer than any external research. Second, technical self-consistency: Langfuse already speaks the universal OTel language (it can ingest your app's OTel data), so observing itself reuses the same trace/span concepts, with emitted and ingested spans isomorphic, at near-zero extra cognitive load. Third, it forces observability to be 'ready at hand'—precisely because it's needed internally everywhere, instrumentAsync got polished into a 'wrap once' thin wrapper rather than hand-written OTel boilerplate everywhere. A tool's best endorsement is that it can't live without itself.
2. Langfuse's winston logging has a tracingFormat formatter: every log line auto-injects the current active OTel span's trace_id/span_id. What debugging pain point does this solve?
  1. makes log files smaller
  2. auto-reconciles 'logs' and 'distributed traces': from one error log follow trace_id to jump to that request's full trace (which services, each step's latency, which hop errored) in one click, and in reverse scoop all logs from a slow trace—otherwise the two are isolated islands and debugging means manually matching timestamps
  3. encrypts the logs
  4. makes log writes faster
Show answer & explanation click to expand
Answer: B. Observability often speaks of 'three pillars': traces, logs, metrics. But if the three are isolated, their value drops sharply—the classic pain: you see an error among a sea of logs but don't know which request it belongs to or what happened around it; or you eye a slow trace but can't scoop the logs it emitted along the way. tracingFormat connects these two islands with one line of 'auto-inject trace_id/span_id': logs carry the trace id, so logs↔traces jump in one click, cause and effect clear at a glance. Without it, debugging degrades to 'manually aligning timestamps', slow and error-prone. This is the key move of 'making the observability pillars work together'.
3. Rather than reading process.env.XXX everywhere, Langfuse runs all env vars through Zod validation (shared EnvSchema, web createEnv splitting server/client). Which of the following is NOT a benefit of this?
  1. fail-fast: invalid config errors clearly at startup, rather than running impaired and crashing mysteriously mid-run
  2. server/client boundary: createEnv enforces isolating server-only secrets from public config, preventing secrets leaking into the browser bundle
  3. type-safe: z.coerce turns a port string into a number, z.enum limits values, no longer all string|undefined
  4. makes the program use less memory at runtime
Show answer & explanation click to expand
Answer: D. Zod-validating env has three main benefits—fail-fast, type-safety, server/client boundary—but 'saving memory' isn't one: validation runs once at startup with no real impact on runtime memory, so choosing it as a 'benefit' is wrong. The real benefits: ① fail-fast—misconfig errors clearly the moment of startup (e.g. invalid DATABASE_URL), not crashing on first use with an error far from the root cause; ② type-safety—z.coerce/z.enum turn 'always string|undefined' raw env into values with definite types and value constraints; ③ server/client boundary—createEnv mechanically separates 'server-only secrets' from 'public config that can ship to the front end', preventing a fat-fingered DB password from leaking to every visitor. Core idea: treat config as 'input needing validation', not 'freely-readable global strings'.
💭 Open questions (no single right answer - just think or try)
  • This lesson pairs 'observability' (runtime transparency: the trace/log/metric three pillars working together) with 'config validation' (boot-time error-plugging: Zod fail-fast), because both serve 'making the system transparent, predictable, and early-surfacing to operators'. Recall systems you've operated or built: how did you correlate traces, logs, and metrics (or did they each go their own way)? Have you been burned by 'misconfig that only crashed mid-run'? If defining a 'minimum operability' checklist for a new service, which items would you list?