🪢 Langfuse 图解教程Langfuse Visual Guide 第十一部分 · 设计专题与终章Part 11 · Design Themes & Capstone 55 / 55
第十一部分 · 设计专题与终章Part 11 · Design Themes & Capstone

终章·一条 trace 的完整一生Capstone: the full life of a trace

🧭 读前Before you start
先读 第5课 的一生鸟瞰,本课跟一条 trace 走完贯穿 55 课的完整旅程。
Read Lesson 5 for the bird's-eye life; this capstone follows one trace through the full journey across all 55 lessons.

最后一课,我们做一件浪漫的事:跟着一条 trace,走完它的一生。前面 54 课,每一课都像在解剖一个器官——这一课,让这条 trace 活过来,从它在你应用里出生的那一毫秒,到被摄取、落库、被查看、被评分、触发自动化、最终在保留期到点时谢幕。沿途,我们会在每个驿站标出「这是第几课讲的」,让你亲眼看见:那些你逐课学过的零件,是怎么严丝合缝地咬合成一台完整机器的。 读完这一程,你对 Langfuse 的理解会从「知道每个部件」升级为「看见整个系统如何协作」。而你会发现,这整台机器,归根到底只是在从容地回答一个问题:你的 AI 应用,到底做了什么——而你怎么记录它、找到它、评判它、并据此行动?

📋 生活类比
把一条 trace 想象成一个「快递包裹」的一生。它在寄件人手里诞生(你的应用调用 LLM,SDK 把这次调用打包);在分拣中心被高速处理(摄取链路:扫码登记、分拣、入库);在仓库里被妥善存放(三个库各放一部分:单号信息、明细、大件);被人在系统里查询追踪(列表、详情、按时间线看它的来龙去脉);被质检员抽检评分(评估与监控);触发下游联动(到货通知、转寄、归档);最后在保管期满时被销毁,且三个库一处不漏地清干净。 你前面 54 课,是分别参观了寄件台、分拣机、货架、查询台、质检站、通知中心、销毁间——这一课,是跟着同一个包裹,把这些站点一次性串成一条流水线,看它们如何首尾相接、共同完成「让每一次 AI 调用都有据可查、可评可控」这件事。

第一程:出生 → 摄取 → 落库

① 出生(第12课)。你的应用调一次 LLM,SDK 就地创建一条 trace 和它下面的若干 observation,把输入、输出、模型、用量都记下来,攒成一批,异步 POST 到摄取 API——你的主流程几乎无感。② 摄取(第13-19课)。API 先用 API key 认证(第49课的两层哈希校验),把原始事件落进 S3、把任务塞进 Redis 队列就立刻 207 返回——这就是「快接收」。然后 worker 从队列取出,做校验、解析、可能的合并,upsert 进 ClickHouse。③ 落库(第13课)。这条 trace 最终散成三份安家:可分析的事件明细进 ClickHouse(不可变、ReplacingMergeTree 查时去重保留最新)、大块输入输出进 S3、相关元数据进 Postgres。出生到落库,第10课「双(三)存储」与第54课「异步、不可变」两大主题,在这里第一次合奏。

一条 trace 的一生:七个驿站,串成一条流水线 ①出生SDK创建L12 ②摄取S3+队列+workerL13-19/L49 ③落库CH+S3+PGL13 ④被读列表/详情/RESTL20-27 ⑤被评估eval/监控/数据集L28-36 ⑥被作用自动化/导出L44-47 ⑦退场跨存储删除L52 贯穿全程的三条隐线全程被平台自己的 OTel/日志观测(L51) · 按组织 plan 门控功能(L50) · 每一步都按 projectId 隔离(多租户) 六大主题在这条流水线上各就各位宽事件(数据形状)·不可变(怎么写)·异步(在哪干)·双存储(存哪)·多租户(归谁)·成本(什么值得做)
一条 trace 的完整生命周期:出生(L12 SDK) → 摄取(L13-19 摄取链路 + L49 API key 认证) → 落库(L13 ClickHouse/S3/Postgres) → 被读(L20-27) → 被评估(L28-36) → 被作用(L44-47) → 退场(L52)。三条隐线贯穿:自观测(L51)、plan 门控(L50)、projectId 多租户隔离。
终章:一条 trace 的一生(地铁线) 诞生 SDK 埋点 ① L06 摄取 API→S3+Redis→207 ② L12 入库 worker→ClickHouse ③ L13·17 被读 列表/详情 ④ L24 被评估 裁判/评分 ⑤ L29 被作用 监控/自动化 ⑥ L44 退休 保留/删除 ⑦ L52 ↓ 三条贯穿每一站的隐藏轨 🔑 projectId 多租户:每行每查询都被它圈住 L06·49 📊 OTel 自我可观测:每一步自己也是一条 trace L51 🎫 plan 权益:每一步能不能用由它决定 L50 一条 trace 从生到退,全程被这三条轨贯穿——这就是 Langfuse 的全貌
把整本书连成一条线:一条 trace 的一生(课号为导航):它在你的应用里诞生(SDK 埋点,L06),被摄取(API 落 S3+Redis 即返 207,L12),由 worker 合并入库(ClickHouse,L13/L17),随后被读(列表/详情,L24)、被评估(裁判/评分,L29)、被作用(监控/自动化,L44),最终退休(保留窗口到期后跨存储删除,L52)。而三条隐藏轨贯穿每一站:projectId(多租户,L06/L49)圈住每行每查询、OTel(L51)让每一步自己也成为一条 trace、plan(权益,L50)决定每一步能不能用。读懂这条线,你就读懂了 Langfuse。
1

出生:SDK 就地打包(L12)

应用调 LLM,SDK 创建 trace + observation,记下输入/输出/模型/用量,攒批异步 POST——主流程几乎无感。

2

摄取:快接收 + 异步处理(L13-19, L49)

API 用 API key 两层哈希认证、原始事件落 S3、塞 Redis 队列即返回;worker 取出做校验解析、upsert 进 ClickHouse。

3

落库:散成三份安家(L13)

事件明细→ClickHouse(不可变,查时合并)、大块负载→S3、元数据→Postgres。「双(三)存储 + 异步 + 不可变」三主题首次合奏。

worker/src/queues/ingestionQueue.ts落库这一刻
// 「落库」的真相:worker 取出队列消息后,把这条实体的多条事件 merge 后写入
await new IngestionService(redis, prisma, clickhouseWriter, clickhouseClient())
  .mergeAndWrite(
    getClickhouseEntityType(events[0].type),        // trace / observation / score
    job.data.payload.authCheck.scope.projectId,     // 多租户隔离键
    canonicalEntityId,
    firstS3WriteTime,
    events,                                          // 同一 id 的多条事件
    forwardToEventsTable,
  );

第二程:被读 → 被评估

④ 被读(第20-27课)。数据进来了,人要看。trace 出现在列表页——这里只查 ClickHouse 的紧凑表示(窄选列、时间窗、token 分页,第24课),快而省;你点进详情页,才把完整的 observation 树和大块输入输出从 S3 取出来(第25课);会话视图把同一用户的多条 trace 串起来看(第26课);而程序想读,走公共 REST API(第27课,scale-aware 契约)。⑤ 被评估(第28-36课)。光看不够,还要判好坏:一条 LLM-as-judge 评估器读这条 trace 的输入输出、调一个裁判模型给它评分写回 score(第28-31课);监控盯着分数的聚合,越过阈值就发告警(第33课);这条 trace 还可能被加进数据集当测试用例、在实验里和别的 prompt/模型对比(第34-36课)。注意这里出现了一个优雅的闭环:trace 产生 score、score 又能驱动监控与实验,观测的产物反过来改进应用。

既被人看,也被评判——还形成改进闭环 落库的 traceCH 明细 + S3 负载 + PG 元数据 ④ 被读(L20-27)列表:CH 紧凑表示(L24)详情:取 S3 大字段(L25)会话:串同用户(L26)REST:scale-aware 契约(L27)人/程序读它 ⑤ 被评估(L28-36)LLM 裁判评分→score(L28-31)监控盯聚合越阈值告警(L33)加数据集/做实验(L34-36)人工标注补 score(L32) 闭环:score 驱动监控与实验 → 反过来改进 prompt/模型/应用
被读 + 被评估 + 闭环:读路径 L20-27(列表 CH 紧凑表示、详情取 S3、会话、公共 REST);评估 L28-36(LLM-as-judge 写回 score、监控聚合告警、人工标注、数据集/实验)。score 既是观测产物,又反向驱动监控与实验——形成「观测→评估→改进」的闭环。
4

被读:列表快、详情全(L20-27)

列表只查 CH 紧凑表示(窄列/时间窗/token 分页),详情才从 S3 取完整树与大字段;会话串同用户;REST 给程序读。「双存储 + scale-aware 契约 + 成本」主题在此。

5

被评估:判好坏、成闭环(L28-36)

LLM 裁判/人工标注给 trace 评分写回 score;监控盯分数聚合越阈值告警;加数据集做实验。score 反向驱动改进——观测的产物反哺应用。

第三程:被作用 → 退场(与三条隐线)

⑥ 被作用(第44-47课)。这条 trace 不只是躺着被看——它能触发行动。它的某个变化(或它引发的告警)匹配上一个自动化的触发器,就投递一个 webhook(第44课,一路 SSRF 纵深防御)或发到 Slack(第45课);它会被分析集成源源不断地导进你自己的 PostHog/S3(第46课,两级 fan-out + 增量水位);你也可以把它连同一票筛出来的 trace 一起批量导出成 CSV(第47课)。⑦ 退场(第52课)。数据不是永生的:保留期一到,它会被跨三个存储一处不漏地删干净(先 ClickHouse + S3、最后 Postgres,留住重试锚点)。一条 trace 的一生,至此圆满落幕。

而这整段旅程,始终被三条隐线笼罩:① 它从生到死的每一步,都被平台自己的 OTel/日志观测着(第51课,dogfooding——观测工具观测自己);② 它能用到的每个功能,都被这个组织的 plan/entitlement 默默门控(第50课);③ 它的每一次读、写、删,都被严格按 projectId 隔离(多租户),别的租户碰不到。七个驿站串起骨架,三条隐线织成底色——这,就是 Langfuse 这台机器的全貌。

被作用:触发下游行动;退场:跨三存储删干净 落库的 trace 活跃期 ⑥ 被作用(触发下游) webhook(L44,SSRF 纵深) Slack 富消息(L45) 分析集成 → PostHog/S3(L46) 批量导出 CSV(L47) ⑦ 退场(L52)保留期到 跨三存储删除:CH + S3 → PG (PG 留到最后,留重试锚点) 活跃期结束 三条隐线全程笼罩:自观测(L51) · plan 门控(L50) · projectId 隔离
一生的驿站发生了什么课主导主题
① 出生SDK 创建 trace+observation,攒批异步上报L12异步
② 摄取认证→落 S3→Redis 队列→worker 处理L13-19, L49异步·多租户
③ 落库散成三份:CH 明细 + S3 负载 + PG 元数据L13双存储·不可变
④ 被读列表紧凑/详情取大字段/会话/RESTL20-27双存储·成本
⑤ 被评估LLM 裁判评分→score、监控告警、数据集/实验L28-36宽事件(高基数)
⑥ 被作用webhook/Slack、分析导出、批量导出L44-47异步·成本
⑦ 退场保留期到,跨三存储删除(留重试锚点)L52双存储
贯穿自观测 · plan 门控 · projectId 隔离L51·L50·全栈多租户·成本
🎯 设计取舍 · 全书收尾
跟完这条 trace 的一生,你该带走的「那个大问题」是什么? 是这个:当你的应用把一部分判断交给了一个你无法直接读懂、还时常飘忽的 LLM,你怎么重新拿回「看得见、说得清、控得住」? Langfuse 这台看似庞大的机器,从头到尾只是在从容地回答它。把这条 trace 的七个驿站连起来读,答案的形状就浮现了——记录(出生+摄取+落库,把每次调用变成可查的宽事件)→ 找到(被读,在海量里快速定位)→ 评判(被评估,给「好不好」一个可量化的 score)→ 行动(被作用,让结论自动驱动下游),最后还得负责任地退场(删除)、并且这一切要在高规模、多租户、可控成本下成立。你会发现,第54课那六个主题不是抽象口号:宽事件让「记录」保住细节、双存储让「找到」又快又省、异步让「记录与行动」在高吞吐下不塌、多租户让它能服务所有人、成本让它长期养得起。每一个工程取舍,最终都服务于「把 AI 应用的不可观测,变回可观测」这一件事。

而你,读完这 55 课,真正应该带走的不是某段源码,是一种「看系统」的眼光。下次你面对任何一个陌生的复杂系统,别再只问「这个函数干嘛」,而要追问:它的目标和规模前提是什么?数据从哪来、长什么样、到哪去?哪些活儿必须同步、哪些可以异步?哪些状态必须强一致、哪些可以最终一致?它怎么隔离租户、怎么控制成本、怎么观测自己?——这些问题,正是这 55 课反复演练的。Langfuse 只是一个足够真实、足够完整的范本;真正的收获,是你从此能用架构师的眼睛,去看见任何系统表象之下那套彼此印证的取舍。这趟旅程到这里就结束了,但用这双眼睛去读代码的旅程,才刚刚开始。
🎯 本课要点 · 全书终
  • 一条 trace 的七个驿站:出生(L12)→摄取(L13-19,L49)→落库(L13)→被读(L20-27)→被评估(L28-36)→被作用(L44-47)→退场(L52)。每课学的零件,在这条流水线上严丝合缝地咬合。
  • 三条贯穿全程的隐线:自观测(L51 dogfooding)、plan/entitlement 门控(L50)、projectId 多租户隔离(全栈)——七驿站串骨架,三隐线织底色。
  • 一个优雅的闭环:trace 产生 score,score 驱动监控与实验,反过来改进 prompt/模型/应用——观测的产物反哺应用,这是「LLM 工程平台」的灵魂。
  • 六大主题各就各位:宽事件(记录保细节)、不可变+异步(高吞吐扛得住)、双存储(找得快又省)、多租户(服务所有人)、成本(长期养得起)——第54课的六个主题,在这条一生里各司其职。
  • 带走一种眼光:Langfuse 回答的大问题是「把 AI 应用的不可观测变回可观测」——记录→找到→评判→行动→负责任退场。你真正该带走的不是某段源码,而是用「目标+规模→取舍」的架构师眼光去看任何系统的能力。55 课到此为止,但这双眼睛刚刚睁开。

🧪 自测 · 想一想为什么这么设计

1. 这一课跟着一条 trace 走完它的「七个驿站」(出生→摄取→落库→被读→被评估→被作用→退场)。在「摄取」这一站,Langfuse 怎么做到既扛住高吞吐又不丢数据?
  1. 让 SDK 重试直到数据库有空
  2. 「快接收、异步处理」:API 用 API key(两层哈希)认证后,把原始事件落进 S3、任务塞进 Redis 队列就立刻 207 返回(请求路径极薄),真正的校验解析合并由 worker 从队列取出后在后台做、upsert 进 ClickHouse——异步削峰,请求不被重活拖住
  3. 丢弃超过阈值的数据以保证不卡
  4. 把所有数据直接同步写进 ClickHouse 并等待确认
看答案与解析 点击展开
答案:B。摄取这一站是「异步」主题的典范。如果请求路径上同步把数据写进 ClickHouse 并等确认,高峰时数据库一慢,整条上报链路就堵死、SDK 侧超时丢数据。Langfuse 的做法是把请求路径做到极薄:认证 + 把原始事件落进 S3(持久化,不丢) + 把任务塞进 Redis 队列,就立刻返回 207。真正耗时的校验、解析、合并、写 ClickHouse,全交给 worker 异步在后台慢慢做。这样请求侧永远快、永远不被重活拖住(削峰),而数据因为先落了 S3 + 进了队列,即使 worker 一时处理不过来也不会丢(解耦 + 可重试)。这正是第12-19课摄取链路的精髓,也是第54课「异步」主题的第一次亮相。
2. 这一课指出 trace 一生中有一个「优雅的闭环」。这个闭环指的是什么?
  1. trace 被评估产生 score,而 score 又反过来驱动监控告警与实验对比,进而改进 prompt/模型/应用——「观测的产物反哺应用」,观测→评估→改进形成闭环,这正是「LLM 工程平台」(而不只是被动日志收集器)的灵魂
  2. trace 被删除后又被自动重建
  3. trace 在三个存储之间不断循环复制
  4. trace 每次被读都会触发一次重新摄取
看答案与解析 点击展开
答案:A。这个闭环是 Langfuse 之所以叫「LLM 工程平台」而非「日志收集器」的关键。被动的观测系统只是把数据记下来给你看;Langfuse 更进一步:trace 被 LLM 裁判或人工标注评估、产生 score(第28-32课);score 不是终点,它被监控盯着聚合、越阈值发告警(第33课),也被用来在数据集/实验里对比不同 prompt 与模型的优劣(第34-36课);这些反馈最终指导你改进应用——而改进后的应用又产生新的 trace,循环往复。于是「观测」的产物(score)反过来成了「改进」的驱动力,形成 观测→评估→改进→再观测 的闭环。这正是把 AI 应用从「黑盒撞运气」变成「可度量、可迭代」的核心机制。
3. 这一课作为全书终章,反复强调「真正该带走的不是某段源码」。那么按这一课,读完 55 课最该获得的是什么?
  1. 把 Langfuse 的源码全部背下来
  2. 学会照搬 ClickHouse 和 fan-out 队列到任何项目
  3. 记住 55 课每一课的标题
  4. 一种「看系统」的眼光:面对任何复杂系统,会追问它的目标+规模前提、数据从哪来到哪去、什么该同步什么该异步、什么要强一致什么可最终一致、怎么隔离租户控成本观测自己——用「目标+规模→彼此印证的取舍」的架构师视角去理解系统,而非只问「这函数干嘛」
看答案与解析 点击展开
答案:D。全书的终极意图,不是让你成为「Langfuse 源码专家」,而是借由这一个足够真实、足够完整的范本,训练出一种可迁移的架构思维。具体的选型(ClickHouse、BullMQ、fan-out)会过时、会因场景而异,照搬往往是错的;但那套推导方式是通用的:先想清目标与规模前提,再据此推导该用什么数据模型、哪些活儿同步/异步、哪些状态强/最终一致、怎么隔离与控成本。这正是第54课「钉死前提再做减法」、和这一课「用架构师的眼睛看见表象下彼此印证的取舍」想传递的。所以读完 55 课,最该带走的是这双眼睛——下次面对任何陌生系统,你都能穿过实现细节,看见它背后那套自洽的设计逻辑。这趟旅程结束了,但用这双眼睛读代码的旅程才刚开始。
💭 发散思考(没有标准答案,动手或动脑想想)
  • 这是全书的最后一道思考题。回望这条 trace 的一生——出生、摄取、落库、被读、被评估、被作用、退场——它回答的大问题是「把 AI 应用的不可观测,变回可观测」。请你合上这本指南,试着不看答案、用自己的话讲一遍:一条 trace 从你的应用出发,到最终被删除,完整经历了哪些环节?每个环节背后是哪条设计主题在起作用?如果让你给一个朋友用三分钟讲清「Langfuse 到底是怎么工作的、为什么这么设计」,你会怎么讲?(这道题没有标准答案——能讲清楚,就说明这 55 课真的内化成了你的东西。)

For the last lesson, we do something romantic: follow one trace through its whole life. The previous 54 lessons each dissected an organ — this one brings the trace alive, from the millisecond it's born in your app, to being ingested, stored, viewed, scored, triggering automations, and finally taking its bow when the retention period expires. Along the way, we mark each station with "this is the lesson that covered it," so you see with your own eyes how the parts you learned lesson by lesson mesh seamlessly into one complete machine. By the end of this journey, your understanding of Langfuse upgrades from "knowing each part" to "seeing the whole system collaborate." And you'll find this whole machine is, at bottom, calmly answering one question: what did your AI app actually do — and how do you record it, find it, judge it, and act on it?

📋 Analogy
Picture a trace's life as a "parcel." It's born in the sender's hands (your app calls an LLM, the SDK packages that call); it's processed at high speed in a sorting center (the ingestion path: scan, register, sort, shelve); it's properly stored in a warehouse (three stores each hold a part: tracking info, details, oversized items); it's queried and tracked in the system (lists, details, viewing its story on a timeline); it's spot-checked and scored by QC (evaluation and monitoring); it triggers downstream linkage (arrival notice, forwarding, archival); and finally, destroyed when the storage period ends, with all three stores cleaned missing none. Your previous 54 lessons toured the sending desk, the sorter, the shelves, the query counter, the QC station, the notification center, the destruction room separately — this lesson follows the same parcel, stringing these stations into one assembly line, watching them connect end to end to jointly accomplish "making every AI call traceable, assessable, and controllable."

Leg one: born → ingested → stored

① Born (Lesson 12). Your app calls an LLM, the SDK creates a trace and its child observations on the spot, recording input, output, model, usage, batches them up, and async-POSTs to the ingestion API — your main flow barely notices. ② Ingested (Lessons 13-19). The API first authenticates by API key (Lesson 49's two-tier hash check), lands the raw event in S3, pushes the task to a Redis queue, and returns 207 immediately — that's "receive fast." Then the worker pulls from the queue, validates, parses, possibly merges, and upserts into ClickHouse. ③ Stored (Lesson 13). The trace finally splits into three copies to settle: analyzable event detail into ClickHouse (immutable, ReplacingMergeTree dedup-at-query keeping the latest), bulky inputs/outputs into S3, related metadata into Postgres. From birth to storage, Lesson 10's "dual (tri) storage" and Lesson 54's "async, immutability" themes first play in concert here.

A trace's life: seven stations, one assembly line ①bornSDK createsL12 ②ingestS3+queue+workerL13-19/L49 ③storedCH+S3+PGL13 ④readlist/detail/RESTL20-27 ⑤evaluateeval/monitor/datasetL28-36 ⑥act onautomation/exportL44-47 ⑦retirecross-store deleteL52 three hidden threads running throughoutobserved by the platform's own OTel/logs (L51) · features gated by org plan (L50) · every step isolated by projectId (multi-tenancy) the six themes take their places on this assembly linewide events(shape)·immutability(how to write)·async(where work)·dual storage(where stored)·multi-tenancy(whose)·cost(what's worth doing)
A trace's complete lifecycle: born (L12 SDK) → ingested (L13-19 ingestion path + L49 API-key auth) → stored (L13 ClickHouse/S3/Postgres) → read (L20-27) → evaluated (L28-36) → acted on (L44-47) → retired (L52). Three hidden threads throughout: self-observability (L51), plan gating (L50), projectId multi-tenant isolation.
Finale: the life of a trace (subway line) born SDK instrument ① L06 ingest API→S3+Redis→207 ② L12 store worker→ClickHouse ③ L13·17 read list/detail ④ L24 evaluate judge/score ⑤ L29 act monitor/automation ⑥ L44 retire retain/delete ⑦ L52 ↓ three hidden rails running through every station 🔑 projectId multi-tenancy: every row & query scoped by it L06·49 📊 OTel self-observability: every step is itself a trace L51 🎫 plan entitlements: each step gated by it L50 from birth to retirement, a trace is threaded by these three rails — that is Langfuse in full
Connecting the whole book into one line: the life of a trace (lesson numbers for navigation): it is born in your app (SDK instrumentation, L06), ingested (the API lands S3+Redis and returns 207, L12), merged and stored by a worker (ClickHouse, L13/L17), then read (list/detail, L24), evaluated (judge/score, L29), acted on (monitor/automation, L44), and finally retired (cross-store deletion after the retention window, L52). Three hidden rails thread every station: projectId (multi-tenancy, L06/L49) scopes every row and query, OTel (L51) makes each step itself a trace, and plan (entitlements, L50) gates what each step can do. Understand this line and you understand Langfuse.
1

Born: SDK packages on the spot (L12)

The app calls an LLM, the SDK creates trace + observations, records input/output/model/usage, batches and async-POSTs — the main flow barely notices.

2

Ingested: receive fast + process async (L13-19, L49)

The API authenticates by API-key two-tier hash, lands the raw event in S3, pushes to a Redis queue and returns; the worker pulls, validates/parses, upserts into ClickHouse.

3

Stored: split into three to settle (L13)

Event detail→ClickHouse (immutable, merge at query), bulky payloads→S3, metadata→Postgres. The "dual (tri) storage + async + immutability" themes first play in concert.

worker/src/queues/ingestionQueue.tsthe moment it's stored
// What "stored" really is: the worker pops the job, then merges this entity's events and writes them
await new IngestionService(redis, prisma, clickhouseWriter, clickhouseClient())
  .mergeAndWrite(
    getClickhouseEntityType(events[0].type),        // trace / observation / score
    job.data.payload.authCheck.scope.projectId,     // the multi-tenant isolation key
    canonicalEntityId,
    firstS3WriteTime,
    events,                                          // the multiple events for one id
    forwardToEventsTable,
  );

Leg two: read → evaluated

④ Read (Lessons 20-27). The data's in; people want to see it. The trace appears in the list view — which queries only ClickHouse's compact representation (narrow field selection, time windows, token pagination, Lesson 24), fast and cheap; you click into the detail view, and only then are the full observation tree and bulky inputs/outputs fetched from S3 (Lesson 25); the session view strings together one user's multiple traces (Lesson 26); and a program reads it via the public REST API (Lesson 27, a scale-aware contract). ⑤ Evaluated (Lessons 28-36). Seeing isn't enough; you must judge quality: an LLM-as-judge evaluator reads this trace's input/output, calls a judge model to score it, and writes back a score (Lessons 28-31); a monitor watches the aggregate of scores and alerts when crossing a threshold (Lesson 33); the trace may also be added to a dataset as a test case and compared against other prompts/models in experiments (Lessons 34-36). Notice an elegant closed loop here: the trace produces a score, and the score in turn drives monitoring and experiments — the products of observation feed back to improve the app.

Both viewed and judged — and forms an improvement loop the stored traceCH detail + S3 payload + PG metadata ④ read (L20-27)list: CH compact repr (L24)detail: fetch S3 big fields (L25)session: string one user (L26)REST: scale-aware contract (L27)humans/programs read it ⑤ evaluated (L28-36)LLM judge scores→score (L28-31)monitor watches aggregate, alerts (L33)add to dataset / experiment (L34-36)human annotation adds score (L32) loop: score drives monitoring & experiments → in turn improves prompt/model/app
Read + evaluated + closed loop: read path L20-27 (list CH compact representation, detail fetch S3, session, public REST); evaluation L28-36 (LLM-as-judge writes back score, monitor aggregate alerts, human annotation, datasets/experiments). The score is both an observation product and a driver of monitoring and experiments — forming an "observe→evaluate→improve" loop.
4

Read: list fast, detail full (L20-27)

The list queries only CH compact representation (narrow columns/time windows/token pagination), detail fetches the full tree and big fields from S3; session strings one user; REST serves programs. The "dual storage + scale-aware contract + cost" themes are here.

5

Evaluated: judge quality, close the loop (L28-36)

LLM-judge/human annotation scores the trace, writes back; the monitor watches score aggregates and alerts on threshold; add to a dataset for experiments. The score drives improvement in turn — observation products feed the app.

Leg three: acted on → retired (with the three hidden threads)

⑥ Acted on (Lessons 44-47). The trace doesn't just lie there being viewed — it can trigger action. Some change in it (or an alert it raised) matches an automation's trigger, delivering a webhook (Lesson 44, with full SSRF defense in depth) or posting to Slack (Lesson 45); it's continuously exported into your own PostHog/S3 by analytics integrations (Lesson 46, two-level fan-out + incremental watermark); you can also batch-export it together with a filtered set of traces to CSV (Lesson 47). ⑦ Retired (Lesson 52). Data isn't immortal: when the retention period expires, it's deleted across all three stores missing none (ClickHouse + S3 first, Postgres last, keeping the retry anchor). A trace's life thus comes to a complete close.

And this whole journey is always wrapped in three hidden threads: ① every step from birth to death is observed by the platform's own OTel/logs (Lesson 51, dogfooding — the observability tool observing itself); ② every feature it can use is silently gated by the org's plan/entitlement (Lesson 50); ③ its every read, write, delete is strictly isolated by projectId (multi-tenancy), untouchable by other tenants. Seven stations string the skeleton, three hidden threads weave the background — this is the full picture of the Langfuse machine.

acted on: trigger downstream actions; retired: deleted across all three stores stored trace active life 6 acted on (trigger downstream) webhook (L44, SSRF depth) Slack rich message (L45) analytics → PostHog/S3 (L46) batch export to CSV (L47) 7 retired (L52) on retention expiry delete across stores: CH + S3 → PG (PG last, keep the retry anchor) life ends three hidden threads throughout: self-observability (L51) · plan gating (L50) · projectId isolation
Life stationWhat happenedLessonLead theme
① bornSDK creates trace+observation, batches async reportL12async
② ingestedauth→land S3→Redis queue→worker processesL13-19, L49async·multi-tenancy
③ storedsplit into three: CH detail + S3 payload + PG metadataL13dual storage·immutability
④ readlist compact/detail fetches big fields/session/RESTL20-27dual storage·cost
⑤ evaluatedLLM judge scores→score, monitor alerts, datasets/experimentsL28-36wide events(high cardinality)
⑥ acted onwebhook/Slack, analytics export, batch exportL44-47async·cost
⑦ retiredretention expires, cross-store deletion (keep retry anchor)L52dual storage
throughoutself-observability · plan gating · projectId isolationL51·L50·full-stackmulti-tenancy·cost
🎯 Design trade-off · Send-off
Having followed this trace's life, what's "the big question" you should carry away? It's this: when your app hands part of its judgment to an LLM you can't directly read and that often drifts, how do you reclaim "visible, articulable, controllable"? This seemingly-vast Langfuse machine is, from start to finish, calmly answering it. String the trace's seven stations together and the shape of the answer emerges — record (born+ingested+stored, turning each call into a queryable wide event) → find (read, locating fast among the masses) → judge (evaluated, giving "good or not" a quantifiable score) → act (acted on, letting conclusions auto-drive downstream), and finally retire responsibly (deletion) — and all of this must hold under high scale, multi-tenancy, and controlled cost. You'll find Lesson 54's six themes aren't abstract slogans: wide events let "record" keep detail, dual storage lets "find" be fast and cheap, async keeps "record and act" from collapsing under high throughput, multi-tenancy lets it serve everyone, cost lets it stay affordable long-term. Every engineering trade-off ultimately serves one thing: turning an AI app's unobservability back into observability.

And what you should truly carry away from these 55 lessons isn't a snippet of source — it's a way of "seeing systems." Next time you face any unfamiliar complex system, don't just ask "what does this function do," but probe: what are its goal and scale premises? Where does data come from, what shape is it, where does it go? Which work must be sync, which can be async? Which state must be strongly consistent, which can be eventually consistent? How does it isolate tenants, control cost, observe itself? — these are exactly the questions these 55 lessons rehearsed over and over. Langfuse is merely a real-enough, complete-enough specimen; the real gain is that you can now use an architect's eyes to see, beneath any system's surface, the set of mutually-corroborating trade-offs. This journey ends here, but the journey of reading code with these eyes has only just begun.
🎯 Key points · The end
  • A trace's seven stations: born (L12)→ingested (L13-19,L49)→stored (L13)→read (L20-27)→evaluated (L28-36)→acted on (L44-47)→retired (L52). The parts learned per lesson mesh seamlessly on this assembly line.
  • Three hidden threads throughout: self-observability (L51 dogfooding), plan/entitlement gating (L50), projectId multi-tenant isolation (full-stack) — seven stations string the skeleton, three threads weave the background.
  • An elegant closed loop: the trace produces a score, the score drives monitoring and experiments, in turn improving prompt/model/app — observation products feed the app, the soul of an "LLM engineering platform."
  • The six themes take their places: wide events (record keeps detail), immutability+async (survives high throughput), dual storage (find fast and cheap), multi-tenancy (serve everyone), cost (affordable long-term) — Lesson 54's six themes each do their job across this life.
  • Carry away a way of seeing: the big question Langfuse answers is "turn an AI app's unobservability back into observability" — record→find→judge→act→retire responsibly. What you should truly carry away isn't a snippet of source but the ability to see any system with an architect's "goal+scale→trade-offs" eyes. The 55 lessons end here, but these eyes have only just opened.

🧪 Self-test - think about the design

1. This lesson follows a trace through its 'seven stations' (born→ingested→stored→read→evaluated→acted on→retired). At the 'ingested' station, how does Langfuse withstand high throughput without losing data?
  1. make the SDK retry until the database is free
  2. 'receive fast, process async': after the API authenticates by API key (two-tier hash), it lands the raw event in S3, pushes the task to a Redis queue, and returns 207 immediately (a very thin request path); the real validation/parsing/merging is done by the worker pulling from the queue in the background, upserting into ClickHouse — async peak-shaving, the request isn't dragged by heavy work
  3. drop data over a threshold to stay responsive
  4. synchronously write all data straight into ClickHouse and wait for confirmation
Show answer & explanation click to expand
Answer: B. The ingested station is a paragon of the 'async' theme. If the request path synchronously wrote data into ClickHouse and waited for confirmation, then at peak when the DB slows, the whole report chain jams and the SDK side times out and loses data. Langfuse makes the request path razor-thin: authenticate + land the raw event in S3 (persisted, not lost) + push the task to a Redis queue, then return 207 immediately. The truly time-consuming validation, parsing, merging, and ClickHouse writes are all handed to the worker to do slowly in the background. So the request side is always fast, never dragged by heavy work (peak-shaving), and since data first landed in S3 + entered the queue, even if the worker can't keep up momentarily nothing is lost (decoupling + retryable). This is the essence of Lessons 12-19's ingestion path and the 'async' theme's first appearance from Lesson 54.
2. This lesson points out an 'elegant closed loop' in a trace's life. What does this loop refer to?
  1. the trace is evaluated to produce a score, and the score in turn drives monitoring alerts and experiment comparisons, thereby improving the prompt/model/app — 'observation products feed the app', observe→evaluate→improve forms a loop, which is the soul of an 'LLM engineering platform' (not just a passive log collector)
  2. the trace is automatically rebuilt after deletion
  3. the trace continuously cycles and replicates among the three stores
  4. every read of the trace triggers a re-ingestion
Show answer & explanation click to expand
Answer: A. This loop is the key to why Langfuse is an 'LLM engineering platform' not a 'log collector'. A passive observability system just records data for you to view; Langfuse goes further: a trace is evaluated by an LLM judge or human annotation, producing a score (Lessons 28-32); the score isn't the end — it's watched by a monitor's aggregate, alerting on threshold (Lesson 33), and used in datasets/experiments to compare different prompts and models (Lessons 34-36); this feedback ultimately guides you to improve the app — and the improved app produces new traces, round and round. So the 'observation' product (score) in turn becomes the driver of 'improvement', forming an observe→evaluate→improve→observe-again loop. This is the core mechanism turning an AI app from 'black-box luck' into 'measurable, iterable'.
3. As the whole guide's finale, this lesson repeatedly stresses 'what you should truly carry away isn't a snippet of source'. So per this lesson, what's the most important thing to gain from the 55 lessons?
  1. memorizing all of Langfuse's source code
  2. learning to copy ClickHouse and fan-out queues into any project
  3. remembering every one of the 55 lesson titles
  4. a way of 'seeing systems': facing any complex system, you probe its goal+scale premises, where data comes from and goes, what should be sync vs async, what needs strong vs eventual consistency, how it isolates tenants/controls cost/observes itself — understanding systems with an architect's 'goal+scale→mutually-corroborating trade-offs' lens, not just asking 'what does this function do'
Show answer & explanation click to expand
Answer: D. The guide's ultimate intent isn't to make you a 'Langfuse source expert' but, through this real-enough, complete-enough specimen, to train a transferable architectural mindset. Specific choices (ClickHouse, BullMQ, fan-out) date and vary by context; copying them is often wrong. But the way of deriving is universal: first get clear on goal and scale premises, then derive what data model to use, what work is sync/async, what state is strong/eventually consistent, how to isolate and control cost. This is exactly what Lesson 54's 'nail premises then subtract' and this lesson's 'use an architect's eyes to see the mutually-corroborating trade-offs beneath the surface' aim to convey. So after 55 lessons, what you should most carry away is these eyes — next time you face any unfamiliar system, you can see past the implementation details to the self-consistent design logic behind it. This journey ends, but the journey of reading code with these eyes has only just begun.
💭 Open questions (no single right answer - just think or try)
  • This is the guide's final reflection. Look back on this trace's life — born, ingested, stored, read, evaluated, acted on, retired — answering the big question 'turn an AI app's unobservability back into observability'. Close this guide and, without looking at the answer, retell it in your own words: from leaving your app to finally being deleted, what complete sequence of stages does a trace go through? Which design theme is at work behind each stage? If you had three minutes to explain to a friend 'how Langfuse actually works and why it's designed this way', how would you tell it? (There's no standard answer here — being able to explain it clearly means these 55 lessons have truly become your own.)