前三课讲的是「数据长什么样」,这一课换个维度:代码住在哪里。Langfuse 是一个 monorepo(单仓库多工程),
所有东西都在 langfuse/langfuse 一个仓库里,但内部分成几个各司其职的「工作区」。把这张地图记在心里,后面任何一课说「打开
web/src/...」或「这段逻辑在 worker 里」,你都能立刻知道它在哪、为什么在那。
.agents/AGENTS.md 依赖方向规则):箭头永远指向 @langfuse/shared 这个「窄腰」——web、worker、@langfuse/ee 都依赖它,而它不反向依赖任何业务包。空白格=不允许 import,对角线=自身。这把「谁能用谁」从口头约定变成一眼可查的结构。稍微展开一下两个运行时各自在忙什么,你会更有体感。web 是那个你浏览器里打开的 Next.js 应用,它干三件事:渲染界面、
给界面提供类型安全的内部 API(tRPC,第 21 课)、以及给外部 SDK 提供公共 REST API(web/src/pages/api/public,第 27 课)——
摄取事件的入口就在这里。worker 则是个没有界面的后台进程,它订阅几十个 BullMQ 队列,把所有重活、慢活、可重试的活都揽下来:
把摄取的事件合并写进 ClickHouse、跑 LLM 评估、导出数据、删除项目……(第三部分起逐个讲)。一句话:web 负责「快进快出」,worker 负责「埋头苦干」,
两者通过中间的 Redis 队列解耦——这也正是下一课「双存储」和第三部分「摄取链路」要展开的主线。
Langfuse 用 pnpm workspaces 把一个仓库切成几个独立可构建的包。根目录的 pnpm-workspace.yaml 一行行列出了它们:
packages: - "web" # Next.js 应用(UI + tRPC + 公共 REST API) - "worker" # 队列消费者,后台处理 - "packages/**" # 共享包,主要是 packages/shared - "ee" # 企业版功能
packages/** 下其实不止 shared 一个,还有 config-eslint、config-typescript、
eslint-plugin 这些「工具配置包」;但真正承载业务的共享内核是 packages/shared——它就是上图里的中央仓库。
为什么要拆成工作区而不是一个大文件夹?因为这样每个包能独立声明依赖、独立构建、独立测试,pnpm 还能在它们之间做符号链接,
让 web 直接 import ... from "@langfuse/shared" 用上最新代码,不必发版。
这点对开发体验影响很大:你在 packages/shared 里改一个类型,web 和 worker 的编辑器会立刻飘红提示哪里要跟着改——
等于把「契约变更」的影响在编码阶段就暴露出来,而不是等到运行时才报错。
四个工作区之间不是随便互相引用的,而是有一条严格单向的依赖规则(写在 .agents/AGENTS.md 里):
- web -> @langfuse/shared, @langfuse/ee - worker -> @langfuse/shared - ee -> @langfuse/shared - @langfuse/shared -> no imports from web, worker, or ee
这条纪律为什么重要?因为 web 和 worker 是两种完全不同的负载:web 要面向用户、低延迟、可随时重启;worker 要啃重活、可重试、按队列伸缩。
如果让它们互相直接引用,改一个就可能牵连另一个,部署也会绑死。把共享的东西收进 shared、并规定只能向下依赖,
就等于在两个容器之间立了一道清晰的契约墙:队列里传什么、数据库表长什么样,都由 shared 这一处定义(比如队列负载就归
packages/shared/src/server/queues.ts 管)。两边照着同一份契约各干各的,互不踩脚。
举个这条规则挡住的真实 bug:假设没有窄腰,worker 里图省事直接 import 了 web 的某个工具函数。某天 web 为了改界面重构了那个函数、
顺手改了它的参数——结果 worker 在毫不知情的情况下被改坏了,可能要到线上摄取出错才发现。而在窄腰规则下,worker 根本不允许依赖 web;
两边唯一的「共同语言」是 shared 里那份明确的契约。想共享逻辑?那就把它下沉到 shared,让它成为受契约约束的一等公民,而不是从隔壁悄悄借用。
「依赖只能向下」这一条,本质是在用编译器帮你挡住跨容器的隐式耦合。
多个工作区的构建、测试、lint 由 Turbo 统一编排(根目录 turbo.json)。它最有用的两件事是任务依赖图和缓存:
看 turbo.json 里 build 任务的定义:"dependsOn": ["db:generate", "^build"]——意思是「构建任何一个包之前,
先生成数据库客户端、并先把它依赖的包(^ 表示上游依赖)构建好」。于是你只要 pnpm build,Turbo 就会自动算出正确顺序、
把能并行的并行、能缓存的跳过("cache": true)。这就是为什么在一个有几十个包的大仓库里,改一行代码也不必全量重建。
把窄腰和 Turbo 合起来,看一个具体的连锁反应:假设你要给 trace 加一个字段。你只需在
packages/shared/prisma/schema.prisma 改一处、再在 domain/traces.ts 改一处——一次 db:generate 重新生成
Prisma 客户端,web 和 worker 立刻都看到新字段,因为它们都从同一个 @langfuse/shared 取类型。整件事在一个 PR 里完成,
CI 一起跑通。要是分成三个仓库,这同一个改动得发三次版、改三处依赖、还要祈祷三边版本对得上——这就是「原子地一起改」的实际意义。
web/worker/shared 各一个仓。改一个共享结构 → 跨仓发版、对版本、易漂移。好处是各仓更小、更独立。
一个仓装下全部。共享改动一个 PR 原子完成、CI 一起验。代价:仓更大、要靠 Turbo 管构建缓存。
缓存这件事对持续集成(CI)尤其值钱。一个几十包的大仓,如果每次 PR 都把所有包从头 build + lint + test 一遍,CI 会慢到没法用。
Turbo 会给每个任务的输入算一个指纹(源码 + 依赖 + 配置),只要指纹没变,就直接复用上次的输出、整段跳过。于是改了 worker 的一个文件,
web 那些没被波及的构建/测试就命中缓存、秒过。配上远端缓存,团队成员之间还能共享构建结果。这就是大仓也能保持 CI 快的秘诀——
也是「用 monorepo」这个选择能成立的工程前提:没有好的缓存,monorepo 的构建成本会劝退所有人。
最后给你一张「高频目录速查表」,后面的课会反复回到这些位置:
| 位置 | 放什么 |
|---|---|
| web/src/pages | Next.js 页面 + API 路由(含 api/public 公共 REST、api/trpc) |
| web/src/server/api | tRPC 的 root 路由、context、中间件(第 21 课) |
| web/src/features | 按功能切分的「纵切片」目录(traces、evals、prompts…约几十个) |
| worker/src/queues | 各个 BullMQ 队列的消费者(ingestion、eval、export…) |
| worker/src/services | 核心服务(IngestionService、ClickhouseWriter) |
| packages/shared/src/domain | 领域模型(traces/observations/scores…) |
| packages/shared/src/server | 仓储层、ClickHouse/Redis/S3 客户端、队列契约 |
| packages/shared/prisma | Postgres schema 与迁移 |
| packages/shared/clickhouse | ClickHouse 迁移(宽事件表) |
表里没列、但值得单独一提的是 fern/ 和 generated/。Langfuse 的公共 REST API 不是手写的,而是先在 fern/ 里用
Fern 把「有哪些端点、收什么参数、返回什么」声明出来,再自动生成 generated/ 下的 API 客户端代码。
所以 generated/ 是构建产物,绝不能手改——改了下次生成就被覆盖。这也是为什么改公共 API 时要同步更新 fern/ 源(第 27 课)。
这种「契约先行、代码生成」的做法,保证了 API 文档、服务端、各语言 SDK 三者始终对得上。带着这张全景地图,下一课我们就深入支撑这一切的双存储:为什么偏要 Postgres 加 ClickHouse 一起上。
web(门店)、worker(后场车间)两个运行时 + packages/shared(中央仓库)+ ee。web/worker/ee → shared,而 shared 谁都不依赖;契约(领域模型、队列)集中在 shared。build 依赖 db:generate + ^build)+ 缓存,让大仓也能快速、增量构建。web/src/{pages,server,features}、worker/src/{queues,services}、packages/shared/src/{domain,server}——这是后面查代码的速查表。
The first three lessons covered "what the data looks like"; this one switches axes: where the code lives. Langfuse
is a monorepo (one repo, many projects) — everything sits in the single langfuse/langfuse repo, split
internally into a few purpose-built "workspaces". Hold this map in mind and whenever a later lesson says "open
web/src/..." or "this logic is in worker", you'll instantly know where it is and why.
.agents/AGENTS.md dependency rule): arrows always point toward the @langfuse/shared "narrow waist" — web, worker and @langfuse/ee all depend on it, while it depends back on no business package. Blank cell = not allowed; diagonal = self. This turns "who may use whom" from a verbal convention into a structure you can check at a glance.Unpacking what each runtime actually does makes it tangible. web is the Next.js app you open in your browser; it does
three things: render the UI, serve that UI a type-safe internal API (tRPC, L21), and serve external
SDKs a public REST API (web/src/pages/api/public, L27) — the ingestion entry point lives here.
worker is a headless background process that subscribes to dozens of BullMQ queues and takes on all
the heavy, slow, retryable work: merge ingested events into ClickHouse, run LLM evals, export data, delete projects…
(covered from Part 3 on). In a line: web does "fast in, fast out", worker does "heads-down grind", decoupled by a
Redis queue in the middle — exactly the through-line of the next lesson ("dual store") and Part 3 ("the ingestion path").
Langfuse uses pnpm workspaces to slice one repo into independently buildable packages. The root
pnpm-workspace.yaml lists them line by line:
packages: - "web" # Next.js app (UI + tRPC + public REST API) - "worker" # queue consumer, background processing - "packages/**" # shared packages, mainly packages/shared - "ee" # enterprise features
Under packages/** there's more than just shared — also "tooling config" packages like
config-eslint, config-typescript, eslint-plugin; but the real business shared core is
packages/shared — the central warehouse in the figure. Why split into workspaces instead of one big folder?
Because each package can then declare deps, build and test independently, and pnpm symlinks them so
web can simply
import ... from "@langfuse/shared" against the latest code with no publishing step. This hugely helps the dev experience:
change a type in packages/shared and the editors in web and worker light up red
immediately showing what must follow — surfacing the impact of a "contract change" at coding time rather than
at runtime.
The four workspaces don't reference each other freely — there's a strictly one-way dependency rule (written in
.agents/AGENTS.md):
- web -> @langfuse/shared, @langfuse/ee - worker -> @langfuse/shared - ee -> @langfuse/shared - @langfuse/shared -> no imports from web, worker, or ee
Why does this discipline matter? Because web and worker are two very different workloads: web faces users, wants
low latency, must restart anytime; worker chews heavy jobs, must be retryable, scales by queue. Let them import each other
directly and changing one can entangle the other, binding their deployments. Collecting shared things into shared and
allowing only downward dependencies erects a clean contract wall between the two containers: what
goes through queues, what the DB tables look like — all defined in one place (queue payloads, for instance, belong to
packages/shared/src/server/queues.ts). Both sides work to the same contract without stepping on each other.
A concrete bug this rule blocks: imagine no narrow waist, and worker lazily imports a utility function
from web. One day web refactors that function to change the UI and tweaks its parameters — and worker breaks without anyone
knowing, maybe surfacing only as a production ingestion error. Under the narrow-waist rule, worker simply cannot
depend on web; their only "shared language" is the explicit contract in shared. Want to share logic? Sink it down into
shared so it becomes a contract-bound first-class citizen, not something borrowed quietly from next door. "Dependencies only go
down" is really using the compiler to block implicit cross-container coupling.
Build, test and lint across workspaces are orchestrated by Turbo (root turbo.json). Its two most
useful tricks are the task dependency graph and caching:
Look at the build task in turbo.json: "dependsOn": ["db:generate", "^build"] — meaning "before
building any package, generate the DB client and first build the packages it depends on (^ = upstream deps)". So a single
pnpm build lets Turbo compute the right order, parallelize what it can, and skip what's cached
("cache": true). That's why, in a big repo of dozens of packages, changing one line doesn't force a full rebuild.
Put the narrow waist and Turbo together and watch a concrete chain reaction: say you add a field to the trace. You
edit one spot in packages/shared/prisma/schema.prisma and one in domain/traces.ts — one
db:generate regenerates the Prisma client and both web and worker immediately see the new field, because
both pull types from the same @langfuse/shared. The whole thing lands in one PR, verified together by CI.
Split into three repos and this same change needs three releases, three dependency bumps, and a prayer that versions line up — that's
the practical meaning of "change atomically".
web/worker/shared each a repo. Change a shared structure → cross-repo releases, version juggling, easy drift. Upside: smaller, more independent repos.
one repo holds it all. A shared change is one atomic PR, verified together by CI. Cost: bigger repo, Turbo to manage build caching.
Caching is especially valuable for CI. In a dozens-of-packages repo, rebuilding + linting + testing everything on
every PR would make CI unusably slow. Turbo computes a fingerprint of each task's inputs (source + deps + config) and,
if the fingerprint is unchanged, reuses last time's output and skips the whole step. So editing one file in
worker lets the unaffected web builds/tests hit cache and pass instantly. With a remote
cache, teammates even share build results. That's the secret to keeping CI fast in a big repo — and the engineering
precondition that makes "use a monorepo" viable: without good caching, monorepo build cost scares everyone off.
Finally, a "hot directory cheat-sheet" — later lessons return to these spots again and again:
| Location | What's there |
|---|---|
| web/src/pages | Next.js pages + API routes (incl. api/public REST, api/trpc) |
| web/src/server/api | tRPC root router, context, middleware (L21) |
| web/src/features | feature "vertical slices" (traces, evals, prompts… dozens) |
| worker/src/queues | BullMQ queue consumers (ingestion, eval, export…) |
| worker/src/services | core services (IngestionService, ClickhouseWriter) |
| packages/shared/src/domain | domain models (traces/observations/scores…) |
| packages/shared/src/server | repositories, ClickHouse/Redis/S3 clients, queue contracts |
| packages/shared/prisma | Postgres schema & migrations |
| packages/shared/clickhouse | ClickHouse migrations (wide-event tables) |
Not in the table but worth a separate mention: fern/ and generated/. Langfuse's public REST API isn't
hand-written — it's first declared in fern/ with Fern ("which endpoints, what params,
what responses"), then API client code is generated into generated/. So generated/ is a
build artifact you must never hand-edit — edits get overwritten on the next generation. That's why changing the
public API means updating the fern/ sources too (L27). This "contract-first, code-generated" approach keeps the API docs,
the server, and the per-language SDKs always in sync.
web (storefront), worker (workshop) two runtimes + packages/shared (central warehouse) + ee.web/worker/ee → shared, and shared depends on none; contracts (domain models, queues) live in shared.build depends on db:generate + ^build) + caching, so even a big repo builds fast and incrementally.web/src/{pages,server,features}, worker/src/{queues,services}, packages/shared/src/{domain,server}.