🪢 Langfuse 图解教程Langfuse Visual Guide 第二部分 · 前置基础Part 2 · Foundations 10 / 55
第二部分 · 前置基础Part 2 · Foundations

多租户:org → project → environmentMulti-tenancy

🧭 读前Before you start
先读 第9课 的元数据 schema,本课讲 org → project → environment 如何隔离。
Read Lesson 9 on the metadata schema; this lesson covers org → project → environment isolation.

上一课说 project 是枢纽、是「数据边界」。这一课把这件事讲透:Langfuse 是个多租户平台——同一套部署要同时服务成千上万个互不相干的团队, 它靠三层嵌套把大家的数据干净地隔开:organization(组织)→ project(项目)→ environment(环境)。 理解这三层,你就明白了「为什么你只看得到自己的数据」「为什么 ClickHouse 排序键非要以 project_id 打头」。

🔌 生活类比
把多租户想成一栋写字楼:organization 是租下楼层的公司——签合同、付账单、管自己的员工(成员与计费在这一层); project 是公司里的一个部门,有独立的办公区和门禁,部门的东西不会跑到别的部门去;environment 则是部门内部再隔出的 「正式区 / 测试区」——同一个部门的人,把线上数据和测试数据分开摆,互不干扰。大家可能同在一栋楼(同一套部署、同一个数据库), 但靠层层门禁谁也串不了门。
三层嵌套:组织 → 项目 → 环境 Organization · 计费 + 成员(合同主体) Project A · 数据边界 env: production env: staging env: development 一个 project 内可有多个 environment Project B · 另一个数据边界 env: production env: default Project B 看不到 Project A 的任何数据
三层嵌套:一个 organization 下有多个 project,每个 project 内可再分多个 environment(如 production / staging / development,默认 default)。组织管计费与成员,project 是硬数据边界,environment 是项目内的软切片。
org ⊃ project ⊃ environment:一层套一层 Organization · org_xY7计费 + 成员的顶层 Project · proj_ab3(硬数据边界)trace/observation/score 全挂在它下 environmentproduction软切片:普通过滤列 environmentstaging加 WHERE 才分,不加就合 隔离强度 project_id焊进 CH 排序键前缀→ 结构性硬隔离跨 project 绝不可见 environment普通 LowCardinality 列→ 软切片(可选)
两种隔离强度(实体取自 prisma/schema.prisma;值为示例):Organization 套 Project 套 environment。但二者强度不同——project_id 被焊进 ClickHouse 排序键前缀(结构性硬隔离,跨 project 绝不可见,呼应第 10 课 warn 卡);environment 只是普通过滤列,是软切片,查询加 WHERE 才分。

三层各管什么

层级管什么隔离强度
Organization 组织计费、成员、SSO、组织级 API key最外层的「账户」边界
Project 项目实际数据归属:trace/observation/score、prompt、配置全挂在某 project 下硬隔离:跨 project 互不可见
Environment 环境同一 project 内区分 production / staging / development 等软切片:同 project 内可一起查、也可按环境过滤

关键区别在最后两层:project 是「硬边界」——你拿着某 project 的 key,根本看不到别的 project 的数据; 而 environment 是「软切片」——它在同一个 project 内部,把线上和测试的数据贴上标签分开,你既能只看 production,也能把几个环境放一起比。 为什么这么设计?因为「不同团队的数据绝不能串」是安全红线(硬隔离),而「线上 vs 测试」只是同一团队自己的视图偏好(软切片),强度不同,待遇也不同。

举个具体场景体会这三层。一家公司(organization)在 Langfuse 上签约、付费、邀请了 5 名工程师做成员。他们开了两个项目(project): 「客服机器人」和「文档问答」。这两个项目的数据互不可见——查客服机器人时绝不会混进文档问答的 trace。在「客服机器人」项目内,他们又用 environment 把线上流量(production)和压测/调试流量(staging)分开,于是看线上成本时不会被压测数据污染, 但需要时也能把两个环境放一起对比。最外层的「公司」只关心账单和谁是成员,不直接持有数据;数据全都落在某个 project + environment 上。 这套「组织管钱和人、项目管数据、环境管视图」的分工,就是多租户的全部骨架。

project_id 是硬隔离键,贯穿始终

多租户隔离不是事后加一个 WHERE project_id = ? 过滤这么简单。在 Langfuse 里,project_id 是一条贯穿全链路的主线:

API key解析出 project_id 写入每行都带 project_id ClickHouseORDER BY project_id 打头 查询只扫该 project 区间 project_id:一把钥匙,串起鉴权·写入·存储·查询 隔离不是“查时过滤”,而是把 project_id 焊进了排序键——隔离即定位,定位即高效
project_id 贯穿全链路:鉴权时从 API key 解析出 project_id(第 9 课 key 属于某 project);写入时每行都带上它;ClickHouse 的排序键以 project_id 打头(第 8 课);查询时据它定位到连续区间。所以隔离不是事后过滤,而是写进了物理排布——隔离和「查得快」是同一件事。

这条「project_id 贯穿全链路」的设计,最大的价值是把安全从「靠程序员记得写过滤」变成「靠结构保证」。想想看:如果隔离只靠每个查询都记得加 WHERE project_id = ?, 那只要有一处查询忘了写、或写错了,就可能让 A 公司看到 B 公司的数据——这种 bug 在多租户系统里是灾难级的。Langfuse 的做法是把 project_id 焊进存储与鉴权的底层: 仓储层的读取 API 统一要求传入 project_id,tRPC 的 protectedProjectProcedure 在中间件里先校验你属不属于这个 project(第 21 课)。 于是「越权看别人数据」不是「容易写错」,而是压根没有那条路。这就是「安全做进结构」与「安全靠自觉」的本质区别。

⚠️ 多租户最该警惕的坑
靠「每个查询都记得加 WHERE project_id = ?」来隔离,只要一处漏写或写错,A 公司就可能看到 B 公司的数据——多租户系统里的灾难级 bug。Langfuse 的对策是把 project_id 焊进排序键前缀与鉴权中间件:越权这条路压根不存在,靠结构而非靠自觉。

environment:项目内的软切片

environment 是后来加上的一层。看它在 ClickHouse 里的样子——三张宽事件表都加了一个 environment 列:

clickhouse/migrations/unclustered/0008_add_environments_column.up.sqlALTER
ALTER TABLE traces       ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
ALTER TABLE observations ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
ALTER TABLE scores       ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;

三个细节值得注意:① 类型是 LowCardinality(String)——环境名翻来覆去就那么几个(prod/staging/dev),低基数编码又省空间又查得快; ② 默认值 'default'——你不显式指定环境时,数据自动落到 default,向后兼容(老数据天然属于 default);③ 位置 AFTER project_id—— 这只是让它在表结构里紧挨 project_id 排列(纯粹是列的书写顺序,便于阅读),注意它并没有进入排序键。所以和 project_id 不同,environment 本质是行上的一个 普通过滤列,也就是软切片:想按环境过滤就在查询里加个条件,不想分就忽略它。

这正好对照出 project 与 environment 隔离机制的不同:project_id 焊进了排序键(隔离即定位,结构性安全),而 environment 只是一个普通列(查询时过滤)。 这也合理——硬隔离要靠结构保证,软切片只是个方便的视图维度,不必、也不该动用排序键这种重武器。

同一行两种隔离:project_id 硬隔离(排序键)vs environment 软切片(过滤列) project_id environment timestamp …宽事件列 … project_id = 硬隔离 焊进 ORDER BY (project_id, toDate(ts), id) → 跨 project 绝不可见(结构性安全) 隔离即定位,无法被某个查询绕过 environment = 软切片 LowCardinality DEFAULT 'default',不进排序键 → WHERE environment='prod' 才分 普通过滤列:想分就加条件,不分就忽略

🔒 project = 硬隔离

安全红线:不同 project 的数据绝不可见。焊进排序键前缀 + API key 归属,从存储到鉴权层层保证。

🏷️ environment = 软切片

视图偏好:同一 project 内给数据贴 prod/staging/dev 标签,可分可合,只是查询时的一个过滤维度。

一次请求怎么定位租户

1

带上凭证

SDK 用 API key(写入),或用户登录后带 session(读取)发起请求。

2

解析出 project

服务端从 key/session 解析出「这是哪个 project」,并校验是否有权限(第 9 课 ApiKey、第 21 课 protectedProjectProcedure)。

3

所有读写都锁定该 project

写入时 project_id 落到每行;查询时 project_id 进排序键前缀。你没有办法越过它看到别的 project。

4

可选:按 environment 再切

在该 project 内,按需用 environment 过滤出 prod / staging。

🎯 设计取舍
为什么把 project_id 焊进排序键,而不是当成一个普通的过滤列? 因为这样「隔离」和「高效」就成了同一件事。如果 project_id 只是个普通列, 那「只看我的项目」就是一次全表过滤,海量数据下既慢又危险(万一过滤写漏了,就串了租户)。把它放进排序键最前面,同一 project 的数据物理相邻, 「只看我的项目」退化成「扫一段连续区间」——又快又结构性地不可能看到别人。代价是排序键一旦定了就难改、且要求几乎所有查询都带上 project_id(这正是仓储层 API 的统一约定)。 把多租户隔离做成存储结构的一部分,而不是一道可能写漏的过滤。
🎯 本课要点
  • 三层嵌套:organization(计费/成员)⊃ project(数据边界)⊃ environment(项目内 prod/staging/dev)。
  • project = 硬隔离(跨项目绝不可见),environment = 软切片(同项目内可分可合的标签)。
  • project_id 贯穿全链路:从 API key 解析 → 写入每行 → ClickHouse 排序键打头 → 查询定位。隔离不是事后过滤,而是焊进排序键。
  • environment 列:LowCardinality(String) DEFAULT 'default' AFTER project_id(第 8 课迁移 0008),低基数、向后兼容、紧贴 project_id。
  • 取舍:把隔离做进存储结构 → 隔离与高效合一、且结构性安全;代价是排序键难改、查询须带 project_id。

🧪 自测 · 想一想为什么这么设计

1. Langfuse 的多租户隔离,project 和 environment 的本质区别是什么?
  1. project 是硬隔离(跨项目数据绝不可见,安全红线),environment 是软切片(同一项目内给 prod/staging/dev 贴标签,可分可合)
  2. environment 比 project 隔离更强
  3. 两者完全一样,只是名字不同
  4. project 是软的,environment 是硬的
看答案与解析 点击展开
答案:A。“不同团队数据绝不能串”是安全红线 → project 硬隔离;“线上 vs 测试”只是同团队的视图偏好 → environment 软切片。强度不同,待遇不同。
2. 为什么说 Langfuse 的租户隔离“不是事后加一个 WHERE project_id 过滤”?
  1. 因为 project_id 是随机的
  2. 因为每个项目用独立的数据库实例
  3. 因为 Langfuse 不支持过滤
  4. 因为 project_id 被焊进 ClickHouse 排序键最前面:同项目数据物理相邻,“只看我的项目”退化成扫一段连续区间——隔离与高效合一,且结构性地不可能看到别人
看答案与解析 点击展开
答案:D。若 project_id 只是普通过滤列,海量下既慢又危险(写漏过滤就串租户)。放进排序键最前,隔离变成“定位连续区间”,又快又结构性安全。代价:排序键难改、查询须带 project_id。
3. ClickHouse 里 environment 列定义为 LowCardinality(String) DEFAULT 'default' AFTER project_id。这三点各有什么用意?
  1. 为了删除 project_id
  2. 纯粹为了占位,没有用意
  3. 为了让环境名可以无限多
  4. 低基数:环境名就那么几个,省空间快查询;默认 default:不指定时自动归类、向后兼容老数据;紧跟 project_id:与之构成“项目+环境”定位前缀
看答案与解析 点击展开
答案:D。LowCardinality 适合取值少的列(编码省空间、查得快);DEFAULT 'default' 让老数据/未指定环境的数据天然归到 default(向后兼容);AFTER project_id 让二者构成定位前缀,按“项目+环境”查同样高效。
💭 发散思考(没有标准答案,动手或动脑想想)
  • “把多租户隔离做进存储结构,而不是一道可能写漏的过滤”——这条思路你能用到自己的系统吗?把租户键放进主键/排序键最前,会给你带来什么好处和什么约束?

Last lesson called project the hub and the "data boundary". This lesson makes that concrete: Langfuse is a multi-tenant platform — one deployment serves thousands of unrelated teams at once, isolating their data cleanly via three nesting levels: organization → project → environment. Grasp these three and you understand "why you only see your own data" and "why the ClickHouse ordering key must start with project_id".

🔌 Analogy
Picture multi-tenancy as an office building: the organization is the company renting a floor — signs the lease, pays the bills, manages its staff (members and billing live here); the project is a department with its own area and access control, whose stuff never wanders into another department; the environment is a further "production / test area" split inside the department — the same people keep live and test data apart. Everyone may be in one building (one deployment, one database), but layered access controls keep them from crossing over.
three nesting levels: org → project → environment Organization · billing + members (the lease holder) Project A · data boundary env: production env: staging env: development a project may have multiple environments Project B · another data boundary env: production env: default Project B sees none of Project A's data
Three nesting levels: one organization holds multiple projects, each project may split into multiple environments (e.g. production / staging / development, default default). The org manages billing and members, the project is the hard data boundary, and the environment is a soft slice within a project.
org ⊃ project ⊃ environment: nested boundaries Organization · org_xY7top of billing + membership Project · proj_ab3 (hard data boundary)all trace/observation/score hang under it environmentproductionsoft slice: plain filter column environmentstagingsplit only if you add WHERE isolation strength project_idwelded into CH sort-key prefix→ structural hard isolationnever visible across projects environmentplain LowCardinality column→ soft slice (optional)
Two strengths of isolation (entities from prisma/schema.prisma; values illustrative): Organization wraps Project wraps environment. But their strength differs — project_id is welded into the ClickHouse sort-key prefix (structural hard isolation, never visible across projects, echoing the Lesson 10 warn card); environment is just a filter column, a soft slice split only when a query adds WHERE.

What each level governs

LevelGovernsIsolation strength
Organizationbilling, members, SSO, org-level API keysthe outer "account" boundary
Projectwhere data actually belongs: trace/observation/score, prompts, config all hang under a projecthard isolation: invisible across projects
Environmentdistinguishes production / staging / development within one projectsoft slice: queryable together or filtered by env

The key distinction is the last two: project is a "hard boundary" — hold one project's key and you simply cannot see another project's data; environment is a "soft slice" — within one project it tags live vs test data apart, so you can view only production, or compare several environments together. Why? Because "different teams' data must never mix" is a security red line (hard isolation), while "live vs test" is just one team's own view preference (soft slice) — different strengths, different treatment.

A concrete scenario for all three levels. A company (organization) signs up and pays on Langfuse, inviting 5 engineers as members. They open two projects (project): "support bot" and "doc Q&A". The two projects' data is mutually invisible — querying the support bot never mixes in doc-Q&A traces. Within "support bot" they use environment to separate live traffic (production) from load-test/debug traffic (staging), so live cost isn't polluted by load-test data — yet they can compare the two environments together when needed. The outer "company" cares only about the bill and who's a member, holding no data directly; all data lands on some project + environment. This "org manages money and people, project manages data, environment manages views" division is the entire skeleton of multi-tenancy.

project_id is the hard isolation key, threaded throughout

Multi-tenant isolation is not as simple as adding a WHERE project_id = ? filter afterward. In Langfuse, project_id is a thread running through the whole pipeline:

API keyresolves project_id writeevery row carries project_id ClickHouseORDER BY project_id first queryscan this project's range project_id: one key threading auth · write · storage · query isolation isn't "filter at query time" — project_id is welded into the ordering key: isolate = locate = fast
project_id threads the whole pipeline: auth resolves project_id from the API key (L09: a key belongs to a project); writes stamp it on every row; ClickHouse's ordering key starts with project_id (L08); queries locate the contiguous range by it. So isolation is not an afterthought filter — it's baked into the physical layout; isolation and "fast" are the same thing.

This "project_id threads the whole pipeline" design's biggest value is turning security from "the programmer remembers to filter" into "the structure guarantees it". Consider: if isolation relied on every query remembering to add WHERE project_id = ?, then one forgotten or wrong filter could let company A see company B's data — a catastrophic bug in a multi-tenant system. Langfuse instead welds project_id into the storage and auth foundations: the repository read APIs uniformly require a project_id, and tRPC's protectedProjectProcedure middleware checks first whether you belong to this project (L21). So "see another tenant's data by accident" isn't "easy to get wrong" — there simply is no such path. That's the essential difference between "security in the structure" and "security by discipline".

⚠️ The trap to watch in multi-tenancy
Relying on "every query remembering to add WHERE project_id = ?" means one forgotten or wrong filter lets company A see company B's data — a catastrophic multi-tenant bug. Langfuse instead welds project_id into the sort-key prefix and the auth middleware: the cross-tenant path simply doesn't exist — structure, not discipline.

environment: a soft slice within a project

environment was added later. Here's how it looks in ClickHouse — all three wide-event tables gained an environment column:

clickhouse/migrations/unclustered/0008_add_environments_column.up.sqlALTER
ALTER TABLE traces       ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
ALTER TABLE observations ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
ALTER TABLE scores       ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;

Three details: (1) the type is LowCardinality(String) — env names are a small fixed set (prod/staging/dev), and low-cardinality encoding saves space and speeds queries; (2) the default 'default' — without an explicit environment, data lands in default, backward-compatible (old data naturally belongs to default); (3) the position AFTER project_id — this only places it right after project_id in the table's column layout (purely a cosmetic column order for readability); note it is NOT added to the ordering key. So unlike project_id, environment is essentially a plain filter column on the row — a soft slice: add a condition to filter by env, or ignore it to see them together.

This neatly contrasts the two isolation mechanisms: project_id is welded into the ordering key (isolate = locate, structurally safe), while environment is just an ordinary column (filtered at query time). Reasonably so — hard isolation must be guaranteed by structure, while a soft slice is just a convenient view dimension that needn't (and shouldn't) wield the heavy weapon of the ordering key.

two isolations in one row: project_id hard (ordering key) vs environment soft (filter column) project_id environment timestamp …wide columns … project_id = hard isolation welded into ORDER BY (project_id, toDate(ts), id) → never visible across projects (structural) isolate = locate; no query can bypass it environment = soft slice LowCardinality DEFAULT 'default', not in sort key → split only via WHERE environment='prod' ordinary filter column: split or merge at will

🔒 project = hard isolation

A security red line: different projects' data is never visible to each other. Welded into the ordering-key prefix + API-key ownership, guaranteed from storage to auth.

🏷️ environment = soft slice

A view preference: within one project, tag data prod/staging/dev — separable or combinable, just one filter dimension at query time.

How one request resolves the tenant

1

carry a credential

the SDK uses an API key (writes), or a logged-in user carries a session (reads), to make a request.

2

resolve the project

the server resolves "which project" from the key/session and checks access (L09 ApiKey, L21 protectedProjectProcedure).

3

all reads/writes lock to that project

writes stamp project_id on every row; queries put project_id in the ordering-key prefix. You cannot bypass it to see another project.

4

optional: slice by environment

within that project, filter prod / staging by environment as needed.

🎯 Design trade-off
Why weld project_id into the ordering key instead of treating it as an ordinary filter column? Because that makes "isolation" and "efficiency" the same thing. If project_id were just a column, "see only my project" would be a full-table filter — slow at scale and dangerous (miss the filter once and you leak across tenants). Put it first in the ordering key and one project's data is physically adjacent, so "see only my project" degrades to "scan a contiguous range" — fast and structurally unable to see others. The cost: the ordering key is hard to change once set, and nearly every query must carry project_id (the repository layer's uniform contract). Make multi-tenant isolation part of the storage structure, not a filter you might forget.
🎯 Key points
  • Three nesting levels: organization (billing/members) ⊃ project (data boundary) ⊃ environment (prod/staging/dev within a project).
  • project = hard isolation (never visible across projects), environment = soft slice (a tag separable/combinable within a project).
  • project_id threads the whole pipeline: resolved from the API key → stamped on every row → leads the ClickHouse ordering key → locates queries. Isolation is welded into the ordering key, not an afterthought filter.
  • the environment column: LowCardinality(String) DEFAULT 'default' AFTER project_id (migration 0008, L08) — low-cardinality, backward-compatible, right after project_id.
  • Tradeoff: baking isolation into storage → isolation and efficiency become one, and it's structurally safe; the cost is a hard-to-change ordering key and project_id required on queries.

🧪 Self-test - think about the design

1. In Langfuse multi-tenancy, what's the essential difference between project and environment?
  1. project is hard isolation (data never visible across projects, a security red line); environment is a soft slice (tagging prod/staging/dev within one project, separable or combinable)
  2. environment isolates more strongly than project
  3. They're identical, just different names
  4. project is soft, environment is hard
Show answer & explanation click to expand
Answer: A. 'Different teams' data must never mix' is a security red line → project hard isolation; 'live vs test' is one team's view preference → environment soft slice. Different strengths, different treatment.
2. Why is Langfuse's tenant isolation 'not an afterthought WHERE project_id filter'?
  1. Because project_id is random
  2. Because each project uses a separate database instance
  3. Because Langfuse doesn't support filtering
  4. Because project_id is welded into the front of the ClickHouse ordering key: one project's data is physically adjacent, so 'see only my project' becomes scanning a contiguous range — isolation and efficiency in one, and structurally impossible to see others
Show answer & explanation click to expand
Answer: D. If project_id were an ordinary filter column, it'd be slow at scale and dangerous (miss the filter and you leak across tenants). Put it first in the ordering key and isolation becomes 'locate a contiguous range' — fast and structurally safe. Cost: hard-to-change ordering key, project_id required on queries.
3. The ClickHouse environment column is LowCardinality(String) DEFAULT 'default' AFTER project_id. What's the intent of each?
  1. To remove project_id
  2. Pure placeholder, no intent
  3. To allow infinitely many env names
  4. Low-cardinality: few env names, saving space and speeding queries; default 'default': auto-classify when unspecified, backward-compatible with old data; right after project_id: forms a 'project+environment' locating prefix with it
Show answer & explanation click to expand
Answer: D. LowCardinality suits low-distinct columns (compact encoding, fast queries); DEFAULT 'default' makes old/unspecified-env data fall into default (backward compatible); AFTER project_id forms a locating prefix so 'project+environment' queries stay efficient.
💭 Open questions (no single right answer - just think or try)
  • 'Bake multi-tenant isolation into the storage structure, not a filter you might forget' — can you apply this to your own system? What benefits and constraints come from putting the tenant key first in the primary/ordering key?