Last lesson called project the hub and the "data boundary". This lesson makes that concrete: Langfuse is a
multi-tenant platform — one deployment serves thousands of unrelated teams at once, isolating their data cleanly via
three nesting levels: organization → project → environment. Grasp these three and you understand
"why you only see your own data" and "why the ClickHouse ordering key must start with project_id".
🔌 Analogy
Picture multi-tenancy as an office building: the organization is the company renting a
floor — signs the lease, pays the bills, manages its staff (members and billing live here); the project is a
department with its own area and access control, whose stuff never wanders into another department; the
environment is a further "production / test area" split inside the department — the same people keep
live and test data apart. Everyone may be in one building (one deployment, one database), but layered access
controls keep them from crossing over.
Three nesting levels: one organization holds multiple projects, each project may split into multiple environments (e.g. production / staging / development, default default). The org manages billing and members, the project is the hard data boundary, and the environment is a soft slice within a project.
Two strengths of isolation (entities from prisma/schema.prisma; values illustrative): Organization wraps Project wraps environment. But their strength differs — project_id is welded into the ClickHouse sort-key prefix (structural hard isolation, never visible across projects, echoing the Lesson 10 warn card); environment is just a filter column, a soft slice split only when a query adds WHERE.
What each level governs
Level
Governs
Isolation strength
Organization
billing, members, SSO, org-level API keys
the outer "account" boundary
Project
where data actually belongs: trace/observation/score, prompts, config all hang under a project
hard isolation: invisible across projects
Environment
distinguishes production / staging / development within one project
soft slice: queryable together or filtered by env
The key distinction is the last two: project is a "hard boundary" — hold one project's key and you simply
cannot see another project's data; environment is a "soft slice" — within one project it tags
live vs test data apart, so you can view only production, or compare several environments together. Why? Because "different teams' data must
never mix" is a security red line (hard isolation), while "live vs test" is just one team's own view preference
(soft slice) — different strengths, different treatment.
A concrete scenario for all three levels. A company (organization) signs up and pays on Langfuse, inviting 5 engineers as
members. They open two projects (project): "support bot" and "doc Q&A". The two projects' data is mutually
invisible — querying the support bot never mixes in doc-Q&A traces. Within "support bot" they use environment to
separate live traffic (production) from load-test/debug traffic (staging), so live cost isn't polluted by
load-test data — yet they can compare the two environments together when needed. The outer "company" cares only about the bill and
who's a member, holding no data directly; all data lands on some project + environment. This "org manages money and people, project
manages data, environment manages views" division is the entire skeleton of multi-tenancy.
project_id is the hard isolation key, threaded throughout
Multi-tenant isolation is not as simple as adding a WHERE project_id = ? filter afterward. In Langfuse,
project_id is a thread running through the whole pipeline:
project_id threads the whole pipeline: auth resolves project_id from the API key (L09: a key belongs to a project); writes stamp it on every row; ClickHouse's ordering key starts with project_id (L08); queries locate the contiguous range by it. So isolation is not an afterthought filter — it's baked into the physical layout; isolation and "fast" are the same thing.
This "project_id threads the whole pipeline" design's biggest value is turning security from "the programmer remembers to filter"
into "the structure guarantees it". Consider: if isolation relied on every query remembering to add WHERE project_id = ?,
then one forgotten or wrong filter could let company A see company B's data — a catastrophic bug in a multi-tenant system.
Langfuse instead welds project_id into the storage and auth foundations: the repository read APIs uniformly require a
project_id, and tRPC's protectedProjectProcedure middleware checks first whether you belong to this project
(L21). So "see another tenant's data by accident" isn't "easy to get wrong" — there simply is no such path. That's the
essential difference between "security in the structure" and "security by discipline".
⚠️ The trap to watch in multi-tenancy
Relying on "every query remembering to add WHERE project_id = ?" means one forgotten or wrong filter lets company A see company B's data — a catastrophic multi-tenant bug. Langfuse instead welds project_id into the sort-key prefix and the auth middleware: the cross-tenant path simply doesn't exist — structure, not discipline.
environment: a soft slice within a project
environment was added later. Here's how it looks in ClickHouse — all three wide-event tables gained an environment column:
ALTER TABLE traces ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
ALTER TABLE observations ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
ALTER TABLE scores ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
Three details: (1) the type is LowCardinality(String) — env names are a small fixed set (prod/staging/dev), and
low-cardinality encoding saves space and speeds queries; (2) the default 'default' — without an explicit environment, data lands
in default, backward-compatible (old data naturally belongs to default); (3) the position AFTER project_id — this only
places it right after project_id in the table's column layout (purely a cosmetic column order for readability); note it
is NOT added to the ordering key. So unlike project_id, environment is essentially a plain filter column on the row — a
soft slice: add a condition to filter by env, or ignore it to see them together.
This neatly contrasts the two isolation mechanisms: project_id is welded into the ordering key (isolate = locate,
structurally safe), while environment is just an ordinary column (filtered at query time). Reasonably so — hard isolation must
be guaranteed by structure, while a soft slice is just a convenient view dimension that needn't (and shouldn't) wield the heavy weapon of the
ordering key.
🔒 project = hard isolation
A security red line: different projects' data is never visible to each other. Welded into the ordering-key prefix + API-key ownership, guaranteed from storage to auth.
🏷️ environment = soft slice
A view preference: within one project, tag data prod/staging/dev — separable or combinable, just one filter dimension at query time.
How one request resolves the tenant
1
carry a credential
the SDK uses an API key (writes), or a logged-in user carries a session (reads), to make a request.
2
resolve the project
the server resolves "which project" from the key/session and checks access (L09 ApiKey, L21protectedProjectProcedure).
3
all reads/writes lock to that project
writes stamp project_id on every row; queries put project_id in the ordering-key prefix. You cannot bypass it to see another project.
4
optional: slice by environment
within that project, filter prod / staging by environment as needed.
🎯 Design trade-off
Why weld project_id into the ordering key instead of treating it as an ordinary filter column? Because that makes
"isolation" and "efficiency" the same thing. If project_id were just a column, "see only my project" would be a full-table
filter — slow at scale and dangerous (miss the filter once and you leak across tenants). Put it first in the ordering key
and one project's data is physically adjacent, so "see only my project" degrades to "scan a contiguous range" — fast and
structurally unable to see others. The cost: the ordering key is hard to change once set, and nearly every query must carry
project_id (the repository layer's uniform contract). Make multi-tenant isolation part of the storage structure, not a filter you
might forget.
🎯 Key points
Three nesting levels: organization (billing/members) ⊃ project (data boundary) ⊃ environment (prod/staging/dev within a project).
project = hard isolation (never visible across projects), environment = soft slice (a tag separable/combinable within a project).
project_id threads the whole pipeline: resolved from the API key → stamped on every row → leads the ClickHouse ordering key → locates queries. Isolation is welded into the ordering key, not an afterthought filter.
the environment column: LowCardinality(String) DEFAULT 'default' AFTER project_id (migration 0008, L08) — low-cardinality, backward-compatible, right after project_id.
Tradeoff: baking isolation into storage → isolation and efficiency become one, and it's structurally safe; the cost is a hard-to-change ordering key and project_id required on queries.
🧪 Self-test - think about the design
1. In Langfuse multi-tenancy, what's the essential difference between project and environment?
project is hard isolation (data never visible across projects, a security red line); environment is a soft slice (tagging prod/staging/dev within one project, separable or combinable)
environment isolates more strongly than project
They're identical, just different names
project is soft, environment is hard
Show answer & explanation click to expand
Answer: A. 'Different teams' data must never mix' is a security red line → project hard isolation; 'live vs test' is one team's view preference → environment soft slice. Different strengths, different treatment.
2. Why is Langfuse's tenant isolation 'not an afterthought WHERE project_id filter'?
Because project_id is random
Because each project uses a separate database instance
Because Langfuse doesn't support filtering
Because project_id is welded into the front of the ClickHouse ordering key: one project's data is physically adjacent, so 'see only my project' becomes scanning a contiguous range — isolation and efficiency in one, and structurally impossible to see others
Show answer & explanation click to expand
Answer: D. If project_id were an ordinary filter column, it'd be slow at scale and dangerous (miss the filter and you leak across tenants). Put it first in the ordering key and isolation becomes 'locate a contiguous range' — fast and structurally safe. Cost: hard-to-change ordering key, project_id required on queries.
3. The ClickHouse environment column is LowCardinality(String) DEFAULT 'default' AFTER project_id. What's the intent of each?
To remove project_id
Pure placeholder, no intent
To allow infinitely many env names
Low-cardinality: few env names, saving space and speeding queries; default 'default': auto-classify when unspecified, backward-compatible with old data; right after project_id: forms a 'project+environment' locating prefix with it
Show answer & explanation click to expand
Answer: D. LowCardinality suits low-distinct columns (compact encoding, fast queries); DEFAULT 'default' makes old/unspecified-env data fall into default (backward compatible); AFTER project_id forms a locating prefix so 'project+environment' queries stay efficient.
💭 Open questions (no single right answer - just think or try)
'Bake multi-tenant isolation into the storage structure, not a filter you might forget' — can you apply this to your own system? What benefits and constraints come from putting the tenant key first in the primary/ordering key?