The last two lessons starred ClickHouse (massive telemetry). This one switches to the other half: Postgres. If ClickHouse
stores "what happened" (trace/observation/score), Postgres stores "how the platform is configured" — who
your orgs and projects are, who has access, which API keys exist, which prompts are defined, which eval rules are set, how each model is
priced. This side is the control plane — small in volume yet global in importance. Its shape is all in
packages/shared/prisma/schema.prisma.
🔌 Analogy
Think of the two stores as a company's two systems: Postgres is the HR/admin system — who's an
employee, in which department, with what permissions, under which policies. Not much data, but it must be accurate, editable,
instantly effective (a new hire's permissions must work right away). ClickHouse is the transaction
ledger — mountains of records, append-only, used later for monthly/category stats. You wouldn't write policies into the
ledger, nor stuff every transaction into HR files — two kinds of data, two systems, each in its place.
Two halves: Postgres stores "how the platform is configured" — org/project/user, API keys, prompts, eval/dataset config, pricing, integrations, audit; ClickHouse stores "what actually happened" — three wide-event tables. This is the macro version of Lesson 3's "domain shape vs physical storage": one governs control, one governs data.
Everything hangs under Project (model names from packages/shared/prisma/schema.prisma; columns illustrative): Organization 1─N Project, Project 1─N ApiKey/Prompt/Dataset; User is many-to-many to org/project via OrganizationMembership/ProjectMembership with a role. Because "everything hangs under project", auth only needs to answer "do you belong to this project" (Lessons 10, 21).
Postgres holds the "control plane"
Open schema.prisma and you'll see dozens of models. They group by purpose (all map to snake_case tables via
@@map, with cuid primary keys):
Group
Main models
Governs
Identity & tenancy
Organization · Project · User · *Membership · Role · ApiKey
who the org/project is, who has what permission, which key
See the pattern? These are all "config" and "definition", not "mountains of events". They number in the
thousands to millions, are read/written precisely and often one at a time (create a project, edit a
prompt, check a key's validity), and need strong consistency — exactly the relational database Postgres's forte.
✅ Config in Postgres
You want foreign keys (project → its prompts/keys/config), transactions (a multi-table change all-or-nothing), and "readable right after writing". All relational-DB strengths.
❌ Config not in ClickHouse
ClickHouse has weak transactions, eventual consistency, and dislikes frequent single-row edits. Put "create-project-instantly-usable" there and you'd get the awkward "just created but can't find it". Config there is a mismatch.
tier 1identity & tenancy
Organization / Project / User / Membership / Role / ApiKey — decide "who, in which project, with what permission". The prerequisite for everything else.
tier 2feature config
Prompt, Dataset, ScoreConfig, EvalTemplate/JobConfiguration, Model/Price — each feature's "definition & orchestration", all hung under a project.
tier 3integrations & ops
the *Integration models, BatchExport/Action, Media, AuditLog, BackgroundMigration — outbound connections and the platform's own ops records.
Project is the hub
Among these dozens of models, Project is the undeniable center. Open its definition and you'll find few
fields (id, orgId, name, deletedAt soft-delete, retentionDays…) but dozens of relations — almost everything else belongs
to some project:
Project is the center of the control plane: under Organization, almost everything (API keys, members, prompts, datasets/eval, models/pricing, dashboards, automation, integrations, media/export) hangs off some project 1:N. So project is the natural "data boundary" — an API key belongs to a project, and all the data you query is scoped to its project (multi-tenancy, next lesson).
"Project is the hub" has two very real consequences. One is cascade: delete a project and its keys, prompts, datasets and
config are cleaned up with it (many relations are marked onDelete: Cascade), making "delete a project" a clean operation rather
than leaving orphan data everywhere. The other is boundary equals permission: because everything hangs under a project,
authorization only needs to answer "do you belong to this project" — the basis for the next lesson's multi-tenancy and for L21's tRPC
middleware protectedProjectProcedure. Remember this "project-centric" picture and many seemingly-scattered features later turn
out to hang off it.
Why are there trace tables in Postgres?
Browsing schema.prisma you may be puzzled: why are there LegacyPrismaTrace, LegacyPrismaObservation,
LegacyPrismaScore? Isn't telemetry all in ClickHouse? The answer is in the name — Legacy. This is a
historical remnant of Langfuse's early days storing trace/observation/score in Postgres; later, to handle scale, the
analytical data moved to ClickHouse (the very reason for Lessons 7 and 8). These legacy models remain mainly for
compatibility and migration — no longer the hot path. See a Legacy prefix and you know "that's the old way,
don't copy it".
API keys & permissions (preview)
In the identity group, ApiKey has a security detail worth remembering now:
packages/shared/prisma/schema.prismamodel ApiKey
publicKey // public key, plaintext
hashedSecretKey // hash of the secret (no plaintext!)
fastHashedSecretKey // a hash for fast verification
displaySecretKey // a masked fragment, UI display only
scope // ApiKeyScope, default PROJECT
Note: the secret is never stored in plaintext, only hashed — so even a DB leak yields no usable key. Permissions use the
Role enum: OWNER / ADMIN / MEMBER / VIEWER / NONE, combined with org-level and project-level memberships to decide
"who can do what in which project". Auth and permissions get their own lessons (L49, L53); for now just register the impression:
the control plane stores not only "config" but "who may touch that config".
🎯 Design trade-off
What's the biggest payoff of splitting "control plane" and "data plane" into two databases? Letting each
evolve and scale by its own rules. The control plane is small and needs strong consistency — fine for a single (or
primary/replica) Postgres; the data plane is a flood needing a horizontally-scalable ClickHouse cluster. Force them into one store and
you either shortchange config consistency or crush telemetry throughput. The cost: some operations span two stores (e.g.
ingestion reads model pricing and prompt links back from Postgres) — but this "read config, write data" cross-store collaboration is
controllable and heavily cached, far more realistic than "making one store great at both OLTP and OLAP". Split
by responsibility; let each store do only what it's best at.
🎯 Key points
Postgres = control plane: stores "how the platform is configured" — org/project/user, API keys, prompts, eval/dataset config, model pricing, integrations, audit. Defined in schema.prisma.
Project is the hub: few fields but dozens of relations; almost everything belongs 1:N to a project — the natural data boundary (multi-tenancy basis).
LegacyPrismaTrace/Observation/Score are historical remnants: analytical data long moved to ClickHouse; these remain only for compatibility/migration.
API keys are stored hashed, never plaintext; permissions use Role (OWNER/ADMIN/MEMBER/VIEWER/NONE) + org/project memberships (L49, L53).
Control plane vs data plane is the macro version of Lesson 3's "domain shape vs physical storage": split by responsibility, each doing what it's best at.
🧪 Self-test - think about the design
1. Why does Langfuse keep config/metadata (org/project/user, API keys, prompts, pricing…) in Postgres rather than in ClickHouse with telemetry?
Because Postgres is slower, fit for rarely-used data
Pure history, no reason
Config needs foreign keys, transactions and strong consistency (instantly usable) — relational Postgres's strength; ClickHouse has weak transactions, eventual consistency and dislikes frequent single-row edits, so config there is a mismatch
Because ClickHouse is too small
Show answer & explanation click to expand
Answer: C. Config is OLTP: small, precise single-row, strongly consistent, with foreign keys — Postgres's turf; ClickHouse fits massive append + aggregation (OLAP). Split by responsibility, each at its best — a continuation of L07.
2. Why is Project the 'hub' of the control plane, and why is it the multi-tenant 'data boundary'?
Because Project stores all traces
Because it has no relations
Because the Project table has the most rows
Project has few fields but dozens of 1:N relations; almost everything (keys/members/prompts/datasets/dashboards/integrations…) belongs to a project; an API key belongs to a project and queries are scoped to it
Show answer & explanation click to expand
Answer: D. Almost all control-plane objects hang 1:N off a project, making it the center; and since keys and data are scoped by project, project is the natural isolation boundary (matching ClickHouse's project_id-first ordering key). Multi-tenancy next lesson.
3. Why are there still LegacyPrismaTrace/Observation/Score in schema.prisma? And how are API keys stored?
Legacy back up ClickHouse; API keys aren't stored
Legacy are the primary tables; API keys are stored in plaintext
Both have been removed from the code
Legacy models are historical remnants of storing telemetry in Postgres; analytical data moved to ClickHouse, kept only for compat/migration; API key secrets are stored hashed, never plaintext
Show answer & explanation click to expand
Answer: D. Legacy prefix = old way: telemetry long moved to ClickHouse (the reason for L07/L08); these remain for compat/migration. API keys store only hashedSecretKey etc., so a DB leak yields no usable key — a security detail expanded in L49/L53.
💭 Open questions (no single right answer - just think or try)
Langfuse splits 'config' and 'data' into two databases. If your system also had 'small critical config' + 'massive events', how would you divide them? How would you tame the cross-store config-read cost (hint: caching)?