这里藏着一个 ClickHouse 老手才懂的优化。回想第 8 课:traces 是 ReplacingMergeTree,同一 id 可能有多行(create/update 各一行),靠后台合并去重。
查询时若加 FINAL,ClickHouse 会当场把所有分区读出来、去重,结果正确但慢。Langfuse 的取舍是——能不用 FINAL 就不用:
对与快兼得:默认按时间排序时用 toDate(timestamp), event_ts DESC + LIMIT 1 BY id 代替 FINAL,CH 只读最新分区、在内存里取每个 id 的最新行;只有当你按非时间列排序时才退回 FROM traces FINAL 全量去重。源码注释见 traces-ui-table-service.ts:443-455。
「为最常见路径极致优化,把复杂度/代价推迟到真的要了才引入」——紧凑行 vs 指标、按需 JOIN、按需 FINAL 都是这条原则。你做过的列表/详情界面里,有没有「为了 1% 的高级需求,拖慢了 99% 的日常浏览」的地方?你会怎么拆?
The last two lessons covered how queries are built and run. This one looks at Langfuse's flagship list — the trace table — and how it
opens instantly. Two secrets: one, a single generic function serves four "shapes" (count / rows / metrics / identifiers); two, it
splits the cheap rows from the expensive metrics — the table paints rows first, while cost/latency/score metrics needing JOINs stream in
later and merge client-side. Plus a conditional-FINAL query discipline that keeps the default view reading only the latest partition. Get this
lesson and you've got the whole art of a big-data table being "complete yet fast".
🍽️ Analogy
Think of list loading as a restaurant serving in courses. Insisting the whole elaborate meal (rows + all metrics) arrive at once means you sit
waiting a long time. A smart kitchen brings the bread first (compact rows — cheap, everyone wants them) so you start eating
immediately; the labor-intensive mains (metrics: cost, latency, scores, each made and assembled to order) arrive dish by dish,
quietly added to your table. Your experience is "something to eat the moment you sit down", not "staring at an empty table for twenty minutes".
Langfuse's table does exactly this: give what you can instantly, fill in what you must compute as it comes — that's the "compact list model".
Compact rows vs metrics: two parallel calls, merged client-side
Opening the trace list, the UI fires two tRPC calls at once: traces.all for the compact rows (~12
always-wanted columns), and traces.metrics for the metrics (cost, latency, score aggregates — needing 2 JOINs). Whoever returns first is
used: rows back, the table renders immediately; metrics back, they're merged in by id.
Splitting "rows" from "metrics": the UI fires traces.all (rows) + traces.metrics (metrics) in parallel; joinTableCoreAndMetrics merges by id and returns success even when metrics are absent, so the table renders from rows and fills metrics later. Source: web/src/components/table/utils/joinTableCoreAndMetrics.ts:1, services/traces-ui-table-service.ts:206.
The wide table's "compact representation" (columns per the web column definitions; values illustrative): the list page projects ClickHouse wide rows into a few key columns (time/name/user/latency/cost/tokens/scores), with filter chips on top (the Lesson 23FilterState) and pagination. Huge trace volumes aren't fully fetched — only the current page's columns and rows, the user-facing side of the repository's LIMIT + column projection.
// one function, four "shapes" — distinguished by select (type-safe overloads)functiongetTracesTableGeneric(props: { select: "count" | "rows" | "metrics" | "identifiers" })
// LEFT JOIN only when really needed — default row browsing skips two JOINsconst requiresScoresJoin = /* score filter/sort, or select==="metrics" */;
const requiresObservationsJoin = /* likewise */;
...
FROM traces t ${defaultOrder ? "" : "FINAL"} // default ordering skips FINAL (below)
${requiresObservationsJoin ? "LEFT JOIN observations_stats o ..." : ""}
${requiresScoresJoin ? "LEFT JOIN scores_avg s ..." : ""}
Conditional FINAL: the default view reads only the latest partition
Here hides an optimization only ClickHouse veterans appreciate. Recall Lesson 8: traces is a ReplacingMergeTree, where one id may have multiple rows
(create/update each a row), deduped by background merges. Add FINAL to a query and ClickHouse reads all partitions and dedups on the spot —
correct but slow. Langfuse's trade-off: avoid FINAL whenever possible:
Correct AND fast: default time-ordering uses toDate(timestamp), event_ts DESC + LIMIT 1 BY id instead of FINAL, so CH reads only the latest partition and takes each id's latest row in memory; only when you sort by a non-time column does it fall back to FROM traces FINAL full dedup. Source comment: traces-ui-table-service.ts:443-455.
🟢 default path (most common)
time-ordered + no metrics needed → no FINAL, no JOINs, read only the latest partition. This is the 80% "open the list and glance" case — correct and blazing fast.
🟡 heavy path (pay on demand)
sort by cost, or want metrics → only then add FINAL and LEFT JOIN observations/scores. Complexity is introduced only when you actively ask, never burdening the common path.
select shape
what it fetches
used for
rows
~12 compact columns
the table body, rendered immediately
metrics
cost/latency/score aggregates (2 JOINs)
metric columns, filled in later
count
uniqExact(id)
pagination total (deduped count)
identifiers
just the id list
batch actions, cross-page selection
Table state: URL is the source of truth, three layers of persistence
A list isn't just "draw some rows". You tweaked column order, set filters, sorted, saved a "view preset" — where should this table state live?
Langfuse's answer is three layers, with the URL as the source of truth:
URLsharing & source of truth
Filters, sort, search, current view id are all encoded into the URL. So you send a colleague the link and they see exactly the same thing — the URL is the single source of truth for "what this table looks like right now" (echoing Lesson 23's FilterState).
session storagenavigation memory
Hopping between tables, session storage remembers your recent column widths, visible columns, scroll position so you aren't reset each time you return — valid only within this browsing session.
databasepermanent presets
"View presets" (saved views) go into the DB, permanent, cross-device, team-shareable. Deleted a preset, or a column it referenced no longer exists? Graceful degradation: drop the invalid part + warn, never a blank screen.
The three layers each own a timescale — URL "shareable right now", session "this navigation", DB "permanent" — together letting a big table be
deep-linkable, state-preserving across navigation, and a long-term home for the team's go-to views. Paired with a global time range (Zustand default
⊕ URL explicit), it becomes the mandatory timestamp filter feeding Lesson 22's look-back windows, nailing down the scan range. State layering is
the other half of a big list's craft beyond "fast" — "stable" (no loss across navigation) and "shareable" (one link reproduces the whole view).
🎯 Design trade-off
Why not fetch it all at once (rows + all metrics), instead of two calls and a pile of "on-demand" switches? Because the list's experience
goal is "instant", while metrics are inherently expensive. Cost, latency, scores need 2 big-table JOINs to compute; bound to the rows, the whole table must
wait for the slowest call to show — the user stares at blank space. Split them and the compact rows (cheap, everyone wants) arrive
instantly, metrics fill in as "extra info" as computed; perceived latency drops from "slowest call" to "fastest call". Likewise, conditional FINAL
and conditional JOIN are the same wisdom: optimize ruthlessly for the most common, simplest path, deferring complexity and cost until "you actually
ask". This is the golden rule of high-frequency list performance — don't let the 1% advanced need (sort by cost, see all metrics) slow the 99% daily browse.
🎯 Key points
One getTracesTableGeneric serves four shapes (rows / metrics / count / identifiers), distinguished by select with type-safe overloads.
Compact rows vs metrics, split: the UI fires traces.all (rows) + traces.metrics (metrics) in parallel; joinTableCoreAndMetrics merges by id and returns success even without metrics, so the table renders from rows first.
Conditional JOIN: LEFT JOIN observations/scores only when filter/sort needs it or select==="metrics", saving two JOINs on default browsing.
Conditional FINAL: default time-ordering uses toDate(timestamp),event_ts DESC + LIMIT 1 BY id to read only the latest partition; non-default ordering falls back to FROM traces FINAL full dedup.
Golden rule: optimize for the most common path, defer complexity/cost until "actually asked". Table state lives in three layers, URL as source of truth (URL ⊕ session ⊕ DB presets with graceful degradation); the global time range becomes the mandatory timestamp filter feeding Lesson 22's look-back windows.
🧪 Self-test - think about the design
1. Opening the trace list, Langfuse fires traces.all (rows) and traces.metrics (metrics) in parallel rather than one combined query. Why?
because ClickHouse can't query two kinds at once
to make metrics more accurate
to flood the server with more requests
compact rows are cheap and arrive instantly so the table paints first; cost/latency/score metrics need 2 JOINs and are slower, merged in by id later as extra info. Perceived latency drops from 'slowest call' to 'fastest call'
Show answer & explanation click to expand
Answer: D. The list's goal is instant open, but metrics are inherently expensive (2 JOINs). Splitting cheap rows from costly metrics, joinTableCoreAndMetrics merges by id and returns success even without metrics, so the table renders from rows first and fills metrics later. The 'compact list model' — give what you can instantly, fill in what you compute as it comes.
2. traces is a ReplacingMergeTree (one id may have multiple rows). For default time ordering, Langfuse skips FINAL and uses ORDER BY toDate(timestamp),event_ts DESC + LIMIT 1 BY id. Why is this better?
because LIMIT 1 BY id saves memory
because FINAL returns wrong results
because default ordering needs no dedup
FINAL reads all partitions and dedups on the spot, slow; time ordering + LIMIT 1 BY id lets CH read only the latest day's partition from disk and take each id's latest row in memory — correct and fast, no pulling a month of data just to dedup
Show answer & explanation click to expand
Answer: D. FINAL is correct but slow (reads all to dedup). For default time ordering, event_ts DESC + LIMIT 1 BY id takes each id's latest row in memory, CH reading only the latest partition. Only sorting by a non-time column (e.g. cost) falls back to FROM traces FINAL full dedup — pay on demand, the common path unburdened.
3. Langfuse stores table state (filters, sort, column settings, view presets) in three layers, with the URL as the source of truth. The most direct benefit?
to hide filter conditions
to reduce database queries
to render the table faster
the URL encodes the current view → send a colleague the link and they see exactly what you do (shareable/deep-linkable); session preserves state across navigation; DB stores permanent presets, cross-device shareable, with graceful degradation for invalid columns
Show answer & explanation click to expand
Answer: D. Three layers each own a timescale: URL 'shareable now' (deep links, one link reproduces the whole view, echoing L23's FilterState), session 'this navigation' (column widths/visible columns survive hops), DB 'permanent' (saved views shared cross-device/team, invalid parts dropped+warned, no blank screen). This makes big tables fast, stable, and shareable.
💭 Open questions (no single right answer - just think or try)
'Optimize for the most common path, defer complexity/cost until actually asked' — compact rows vs metrics, conditional JOIN, conditional FINAL all follow it. In your own list/detail UIs, is there a place where 'the 1% advanced need slows the 99% daily browse'? How would you split it?