Lesson 24 was the list — fetch one compact row each for thousands of traces, defer everything expensive. This lesson is its opposite:
the detail page. Open one trace and you want the whole observation tree + all scores — one full fetch. Why dare to fetch
it all? Because "one trace" is bounded (not a list of thousands). But each observation's input/output in the tree could be huge, so the
detail still has discipline: the tree query carries no IO (lazy-loaded per node when you expand it), with a verbosity knob capping the
trace's own IO. The same "bound the unbounded dimension" craft, applied at a different scale.
📖 Analogy
Think of list vs detail as a library catalog vs one book. The catalog (list) gives you tiny cards for thousands of
books — nobody transcribes a whole book onto a card. But when you pull one book (detail), you want the whole book's structure — table
of contents, chapters, page numbers at a glance. Yet you don't photocopy every page's long footnotes upfront; only when you flip to a page and want
the detail do you read that footnote (an observation's IO, lazy-loaded). "The whole book's skeleton at once, a page's long content fetched on demand"
— that's the detail page's balance between "complete" and "restraint".
Fetch the whole tree at once, but hang no IO on it
The detail page's main query is byIdWithObservationsAndScores. It runs on protectedGetTraceProcedure — that door (Lesson 21) has already
prefetched the trace itself and verified access (allowing publicly shared traces too). It then fetches two things in parallel: the
whole observation tree and all of this trace's scores:
Fetch the skeleton, defer the flesh: byIdWithObservationsAndScores fetches the observation tree (includeIO:false) and scores in parallel, partitions out CORRECTION-type scores, derives latency from observation start/end, returns the IO-less tree; opening a node lazy-loads its IO. Plus getAgentGraphData for the agent graph. Source: web/src/server/api/routers/traces.ts:368,610.
The waterfall draws "time" (based on observation start_time/end_time; values illustrative): each bar is offset by start_time and sized by latency, with indentation for nesting. At a glance you see who waits on whom and which step is slowest — here the two serial generations plan and summarize dominate the duration (the critical path). This is the trace detail page's most valuable figure.
// ctx.trace prefetched by the door (with access check); fetch tree & scores in parallelconst [observations, traceScores] = await Promise.all([
getObservationsForTrace({ traceId, projectId, includeIO: false }), // tree without IOgetScoresAndCorrectionsForTraces({ projectId, traceIds: [traceId] }),
]);
// split out CORRECTION-type scoresconst [corrections, scores] = partition(scores, (s) => s.dataType === "CORRECTION");
// derive the whole trace's latency from earliest start, latest end of observationsconst latencyMs = maxEnd - minStart;
return { ...trace, observations: obs.map((o) => ({ ...o, input: undefined, output: undefined })) };
Note the unremarkable includeIO: false — it's the key to the detail page's performance. Imagine an agent trace with three hundred
observations, each observation's input/output a sizeable JSON: load all the IO when opening the tree and that one request could be tens of MB and several
seconds. Langfuse's choice is tree first, IO later:
😖 tree with all IO
Fetch three hundred observations + all their IO at once: a tens-of-MB request, slow and memory-heavy. But when you open the detail, you won't expand most nodes — most of that IO was fetched for nothing.
😀 tree first, IO lazy
The tree query carries only structure and timing (light, fast), so the whole tree is visible immediately; expand an observation and only that one's IO is fetched. Pay on demand, never for nodes you didn't look at — so even a huge trace's detail page opens instantly.
The verbosity knob: capping IO for "giant traces"
Even one trace's own input/output could be a multi-MB JSON (say, stuffed with a long context). For this, the single-trace query byId
carries a verbosity param (compact / truncated / full) that decides server-side whether IO is parsed or truncated — don't let a
giant trace blow up the response:
Why truncate server-side, not on the frontend? Because trimming after the data reaches the frontend is too late — those few MB
already got read from ClickHouse, serialized, and crossed the network. verbosity puts the trimming closest to the data source: what should
carry no IO (compact) isn't fetched at all, what should be truncated (truncated) is cut before serialization, and only an explicit "show all" (full) returns the
whole thing. So one byId endpoint serves both "a lightweight preview in a list" and "see the full text on a detail page", switching posture by a single
parameter.
Three-tier cap: byId's verbosity (compact / truncated / full, default full) decides server-side whether the trace's own IO is parsed/truncated, capping the response for huge traces. Source: web/src/server/api/trpc.ts:480,524-525 (parseIO(input, verbosity)).
From the user's view, opening a trace detail is five steps — the tree appears almost instantly, details fill in as you click:
1
door prefetch + verify
protectedGetTraceProcedure has already fetched the trace and confirmed your access (publicly shared traces pass too).
2
parallel fetch tree & scores
Promise.all pulls the observation tree (no IO) and all scores at once — both fired together, wait for the slower.
3
assemble & render skeleton
Partition out CORRECTION, derive latency, and the tree's structure and timing paint immediately — you see the whole call at once.
4
open a node, lazy-load IO
Click a generation to see its prompt/completion and the frontend fetches that one observation's IO, full or truncated per verbosity.
5
(optional) switch to the agent graph
Want the agent's step topology? getAgentGraphData takes another path, rendering the tree as a directed graph (trace-graph-view).
Place Lessons 24 and 25 side by side and "one discipline, two scales" is plain — whether list or detail, fence the bounded dimension first and defer the
unbounded one:
tree query without IO, verbosity truncating the trace's own IO
🎯 Design trade-off
Why does detail dare to "fetch the whole tree", while the list "defers aggressively"? The key is boundedness. The list faces an
unbounded number of rows (thousands, tens of thousands of traces), so even a slightly pricey column per row must be deferred; the detail faces
one trace — however many observations it has, that count is bounded, and "seeing this one in full" is precisely the detail page's value, so
the structure and timing are worth fetching all at once. But a bounded entity still hides an unbounded dimension inside: a single observation's IO can be
a 2MB JSON, and hundreds stacked up explode again. So the same "fence the bounded, defer the unbounded" discipline descends into the tree — fetch the tree
in full (the bounded structure), defer the IO (the unbounded content), and cap the trace's own IO with verbosity. It's not "list restrained, detail
indulgent"; both do the same thing at their own scale: only rush to fetch the part you can bear.
🎯 Key points
The detail page's main query byIdWithObservationsAndScores runs on protectedGetTraceProcedure (the door prefetched the trace and verified access, including public shares).
Parallel fetch of the observation tree + scores; partition out CORRECTION-type scores; derive latency from observation start/end.
Tree without IO (includeIO:false, observation input/output stripped) — opening a node lazy-loads its IO, avoiding pulling hundreds of observations' huge content at once.
verbosity (compact / truncated / full) tiers/truncates the trace's own IO server-side, preventing huge traces from blowing up the response. Plus getAgentGraphData for the agent graph view.
With Lesson 24, "one discipline, two scales": fetch the bounded dimension first (list=compact rows / detail=tree structure), defer the unbounded one (list=metrics / detail=observation IO) — only rush to fetch the part you can bear.
🧪 Self-test - think about the design
1. The detail page shows a trace's whole observation tree, yet byIdWithObservationsAndScores fetches observations with includeIO:false (no input/output). Why?
to hide IO from users
because IO lives in another database
because observations have no IO fields
the tree query carries only structure and timing (light, fast) so the whole tree shows instantly; one observation's IO can be a multi-MB JSON and hundreds explode, so only opening a node lazy-loads that one's IO — pay on demand, never for nodes you didn't look at
Show answer & explanation click to expand
Answer: D. An agent trace may have hundreds of observations, each IO a sizeable JSON. Loading all IO when opening the tree would be tens of MB and seconds, yet you won't expand most nodes — wasted. Langfuse sends the tree first (structure+timing) and lazy-loads IO on click, so even a huge trace's detail opens instantly.
2. Lesson 24's list 'defers aggressively' while Lesson 25's detail 'fetches the whole tree' — seemingly opposite. What single principle do they share?
list uses ClickHouse, detail uses Postgres
fetch the bounded dimension first, defer the unbounded one: the list faces unbounded rows → defer even slightly pricey columns; the detail faces one trace (bounded observation count) → fetch structure in full, but a single observation's IO is still an unbounded dimension within → lazy-load. Only rush to fetch the part you can bear
both fetch as much data as possible
both fetch as little data as possible
Show answer & explanation click to expand
Answer: B. The key is boundedness. The list's row count is unbounded (thousands+), so even per-row metrics are deferred; the detail faces one trace whose observation count, though large, is bounded, and seeing this one in full is the detail's value, so structure and timing are fetched at once. But a bounded entity hides an unbounded dimension (per-observation IO), so the same discipline descends into the tree: fetch the tree, defer IO, cap with verbosity.
3. byId carries a verbosity param (compact/truncated/full) deciding server-side whether the trace's own IO is parsed/truncated. Why truncate server-side rather than on the frontend?
the frontend can't truncate strings
because the server is more secure
to simplify frontend code
trimming after data reaches the frontend is too late — those MB already got read from ClickHouse, serialized, crossed the network. verbosity puts trimming closest to the source: compact doesn't fetch, truncated cuts before serialization, only full returns it whole
Show answer & explanation click to expand
Answer: D. Truncation aims to save 'read+serialize+transfer' cost; those costs already happen as data leaves ClickHouse, so trimming at the frontend is futile. verbosity tiers server-side, closest to the source: compact fetches no IO, truncated cuts before serialization, full returns whole. So one byId serves both a list's lightweight preview and a detail's full text.
💭 Open questions (no single right answer - just think or try)
'Fence the bounded dimension and fetch it first, defer the unbounded one' — can you apply this to your own 'list + detail' UIs? Which dimensions of your entities are bounded (fetch at once) and which unbounded (lazy-load or cap)? At which layer and by what means would you cap the unbounded ones?