上一课你知道了张量从 ggml_context 的内存池里来。但这里有个会让很多人意外的事实：当你写下 c = ggml_mul_mat(a, b) 时，那个矩阵乘根本没有发生——ggml 只是默默记下了一句"c 是由 a 和 b 经矩阵乘得到的"。 这一课就讲这种"先记账、不动手"的惰性建图，它是整个 ggml 引擎最精巧的设计之一。
这件事为什么重要？因为它颠覆了你对"调用一个函数就该立刻得到结果"的直觉。在 ggml 里，调用算子更像是在 下订单、写清单，而不是当场交付货物。整张神经网络的前向过程，会先被完整地"记成一张图"， 然后才在某个时刻被一次性地、有计划地算出来。理解了这个"记账在前、计算在后"的两段式，你才算真正看懂 ggml 为什么能又快又省、还能跑遍各种硬件。
顺便说一句，"惰性"（lazy）在编程里是个褒义词，不是"偷懒"的意思，而是"不到非算不可的那一刻，绝不提前算"。这种推迟，往往能让程序在真正动手前 看清全局、做出更聪明的安排。ggml 把这个思想用在了计算图上：所有算子调用都先攒着，等图齐了再一次性算——这正是它高效的源头之一。
先把最核心的误解纠正过来。在很多框架里，c = a @ b 写下去，乘法立刻就算完了，c 里装着结果数字。 但在 ggml 里完全不是这样：ggml_mul_mat(a, b) 返回的 c，此刻还是一个空壳—— 它知道自己的形状、知道自己将由谁经什么运算得到，但里面一个数都还没算。它记下的，只是"身世"：
请特别留意"反向"这两个字。在你脑子里画神经网络时，箭头通常是从输入流向输出的（数据怎么走）； 但 ggml 在张量里存的指针方向恰好相反——是结果指回它的输入。为什么反着存？因为执行时 ggml 最关心的问题是"要算出这个结果，我得先有谁"， 顺着结果往回找输入，正好一步到位。这就像顺着一个人往上查父母、再查祖父母，比从祖先往下逐代点名要直接得多。这个"从输出回溯输入"的方向， 是后面建图、求导都反复用到的关键。
看那个高亮的结果 c：它的 op 字段记着"我是用矩阵乘得到的"，src[0] 和 src[1] 两个指针分别指回 a 和 b（注意箭头方向——是结果指回输入， 所以叫"反向指针"）。这两样东西，L05 介绍 ggml_tensor 字段时就见过，当时只说"记录它怎么来的"，现在你看到它真正的用途了。 把源码摊开看，每个算子函数都是同一个套路：
// 简化自 ggml/src/ggml.c 的 ggml_mul_mat struct ggml_tensor * ggml_mul_mat(ctx, a, b) { result = ggml_new_tensor(ctx, GGML_TYPE_F32, ...); // 只建一个空的结果张量 result->op = GGML_OP_MUL_MAT; // 记下"我是怎么来的" result->src[0] = a; // 记下输入 1 result->src[1] = b; // 记下输入 2 return result; // 直接返回, 一个乘法都没做! }
ggml_add、ggml_rms_norm、ggml_soft_max……几乎所有算子都长这样： 建一个结果张量、填好 op 和 src、返回。真正的浮点运算，要等到后面"执行"时才发生（下一课的事）。 有些算子还需要额外参数（比如 rope 的旋转角度、softmax 的缩放系数），这些会用 ggml_set_op_params_* 之类的辅助函数 存进结果张量的 op_params 里——但同样，只是"记下来"，不计算。
换个角度想，这个"空壳"结果张量其实是一张借条（IOU）：它向你承诺"将来我会等于 a 乘 b 的结果"，但现在还没兑现。 你可以拿这张借条继续往下写——把它当作下一个算子的输入，再得到一张新借条；如此层层叠叠，直到写出最终输出。整个过程里，没有任何真实数字被算出来， 你手里攒下的，是一摞环环相扣的借条。等到"执行"那一刻，ggml 才会顺着这摞借条，从最底层开始，把每一张都兑现成真实的数据。这种"先开借条、后统一兑现"， 正是惰性（lazy）二字的含义，也是这一课从头到尾在反复打磨的那个核心直觉。
一个算子记下两三个 src，看起来不起眼；但当你把整个模型的前向过程都写出来，这些 src 指针就层层相扣，连成了一张有向图。 看一个最小的例子——两层线性变换 y = W2 · (W1 · x)：
输入和权重，它们的 op 是 NONE（没有"来历"，是图的起点）。
第一个算子结果，src = [W1, x]；它是个"节点"。
第二个算子结果，src = [W2, h]；注意它依赖上一步的 h。
把上面这两步画成图就一目了然：x、W1、W2 是叶子，h 和 y 是算子节点，每个节点用 src 指针指回自己的输入——于是"写下算式"就等于"连出一张有向图"。
这张图的妙处在于：从输出 y 出发，顺着 src 指针往回走，就能找到算出它所需的一切—— y 依赖 W2 和 h，h 又依赖 W1 和 x。ggml 用 ggml_build_forward_expand(graph, y) 做的正是这件事： 从你指定的输出张量出发，沿 src 递归回溯，把所有依赖按"先算谁后算谁"的顺序（拓扑排序）收集进一张 ggml_cgraph：
# 对应 ggml/src/ggml.c 的 ggml_build_forward_expand / ggml_visit_parents_graph def build_forward(graph, t): for s in t.src: # 先把依赖都收进来 build_forward(graph, s) # 递归回溯 if t.op == NONE and not t.is_param: graph.leafs.append(t) # 输入/常量 -> 叶子 else: graph.nodes.append(t) # 算子结果 -> 节点(按依赖顺序排好)
因为是"先递归收集依赖、再把自己放进去"，最后 graph.nodes 里的节点天然就是拓扑有序的： 排在前面的，一定不依赖排在后面的。这样执行时只要从头到尾依次算，每算一个节点，它的输入保证已经算好了。 ggml_cgraph 本身就是几个数组：nodes（算子结果）、leafs（输入/常量）、 计数 n_nodes/n_leafs、容量 size（默认 GGML_DEFAULT_GRAPH_SIZE=2048）。
把这套机制放回真实的 llama.cpp 里看：加载一个模型后，每跑一步推理，llama.cpp 都会用一长串算子调用（embedding、几十层的注意力和 FFN、最后的输出投影） 搭出这一步的完整计算图——可能有上千个节点。这一大串调用，没有一个真的在算，全是在填 op/src、连依赖； 直到图搭完、交给后端，才一次性算出这一步的 logits。所以你之前学的"一次 decode 内部是先建图、再执行"（L03），到这里就有了精确的含义： 建图 = 这一串惰性的算子调用，执行 = 后端按拓扑序把图算完。
建图时，ggml 把遇到的张量分成两类放好。判据很简单：看它有没有"来历"（op 是不是 NONE）：
| 类别 | 是什么 | 判据 | 执行时 |
|---|---|---|---|
| leafs（叶子） | 输入、权重、常量 | op == NONE（没有算子来历） | 不计算，直接用它的数据 |
| nodes（节点） | 算子的结果 | 有 op（如 MUL_MAT） | 按拓扑序逐个计算 |
用上面那个例子：x、W1、W2 是叶子（它们是给定的，不用算）； h、y 是节点（要算出来）。执行引擎只对 nodes 逐个动手，leafs 提供原料即可。 源码里这套分类逻辑在 ggml_visit_parents_graph（ggml/src/ggml.c）：碰到 op==NONE 且不是参数的张量， 就当叶子；否则当节点。理解了这条线，你看 ggml 调试输出里那一长串 nodes/leafs 就不再陌生。
为什么非要把这两类分开？因为它们在执行时的待遇完全不同。叶子是"已知量"——权重早就从模型文件里加载好了，输入也是你给定的， 它们的数据现成就在那儿，执行引擎碰都不用碰，直接拿来当原料。节点才是"未知量"——要靠算子把输入加工出来，是执行引擎真正干活的地方。 把"现成的"和"待算的"分开放，执行时就一目了然：跳过所有叶子，只对节点从头到尾算一遍，整张图就算完了。
还有个容易混淆的小问题：同一个张量，会不会既是这张图的叶子、又是另一张图的节点？会的。比如某层的输出 h，在"算 h 的那张子图"里它是节点（要算出来）， 但如果你把它当作另一段计算的给定输入，它就成了那段计算的叶子。叶子和节点不是张量的固有属性，而是它在当前这张图里扮演的角色—— 是起点（叶子），还是中间/末端的产物（节点）。想通这一层，你对"图"的理解就更灵活了。这也呼应了上一节的"族谱"比喻：同一个人，在自己这一支里是后辈，在更年轻一辈眼里又是长辈，全看你以谁为参照。
下面三个问题，想深究的同学点开看；只想抓主线的可以先跳过。
延迟计算看似多此一举，其实换来了三样宝贵的东西。其一，全局视野：建完图，ggml 能一眼看到"总共要算哪些、谁依赖谁"， 于是能精打细算地复用内存（下一课的 ggml-alloc 正靠这个把峰值内存压到极低）、还能合并或重排算子。
其二，多后端通用：图只是"该算什么"的纯描述，不绑定任何硬件，于是同一张图能交给 CPU、CUDA、Metal 任意后端去执行（呼应 L01、L07）。 其三，能反向求导：有了显式的依赖图，按链式法则反着走一遍就能自动算梯度（训练用）。这三样，都建立在"先不算、先记下来"之上。
反过来想，如果不建图、边调用边算，会失去什么？你会陷入"只见树木、不见森林"：算每一步时都不知道后面还要算什么、哪些中间结果以后还用得到， 于是只能保守地把每个中间结果都留着（内存爆炸），也没机会把相邻算子合并、或挑个更优的执行顺序。即时计算（eager）写起来直观，但把优化的余地全堵死了； 惰性建图牺牲了一点"所见即所得"的直觉，换来的是一整张可供优化的蓝图。对追求极致性能的推理引擎来说，这笔交易非常值。
是同一个，但这一课你才真正看到它的用途。L05 讲 ggml_tensor 字段时，只是告诉你"有 op 和 src 这两样，记录张量怎么来的"； 当时它们还像是孤立的标签。
到这一课，这些 src 指针串了起来：每个张量都记得自己的父节点，于是整张计算图，本质上就是"张量们靠 src 互相牵着手连成的一张网"。 op 说"这一步做什么运算"，src 说"输入从哪来"——两者合起来，一个张量就既是"一块数据"，又是"图里的一个运算节点"。 这个双重身份，是读懂 ggml 的关键。把"数据"和"图节点"这两个身份合在一个结构体里，是 ggml 区别于"先定义网络结构、再灌数据"那类框架的一个鲜明特点。
一个形象的说法：op 和 src 让每个张量都自带一张"出生证明"，写着"我是谁、由哪些张量经什么运算生出来的"。 把所有张量的出生证明顺着 src 串起来，就还原出了整个家族的族谱——这正是计算图。建图，本质上就是把散落的出生证明汇总成一本族谱。
不占多少。ggml_cgraph 本身只是几个指针数组——nodes 和 leafs 里装的是 指向张量的指针，不是张量数据的拷贝。一张几千节点的图，这些指针数组也就几十 KB。
张量的元数据（形状、op、src）则在 L08 说的 ggml_context arena 里，同样很轻。真正占内存的是张量的数据 （那一大片浮点数）——而在惰性建图阶段，配合 no_alloc=true，这些数据还没分配呢！要等下一课，看清整张图后，才由后端统一分配。 所以"建图"这一步，是出了名的轻。
这也带来一个实践上的好处：因为图这么轻、建起来这么便宜，llama.cpp 可以在每一步推理时都重新搭一张新图，而不必费心去复用上一步的图。 每步的 token 数、KV cache 长度可能都不一样，与其小心翼翼地改旧图，不如干脆重建一张——反正只是填一串指针，几乎不花时间。"轻量到可以随手重建"，是惰性建图的一个隐藏福利。
Last lesson you learned tensors come from ggml_context's memory pool. But here is a fact that surprises many: when you write c = ggml_mul_mat(a, b), that matrix multiply does not happen at all - ggml merely quietly records "c is produced from a and b by matmul". This lesson is about that "book it, don't do it" lazy graph building, one of the most elegant designs in the whole ggml engine.
Why does this matter? Because it upends your intuition that "calling a function should give a result immediately". In ggml, calling an operator is more like placing an order, writing a list than handing over goods on the spot. A whole network's forward pass is first fully "recorded as a graph", and only later computed at once, by plan. Grasp this two-stage "record first, compute later" and you truly see why ggml can be fast, frugal, and run across all kinds of hardware.
By the way, "lazy" is a compliment in programming, not "slacking" - it means "never compute ahead of the moment you truly must". This deferral often lets a program see the whole picture before acting, and make smarter arrangements. ggml applies this idea to the compute graph: all operator calls are saved up, then computed once the graph is complete - one of the roots of its efficiency.
First, correct the core misconception. In many frameworks, writing c = a @ b computes the multiply immediately, and c holds the result numbers. In ggml it is nothing like that: the c returned by ggml_mul_mat(a, b) is still an empty shell right now - it knows its shape, knows by whom and by what op it will be produced, but not a single number is computed yet. All it records is its "origin":
Pay special attention to the word "back". When you picture a neural network, arrows usually flow from input to output (how data travels); but the pointers ggml stores inside a tensor point the opposite way - the result points back to its inputs. Why store it backwards? Because at execution ggml's key question is "to compute this result, whom must I have first", and following a result back to its inputs answers that in one step. It is like tracing a person up to their parents, then grandparents - far more direct than calling the roll downward from an ancestor. This "from output back to inputs" direction is the key reused again and again in graph building and differentiation.
Look at the highlighted result c: its op field records "I was produced by matmul", and the two pointers src[0] and src[1] point back to a and b (note the arrow direction - the result points back to its inputs, hence "back-pointers"). You met these two when L05 introduced the ggml_tensor fields, where we only said "they record how it arose"; now you see their real use. Open the source and every operator function follows the same routine:
// simplified from ggml_mul_mat in ggml/src/ggml.c struct ggml_tensor * ggml_mul_mat(ctx, a, b) { result = ggml_new_tensor(ctx, GGML_TYPE_F32, ...); // just build an empty result tensor result->op = GGML_OP_MUL_MAT; // record "how I arose" result->src[0] = a; // record input 1 result->src[1] = b; // record input 2 return result; // return - not one multiply done! }
ggml_add, ggml_rms_norm, ggml_soft_max... nearly every operator looks like this: build a result tensor, fill in op and src, return. The real floating-point math waits for "execution" later (next lesson). Some operators need extra parameters (rope's rotation angle, softmax's scale), stored into the result's op_params via helpers like ggml_set_op_params_* - but again, only "recorded", not computed.
Think of it another way: this "empty shell" result tensor is really an IOU: it promises you "I will equal a times b in the future", but has not paid up yet. You can keep writing with this IOU - use it as the next operator's input and get a new IOU; layering on and on until you write the final output. Throughout, no real numbers are computed; what you accumulate is a stack of interlocking IOUs. Only at "execution" does ggml follow this stack, from the bottom up, redeeming each into real data. This "issue IOUs first, redeem them all later" is the meaning of lazy, and the core intuition this whole lesson keeps polishing.
One operator recording two or three srcs looks unremarkable; but once you write out a whole model's forward pass, these src pointers interlock layer by layer into a directed graph. Take the smallest example - two linear transforms y = W2 . (W1 . x):
inputs and weights; their op is NONE (no "origin", the graph's starting points).
the first operator result, src = [W1, x]; it is a "node".
the second operator result, src = [W2, h]; note it depends on the previous h.
Draw those two steps as a graph and it clicks: x, W1, W2 are leaves, h and y are operator nodes, and each node points back at its inputs via src - so "writing the expression" is the same as "wiring up a directed graph".
The beauty of this graph: starting from the output y and walking back along src pointers, you can find everything needed to compute it - y depends on W2 and h, and h depends on W1 and x. That is exactly what ggml_build_forward_expand(graph, y) does: starting from the output tensor you specify, recurse back along src and collect all dependencies in "who-computes-first" order (topological sort) into a ggml_cgraph:
# cf. ggml_build_forward_expand / ggml_visit_parents_graph in ggml/src/ggml.c def build_forward(graph, t): for s in t.src: # pull in all dependencies first build_forward(graph, s) # recurse back if t.op == NONE and not t.is_param: graph.leafs.append(t) # input/constant -> leaf else: graph.nodes.append(t) # operator result -> node (in dependency order)
Because it is "recurse to collect dependencies first, then add itself", the nodes in graph.nodes end up naturally topologically ordered: anything earlier never depends on anything later. So at execution you just compute front to back, and for each node its inputs are guaranteed already computed. ggml_cgraph itself is just a few arrays: nodes (operator results), leafs (inputs/constants), counts n_nodes/n_leafs, and capacity size (default GGML_DEFAULT_GRAPH_SIZE=2048).
Put this back into real llama.cpp: after loading a model, each inference step has llama.cpp build that step's full compute graph with a long string of operator calls (embedding, dozens of layers of attention and FFN, the final output projection) - possibly thousands of nodes. None of this big string actually computes; it all fills op/src and wires dependencies; only once the graph is built and handed to the backend are that step's logits computed at once. So the "one decode is build-then-execute inside" you learned (L03) now has a precise meaning: building = this string of lazy operator calls, executing = the backend computing the graph in topological order.
While building, ggml sorts the tensors it meets into two kinds. The criterion is simple: does it have an "origin" (is op NONE)?
| kind | what | criterion | at execution |
|---|---|---|---|
| leafs | inputs, weights, constants | op == NONE (no operator origin) | not computed, its data is used directly |
| nodes | operator results | has an op (e.g. MUL_MAT) | computed one by one in topological order |
With the example above: x, W1, W2 are leafs (given, not computed); h, y are nodes (to be computed). The execution engine only acts on nodes; leafs just supply raw material. In source this classification lives in ggml_visit_parents_graph (ggml/src/ggml.c): a tensor with op==NONE and not a param becomes a leaf, otherwise a node. Grasp this and the long list of nodes/leafs in ggml's debug output is no longer a mystery.
Why insist on separating these two kinds? Because their treatment at execution is completely different. Leafs are "known quantities" - weights were long since loaded from the model file, inputs are given by you; their data is right there, and the execution engine need not touch them, using them as raw material. Nodes are the "unknowns " - they must be produced from inputs by operators, where the execution engine actually works. Keeping "ready-made" and "to-be-computed" apart makes execution obvious: skip all leafs, compute only nodes front to back, and the whole graph is done.
One more easily-confused point: can the same tensor be a leaf of this graph and a node of another? Yes. A layer's output h is a node in "the subgraph that computes h" (it must be computed), but if you treat it as a given input to another computation, it becomes that computation's leaf. Leaf and node are not a tensor's inherent property but the role it plays in the current graph - a starting point (leaf) or a mid/end product (node). See this and your grasp of "graphs" grows more flexible. It echoes the "family tree" metaphor: the same person is a junior in their own branch and an elder to a younger generation, all depending on your reference point.
Three questions below; open them if you want depth, skip them if you only want the main line.
Deferring computation looks redundant but buys three precious things. One, a global view: once the graph is built, ggml sees at a glance "everything to be computed and who depends on whom", so it can carefully reuse memory (next lesson's ggml-alloc rides on this to crush peak memory) and merge or reorder operators.
Two, backend-agnostic: the graph is a pure description of "what to compute", bound to no hardware, so the same graph can go to CPU, CUDA, or Metal to execute (echoing L01, L07). Three, autodiff: with an explicit dependency graph, walking it backwards by the chain rule computes gradients automatically (for training). All three rest on "don't compute yet, just record".
Conversely, what would you lose by not building a graph, computing as you call? You would be stuck "seeing trees, not the forest": computing each step with no idea what comes later or which intermediates are still needed, so you can only conservatively keep every intermediate (memory blow-up) and have no chance to merge adjacent operators or pick a better order. Eager computation is intuitive to write but seals off all room for optimization; lazy building sacrifices a bit of "what you see is what you get" for a whole blueprint open to optimization. For an inference engine chasing peak performance, that is a very worthwhile trade.
The same ones - but only now do you see their use. When L05 covered the ggml_tensor fields, it just told you "there are op and src that record how a tensor arose"; back then they seemed like isolated labels.
This lesson strings them together: each tensor remembers its parents, so the whole compute graph is essentially "tensors holding hands via src into a web". op says "what op this step does", src says "where inputs come from" - together, a tensor is both "a block of data" and "a compute node in the graph". This dual identity is the key to reading ggml. Fusing the two identities - "data" and "graph node" - into one struct is a hallmark distinguishing ggml from frameworks that "define the network structure first, then pour in data".
A vivid way to put it: op and src give every tensor its own "birth certificate", stating "who I am, and from which tensors by what operation I was born". String all the birth certificates along src and you reconstruct the whole family's genealogy - that is the compute graph. Building the graph is essentially gathering scattered birth certificates into one genealogy book.
Not much. ggml_cgraph itself is just a few pointer arrays - nodes and leafs hold pointers to tensors, not copies of tensor data. A graph of a few thousand nodes is just tens of KB of pointer arrays.
A tensor's metadata (shape, op, src) sits in L08's ggml_context arena, also light. What really takes memory is a tensor's data (that big slab of floats) - and during lazy graph building, with no_alloc=true, that data is not even allocated yet! It waits for the next lesson, where after seeing the whole graph the backend allocates it in one pass. So "building the graph" is famously light.
This brings a practical perk: because the graph is so light and cheap to build, llama.cpp can build a fresh graph at every inference step, without bothering to reuse the previous step's. Each step's token count and KV-cache length may differ, so rather than carefully patching the old graph, it just rebuilds one - it is only filling a string of pointers, costing almost no time. "Light enough to rebuild casually" is a hidden bonus of lazy graph building.