前面好几课都在提那个 .gguf 文件——模型就装在里面。可它到底长什么样？这一课我们把一个 GGUF 文件从头到尾拆开：文件头、元数据、张量清单、对齐填充、数据段， 再看 llama.cpp 怎么用 mmap 零拷贝地把它加载进内存、实现大模型的"秒加载"。这是第三部分的收尾课，把前面学的内存、图、算子、量化，落到磁盘上的一个文件里。
为什么值得专门讲文件格式？因为 GGUF 是 llama.cpp 的"统一入口"：你从 HuggingFace 下载、用 L02 的转换脚本得到的，就是这个文件；运行时被读进来的，也是它。 读懂 GGUF，你就把"磁盘上的模型"和"内存里的张量"两端连了起来。
先看全景。一个 GGUF 文件从头到尾是这样排的：开头 4 字节是 magic "GGUF"（一眼认出"这是 GGUF"），接着 version（当前是 3），然后是张量数量和 KV 数量， 再往后是一串 metadata 键值对、一串 tensor info（张量清单），按对齐补一段 padding，最后才是真正的张量数据（很可能就是 L12 那些量化块）。
文件最开头的 4 个字节，一眼认出"这是一个 GGUF 文件"。
格式版本号，当前是 3；向后兼容靠它来判断。
先报数：后面有多少个张量、多少个键值对。
自描述信息：架构、超参、词表、聊天模板……模型的"说明书"。
张量清单：每个张量的 name / dims / type / offset。
补几个字节，让数据段起点对齐到 32 字节边界。
真正的权重字节，常常就是 L12 讲的那些量化块。
把这张图翻译成"伪结构"，就是这样（布局见 ggml/include/gguf.h 的头部注释）：
// 简化自 ggml/include/gguf.h 的格式说明 "GGUF" // 4 字节 magic version : u32 // = 3 n_tensors : i64 n_kv : i64 kv_pairs : [ (key:str, type:gguf_type, value), ... ] // 自描述 tensor_infos : [ (name:str, n_dims:u32, dims[]:i64, type, offset:u64), ... ] <padding to alignment> // 默认 32 字节对齐 tensor_data : <raw bytes> // 权重(常是 L12 的量化块)
逐段看：magic + version 是"身份证"，加载器一上来就核对——magic 不是 "GGUF" 直接拒绝，version 不认识就报错。n_tensors / n_kv 是两个计数， 告诉加载器"接下来要读多少条"。再往后两大块——metadata 和 tensor infos——是这一课的重点，分别回答"模型是什么"和"每个张量在哪"。
顺便说说名字：GGUF 是 "GGML Universal File" 的意思，G-G-M-L 来自作者 Georgi Gerganov 的名字缩写（也是 ggml 库名的由来）。它的前身是更简单的 GGML/GGJT 等格式， 因为不够灵活、扩展时老破坏兼容，才演进成今天这个带版本号、可自由扩展的 GGUF。了解这段渊源，能帮你看懂网上一些老教程里为什么会出现 ".bin"、"ggml-model" 这类旧叫法。
也许你会问：为什么不直接用现成的格式（比如 PyTorch 的 .pt、或 safetensors）？因为 llama.cpp 要的东西很特别——它要把量化块（L12 那些 q4_K、q6_K）原样存进去、要能 mmap 零拷贝加载、 还要把超参和词表自带在文件里，好让纯 C/C++ 端独立读取。这些需求叠加起来，催生了 GGUF 这个为"端侧推理"量身定做的格式。
metadata 是一串键值对，专门回答"这个模型是什么"。它最重要的特性是自描述——加载器不必去别处找配置文件、也不必"猜"模型结构，所有超参、词表、聊天模板全写在文件里。 下面是几个真实的键：
| 键 key | 类型 type | 含义 |
|---|---|---|
| general.architecture | str | 模型架构，如 "llama"、"qwen2"——决定怎么建图 |
| llama.block_count | u32 | 层数（L04 的 n_layer） |
| llama.embedding_length | u32 | 隐藏维度（L04 的 n_embd） |
| tokenizer.ggml.tokens | array | 词表：所有 token 的字符串 |
| general.alignment | u32 | 对齐字节数（可覆盖默认 32） |
看这些键就明白：L04 说"从 GGUF 头里直接读到 n_layer、n_embd"，读的就是 llama.block_count、llama.embedding_length 这两个 KV。 键名还带命名空间（general.、llama.、tokenizer.），架构相关的超参用架构名做前缀，于是同一套 GGUF 结构能装下任意模型。
每个值都有一个 gguf_type 标明类型：u8/i8/u32/i32/f32/bool/string/array 等等（见 ggml/include/gguf.h 的 enum gguf_type）。 正因为类型是写在文件里的，读取方不用预先知道"这个键是数还是字符串"，照着 type 解析即可——这就是"自描述"在字节层面的落实。
还有一类 KV 专门描述"这个文件本身"：general.name（模型名）、general.file_type（整体量化档位，对应 L12 的 Q4_K_M 之类）、general.quantization_version 等。 它们不影响怎么建图，却让工具能一眼报出"这是什么模型、量化到几 bit"——你在加载日志里看到的那些模型信息，多半就是从这些 KV 读出来的。
再看 gguf_type 这套类型本身。它覆盖了从 8 位到 64 位的整数、32/64 位浮点、布尔、字符串，还有一个 array 表示"一串同类型的值"——词表 tokenizer.ggml.tokens 就是个字符串数组。 有了这套类型系统，metadata 几乎能装下任意结构化的配置，而读取方只靠一个 type 标记就知道该怎么解析每个值。
顺带一提，这套结构在 Python 侧由 gguf-py 读写（L02 的转换脚本就用它）：写入时先攒齐所有 KV 和 tensor info、算好对齐和 offset，再一次性落盘。所以一个 GGUF 文件总是头部完整、布局规整的， 不会出现"写了一半结构不全"的情况——这也方便了 mmap 这种"信任头部、直接定位"的读取方式。
metadata 讲完"模型是什么"，tensor info 回答"每个张量在哪、长什么样"。每条 tensor info 记四样东西：name（张量名，如 blk.0.attn_q.weight）、 dims（各维大小）、type（L05 的 ggml_type，也包括 L12 的量化类型）、以及 offset（在数据段里的相对偏移）。
为什么 offset 记的是"相对数据段起点"的偏移，而不是文件里的绝对位置？因为这样更稳健：头部（KV、tensor info）的长度会随模型不同而变，要是用绝对偏移，头部一变所有 offset 都得重算； 用相对偏移，数据段内部的排布就和前面头部有多长解耦了——加载器把"数据段起点"算出来一次，再加上各张量的相对 offset 即可；这也是为什么往文件里追加张量时，已有张量的 offset 大多不用改动。
张量的 name 不是随便起的，而是一套有规律的命名约定。比如 blk.0.attn_q.weight 表示"第 0 层（block 0）的注意力 Q 投影权重"，token_embd.weight 是词嵌入表。 加载器正是靠这套名字把文件里的张量一一对应到 L08 建出的模型结构上——名字对不上，权重就装不进对应的位置。
这里就用上了对齐。数据段的起点、以及每个张量的 offset，都会对齐到 GGUF_DEFAULT_ALIGNMENT = 32 字节（可被 general.alignment 覆盖）。 为什么要对齐？因为 CPU/GPU 的 SIMD 指令按对齐地址读取最快，mmap 也按内存页管理；让数据落在整齐的边界上，后端读起来更高效、也更省事（详见深挖 1）。
有了清晰的布局，加载就分两步：先读头部、再映射数据。第一步用 gguf_init_from_file 把 magic、version、所有 KV 和 tensor info 读进来，建好一张"张量清单"； 第二步把整个文件用 mmap 只读映射进地址空间，每个张量的 data 指针直接落在映射上的对应位置。
// 伪代码: gguf_init_from_file + llama_mmap (src/llama-mmap.cpp) ctx = gguf_init_from_file(path) // 读 magic/version/KV/tensor infos assert magic == "GGUF" and version == 3 mapping = mmap(file, PROT_READ) // 整文件只读映射, 不拷贝 for t in tensors: t.data = mapping + data_off + t.offset // 张量数据直接指进映射
再串一遍整条加载链路，把第三部分前几课都接起来：gguf_init_from_file 读出超参（metadata）-> 按超参建出 L08 的 ggml_context 和张量结构 -> 张量的 data 指针指进 mmap 映射（权重零拷贝就位）-> 之后就是 L09 建图、L10 执行、L11 算子、L12 解量化。一个 .gguf 文件，就这样变成了一张能跑的计算图。
为什么要"先读头部、再映射数据"分两步，而不是一股脑全读进来？因为这两步的代价天差地别：头部（metadata + tensor info）通常只有几十 KB，实读进内存毫无压力；而数据段动辄几 GB， 要是也老老实实读进来就太慢太占内存了，于是改用 mmap 把它留在磁盘上、按需取页。这种"小的实读、大的映射"的分工，是大模型加载又快又省的关键。
当然 mmap 也不是没有代价。它依赖操作系统的页缓存，第一次真正用到某页时仍要从磁盘读，所以"秒加载"省的是"启动时的整体拷贝"，而不是把磁盘读取变没了。 此外某些场景（如需要把权重整体搬上 GPU 显存）也未必用得上 mmap。但对"CPU 推理、内存就是权重所在地"的常见情形，mmap 几乎是免费的加速。
两个原因。其一是计算效率：后端（CPU 的 SIMD、GPU 的 kernel）按对齐地址成批读数最快，地址错位会拖慢、甚至需要额外处理。让每个张量数据从对齐边界开始，后端就能用最高效的加载指令。
其二是mmap 友好：mmap 按内存页映射，数据对齐到规整边界，按页处理更顺、更不容易跨页拖慢。GGUF_DEFAULT_ALIGNMENT 默认 32 字节，模型也可以用 general.alignment 这个 KV 覆盖它。
这其实和 L12 的"字节布局"是同一种思维：为了让机器读得快，愿意花一点点空间在对齐/填充上。几个 padding 字节换来整个数据段的高效访问，非常划算。
核心是自描述 + 可扩展。老格式把少量超参硬编码在文件头，加一个新字段就可能破坏兼容；GGUF 用键值对存元数据，加一个新超参只是多一个 KV，老加载器读到不认识的键直接跳过，不会崩。
而且 GGUF 把超参、词表、聊天模板统一收进一个文件，免去了"权重 + 一堆外部配置"的拼凑。version 字段（现在是 3）则明确标记格式演进，让工具能判断兼容性。
这种"自描述 + 版本化"的设计，是 GGUF 能成为生态通用格式的关键：模型作者、转换工具、推理引擎各自独立演进，只要遵守同一套 KV 约定就能互通。
要分清虚拟内存和物理内存。mmap 会让进程的虚拟地址空间一下子"涨"出几 GB（整个文件的大小），但这只是地址映射，物理内存是按需、惰性载入的——用到哪页才占哪页。
更妙的是这些页是文件页：可被操作系统在内存紧张时回收（反正磁盘上有原件），多个进程映射同一个文件时还能共享同一份物理页。所以同一台机器起多个实例，权重内存可以共用，省得多。
这也解释了为什么用 top 看 llama.cpp 进程，VIRT（虚拟）很大、RES（常驻）却没那么夸张：差的那部分就是"映射了但还没真正读进来、或已被回收"的文件页。
Several earlier lessons kept mentioning that .gguf file - the model lives inside it. But what does it actually look like? This lesson takes a GGUF file apart end to end: the header, the metadata, the tensor list, the alignment padding, the data section - then sees how llama.cpp loads it into memory with mmap zero-copy to achieve a large model's "instant load". This is Part 3's closing lesson, landing everything learned about memory, graphs, operators, and quantization into a single file on disk.
Why devote a lesson to a file format? Because GGUF is llama.cpp's "unified entry point": what you download from HuggingFace and convert with L02's script is this file; what gets read in at runtime is also it. Read GGUF and you connect the two ends - "the model on disk" and "the tensors in memory".
Start with the panorama. A GGUF file is laid out, front to back, like this: the first 4 bytes are the magic "GGUF" (instantly recognizing "this is GGUF"), then the version (currently 3), then the tensor count and KV count, followed by a run of metadata key-value pairs, a run of tensor infos (the tensor list), a stretch of padding for alignment, and only then the actual tensor data (very likely those L12 quantized blocks).
The first 4 bytes of the file, instantly recognizing "this is a GGUF file".
The format version, currently 3; backward compatibility hinges on it.
A head count first: how many tensors and how many key-value pairs follow.
Self-describing info: architecture, hyperparameters, vocab, chat template... the model's "manual".
The tensor list: each tensor's name / dims / type / offset.
A few bytes added so the data section starts on a 32-byte boundary.
The actual weight bytes, often exactly those quantized blocks from L12.
Translating that figure into a "pseudo-struct" gives this (layout per the header comment in ggml/include/gguf.h):
// simplified from the format description in ggml/include/gguf.h "GGUF" // 4-byte magic version : u32 // = 3 n_tensors : i64 n_kv : i64 kv_pairs : [ (key:str, type:gguf_type, value), ... ] // self-describing tensor_infos : [ (name:str, n_dims:u32, dims[]:i64, type, offset:u64), ... ] <padding to alignment> // 32-byte default alignment tensor_data : <raw bytes> // weights (often L12 quantized blocks)
Section by section: magic + version are the "ID card", checked the moment the loader starts - if the magic is not "GGUF" it refuses outright, and an unknown version errors out. n_tensors / n_kv are two counts telling the loader "how many entries to read next". After them come the two big blocks - metadata and tensor infos - the focus of this lesson, answering "what the model is" and "where each tensor is" respectively.
A word on the name: GGUF stands for "GGML Universal File", and G-G-M-L comes from the initials of the author Georgi Gerganov (also the origin of the ggml library name). Its predecessors were simpler formats like GGML/GGJT, which were too inflexible and kept breaking compatibility when extended, so they evolved into today's versioned, freely-extensible GGUF. Knowing this lineage helps you understand why some old online tutorials mention ".bin" or "ggml-model" names.
You might ask: why not use an existing format (PyTorch's .pt, or safetensors)? Because llama.cpp needs something special - it must store quantized blocks (L12's q4_K, q6_K) verbatim, load them mmap zero-copy, and carry hyperparameters and vocab inside the file so a pure C/C++ side can read them independently. These needs stacked together gave rise to GGUF, a format tailored for "on-device inference".
Metadata is a run of key-value pairs that answers "what this model is". Its most important property is being self-describing - the loader need not hunt for a config file elsewhere, nor "guess" the model structure; all hyperparameters, vocab, and chat template are written right in the file. Here are a few real keys:
| key | type | meaning |
|---|---|---|
| general.architecture | str | model architecture, e.g. "llama", "qwen2" - decides how the graph is built |
| llama.block_count | u32 | layer count (L04's n_layer) |
| llama.embedding_length | u32 | hidden dimension (L04's n_embd) |
| tokenizer.ggml.tokens | array | vocab: the strings of all tokens |
| general.alignment | u32 | alignment byte count (can override the default 32) |
These keys make it clear: when L04 said "read n_layer and n_embd straight from the GGUF header", it was reading exactly the llama.block_count and llama.embedding_length KVs. Keys also carry a namespace (general., llama., tokenizer.), with architecture-specific hyperparameters prefixed by the architecture name, so one GGUF structure can hold any model.
Every value carries a gguf_type marking its type: u8/i8/u32/i32/f32/bool/string/array and so on (see enum gguf_type in ggml/include/gguf.h). Because the type is written in the file, the reader need not know in advance "is this key a number or a string"; it just parses per the type - this is "self-describing" realized at the byte level.
Another class of KV describes "the file itself": general.name (model name), general.file_type (the overall quantization level, matching L12's Q4_K_M and the like), general.quantization_version, and so on. They do not affect how the graph is built, yet they let tools report at a glance "what model this is, quantized to how many bits" - the model info you see in load logs is mostly read from these KVs.
Look at the gguf_type system itself. It covers integers from 8 to 64 bits, 32/64-bit floats, bool, string, and an array meaning "a run of same-typed values" - the vocab tokenizer.ggml.tokens is a string array. With this type system, metadata can hold almost any structured config, while the reader needs only a type tag to know how to parse each value.
By the way, this structure is read and written on the Python side by gguf-py (used by L02's conversion script): on write it first gathers all KVs and tensor infos, computes alignment and offsets, then flushes in one go. So a GGUF file is always complete-headered and tidily laid out, never "half-written with a partial structure" - which also suits mmap's "trust the header, address directly" style of reading.
With metadata covering "what the model is", tensor info answers "where each tensor is and what it looks like". Each tensor info records four things: name (the tensor name, e.g. blk.0.attn_q.weight), dims (the size of each dimension), type (L05's ggml_type, including L12's quantized types), and offset (the relative offset within the data section).
Why does offset record an offset "relative to the data section start" rather than an absolute file position? Because it is more robust: the header (KVs, tensor info) length varies by model, and with absolute offsets a header change would force recomputing every offset; with relative offsets, the data section's internal layout is decoupled from how long the header is - the loader computes the "data section start" once, then adds each tensor's relative offset. This is also why, when appending tensors to a file, most existing tensors' offsets need no change.
A tensor's name is not arbitrary but follows a regular naming convention. For example blk.0.attn_q.weight means "the attention Q-projection weight of layer 0 (block 0)", and token_embd.weight is the token-embedding table. The loader uses exactly these names to match the file's tensors one-to-one onto the model structure L08 builds - if a name does not match, the weight cannot be placed.
This is where alignment comes in. The start of the data section, and each tensor's offset, are aligned to GGUF_DEFAULT_ALIGNMENT = 32 bytes (overridable by general.alignment). Why align? Because CPU/GPU SIMD instructions read fastest from aligned addresses, and mmap manages memory in pages; letting data fall on tidy boundaries makes backend reads more efficient and simpler (see Dig deeper 1).
With a clear layout, loading is two steps: read the header, then map the data. Step one uses gguf_init_from_file to read the magic, version, all KVs and tensor infos, building a "tensor list"; step two maps the whole file read-only into the address space with mmap, and each tensor's data pointer lands directly at its place in the mapping.
// pseudocode: gguf_init_from_file + llama_mmap (src/llama-mmap.cpp) ctx = gguf_init_from_file(path) // read magic/version/KV/tensor infos assert magic == "GGUF" and version == 3 mapping = mmap(file, PROT_READ) // whole-file read-only map, no copy for t in tensors: t.data = mapping + data_off + t.offset // tensor data points straight into the map
Stringing the whole load path once more, tying together Part 3's earlier lessons: gguf_init_from_file reads the hyperparameters (metadata) -> builds L08's ggml_context and tensor structures per those hyperparameters -> tensors' data pointers point into the mmap mapping (weights in place, zero-copy) -> then comes L09 graph-building, L10 execution, L11 operators, L12 dequantization. A single .gguf file thus becomes a runnable compute graph.
Why two steps - "read the header, then map the data" - instead of reading it all at once? Because the two costs differ enormously: the header (metadata + tensor info) is usually only tens of KB, so actually reading it into memory is trivial; the data section is often several GB, and dutifully reading it in would be far too slow and memory-hungry, so mmap is used to leave it on disk and fetch pages on demand. This "read the small, map the large" division is the key to fast, frugal large-model loading.
Of course mmap is not free. It relies on the OS page cache, and the first real touch of a page still reads from disk, so "instant load" saves "the whole copy at startup", not disk reads themselves. Some scenarios (like moving weights wholesale onto GPU VRAM) may not use mmap either. But for the common case of "CPU inference where memory is where the weights live", mmap is almost-free speedup.
Two reasons. First, compute efficiency: backends (CPU SIMD, GPU kernels) read in batches fastest from aligned addresses, while misaligned addresses slow things down or need extra handling. Starting each tensor's data on an aligned boundary lets the backend use its most efficient load instructions.
Second, mmap-friendliness: mmap maps in memory pages, and data aligned to tidy boundaries is smoother to handle page by page and less prone to cross-page slowdowns. GGUF_DEFAULT_ALIGNMENT defaults to 32 bytes, and a model can override it with the general.alignment KV.
This is the same mindset as L12's "byte layout": to let the machine read fast, spend a little space on alignment/padding. A few padding bytes buy efficient access to the whole data section - a great bargain.
The core is self-describing + extensible. The old format hard-coded a few hyperparameters in the header, where adding a new field could break compatibility; GGUF stores metadata as key-value pairs, so adding a new hyperparameter is just one more KV, and an old loader simply skips keys it does not recognize without crashing.
GGUF also folds hyperparameters, vocab, and chat template into one file, sparing you the "weights + a pile of external configs" patchwork. The version field (now 3) explicitly marks format evolution, letting tools judge compatibility.
This "self-describing + versioned" design is the key to GGUF becoming the ecosystem's common format: model authors, conversion tools, and inference engines can each evolve independently, interoperating as long as they honor the same KV conventions.
Distinguish virtual memory from physical memory. mmap makes the process's virtual address space suddenly "grow" by several GB (the whole file's size), but that is only an address mapping; physical memory is loaded on demand, lazily - a page is occupied only when used.
Better still, these are file pages: the OS can reclaim them under memory pressure (the original is on disk anyway), and multiple processes mapping the same file can share the same physical pages. So running several instances on one machine can share weight memory, saving a lot.
This also explains why, watching a llama.cpp process with top, VIRT (virtual) is huge while RES (resident) is not so dramatic: the difference is the file pages "mapped but not yet really read in, or already reclaimed".