第一次打开 llama.cpp，几百个文件、十几个目录，很容易发懵：到底先看哪儿？其实它分层非常清晰，每个目录各司其职。 这一课先给你一张"校园地图"——认清顶层目录在干什么，再把它们对回上一课的"四层"，接着看清一个模型从训练到运行 要穿过哪几个目录、边界落在哪里。读完这张图，无论你是想跑工具还是钻源码，都不会迷路。
站在仓库根目录，先别急着点开文件，按"这个目录到底在干什么"把主要目录扫一遍。下面这张表就是地图的图例：
| 目录 | 作用 |
|---|---|
| ggml/ | 自研张量引擎：张量 · 计算图 · 算子 · 后端调度；独立子项目（自带 include/ 与 src/），是整个项目最底层、也最"硬"的一块 |
| ggml/src/ggml-cpu · ggml-cuda · ggml-metal · ggml-vulkan … | 各硬件后端，把算子真正算在 CPU / GPU 上（还有 hip / sycl / musa / opencl 等十余种） |
| src/ | llama 推理库：把引擎拼成"会话级"推理——llama-model-loader · llama-graph · llama-kv-cache · llama-sampler · llama-vocab · llama-chat · llama-grammar · llama-quant … |
| include/ | 公共 C API：llama.h（唯一对外契约）；llama-cpp.h（C++ 薄封装 + RAII） |
| common/ | 复用工具 / 胶水：arg（参数解析）· sampling（采样封装）· chat · log · download · json-schema-to-grammar …；给程序用，不是推理库本体 |
| tools/ | 可执行程序：llama-cli · llama-server · llama-quantize · llama-mtmd-cli（多模态）· llama-perplexity · llama-bench … |
| examples/ | 小型示例程序：simple 用两百多行演示最小推理，是最佳"可读"入口 |
| gguf-py/ | Python 的 GGUF 读写库：转换脚本靠它写出 .gguf |
| convert_hf_to_gguf.py 等 | Python 转换脚本（共 4 个 convert_*.py：3 个转换器 + 1 个 tokenizer 哈希维护脚本；主力是 HuggingFace -> GGUF） |
| models/ · tests/ · docs/ · grammars/ · cmake/ | 模型数据 / 测试 / 文档 / GBNF 示例 / 构建系统 |
一个简单的判断法：越往下越"硬"——ggml/ 偏数学与硬件，src/ 偏模型与会话逻辑，tools/ 与 common/ 偏"给人用"。而真正对外暴露的，自始至终只有 include/llama.h 这一个公共头文件。
目录虽多，真正的主角只有四块，每块咬死一件事、互不越界。先认清这四条边界，再回头看那张目录表，剩下的就全是细节了：
| 这一块 | 只干一件事 | 大致落在 |
|---|---|---|
| 引擎 | 造算力：定义张量、把运算组织成计算图、调度到各后端真正算出来 | ggml/ |
| 推理库 | 把算力拼成"会话"：加载权重、搭图、KV cache、采样、分词、聊天模板 | src/llama-* |
| 程序 / 胶水 | 给人用：命令行、HTTP 服务、量化器，外加把库粘成程序的通用件 | tools/ · common/ |
| 模型准备 | 把外部模型搬进来：HuggingFace 权重转成 .gguf | gguf-py/ · convert_*.py |
为什么非要拆这么开？因为这四件事的变化节奏完全不同：后端算子要追着新硬件、新指令集不断改；推理逻辑要随新模型结构演进；命令行参数与服务接口随用户需求增删；转换脚本则要跟着上游模型格式跑。把它们焊死在一起，改一处就得提心吊胆会不会崩另一处。拆开之后，各自独立演进、独立测试、独立复用——给 ggml 加一个新后端，不必动 src/llama-* 一行；新增一个模型结构，也碰不到底层算子。这就是"边界清晰"最实在的回报。
还有个刻意的设计值得单拎出来：上百个内部文件里，真正对外公开的只有 include/llama.h 一个头。把对外的口子收得这么窄，换来三重好处：其一，内部随便改——只要 llama.h 里的函数签名不变，src/llama-* 内部怎么重构、换数据结构、调算法，都不会惊动外面的使用者；其二，对外契约稳定，使用方升级版本时心里有底，不必追着内部细节东奔西跑；其三，因为暴露的是一套 C ABI，几乎所有语言都能绑定——Python、Go、Rust、Node 等都能透过这个 C 接口调用引擎，社区里大量的语言绑定正是这么搭起来的。把对外收成一个小口，内部才换来放手重构的自由。
顺带提一句：紧挨着 llama.h 还有个 llama-cpp.h，它只是给 C++ 用户的一层薄封装——用 RAII（智能指针）自动管理 llama_model / llama_context 的释放，省去手动 free。它并不扩大对外暴露面，只是把同一个 C 接口包得更顺手，所以"对外只有一个契约"这句话依然成立。
把上面这些目录映射回上一课的"四层"结构，就一目了然：
除了这条主干，还有两条支线：① 模型准备——gguf-py/ 加 convert_*.py（Python）把 HuggingFace 模型转成 .gguf，再喂给引擎；② 配套支撑——common/（把库粘成程序的胶水）以及 tests/ · docs/ · cmake/。
为什么叫"支线"？因为一次推理请求只在主干四层里上下穿行，根本不会跑进这两条线：模型准备在跑之前就一次性做完了（产出 .gguf 便退场），配套支撑则像脚手架围在主干周围——common/ 帮程序少写样板，tests/ · docs/ · cmake/ 管测试、文档与构建。把"运行时会经过的"和"运行前 / 运行外的"分清楚，读源码时就不会把脚手架错当成承重墙。
把目录串起来看，一个模型的"一生"其实是一条很直的流水线：左边在 Python 里准备，右边在 C++ 里运行，中间靠一个文件交接。顺着这条线走一遍，就知道每个目录在整条链路里站在哪一站：
左半截是 Python（准备），右半截是 C++（运行），两者的边界就是中间那个 .gguf 文件。
这条边界很关键：转换脚本只在准备阶段跑一次，产出 .gguf 后就退场；运行时完全不碰 Python，只认这一个文件。所以同一个 .gguf，既能喂给 llama-cli，也能喂给 llama-server 或 examples/simple——它们共享同一套加载与推理代码。
把这条流水线落到真实命令上，最常见的就是"转换 -> 量化 -> 运行"三步。下面这段是最小可跑的骨架——左边 Python 准备、右边 C++ 运行，中间仍旧靠那个 .gguf 交接：
# 模型从准备到运行的完整管道 # 1) 转换：HuggingFace 模型 -> GGUF（Python 侧，--outfile 指定输出名） python convert_hf_to_gguf.py ./my-model --outfile my-model.gguf # 16 位浮点（默认 auto） # 2) 量化（可选，压小） llama-quantize my-model.gguf my-model-Q4.gguf Q4_0 # 3) 运行（C++ 侧） llama-cli -m my-model-Q4.gguf -p "你好"
三步正好落在三个目录：第一步在 gguf-py/ + convert_*.py（Python）里跑，吐出一个 FP16 的 .gguf；第二步 llama-quantize（来自 tools/）把它压成 Q4_0 这类低位宽版本，体积骤降到约四分之一；第三步 llama-cli 加载量化后的文件，真正"跑出字"。两点值得记牢：量化是可选的——不在意体积，直接拿第一步的 .gguf 去跑也行；而第二、三步全程不碰 Python，只认那一个文件，这正是前面那句"边界就在 .gguf"落到命令上的样子。
convert_hf_to_gguf.py 这一步具体在做什么？它读入 HuggingFace 目录里的 config.json（超参）、分词器与 safetensors（权重），按模型架构把张量改名、必要时转置，再连同元数据一起写成一个 .gguf。换句话说，这条"码头"把外部世界五花八门的模型，统一翻译成引擎只认的那一种格式——之后 C++ 侧就再不必关心它原本长什么样了。
顺带把体积感建立起来：第一步产出的 FP16 文件，7B 模型约 14 GB；第二步 Q4_0 量化后降到约 4 GB——同一条命令链，跑完就把一个"原本要显卡"的模型压成了"普通内存就能装"的文件。这也正呼应上一课那条"从重到轻"的流水线，只是这次落在了真实命令上。
想读源码，却不知道从哪下手？与其从第一个文件啃到最后一个，不如先想清楚你的目标，再选对应的入口往下钻。常见的三种目标，正好对应三个入口：
llama-cli/llama-server 跑起来，再读 examples/simple 的最小调用llama-model-loader -> llama-graph -> llama-kv-cache -> llama-samplerggml.c 与各 ggml-* 后端，看张量/算子/调度怎么实现如果只想挑一个地方开始，首推 examples/simple：它用两百行左右把"加载模型 -> 分词 -> 解码循环 -> 采样 -> 输出"这条主线完整跑了一遍，没有服务、多模态那些枝节干扰。把它从头读到尾一遍，再把每个调用对回上面四层——这个函数属于推理库还是引擎、走的是 llama.h 里哪个接口——整张地图就从"看过"变成"走过"了。
不管从哪条路进，记住对外只有一个公共契约 include/llama.h：搞不清某个能力归谁管时，先回到这个头文件，看它把哪些函数暴露给了外面。先认入口，再逐层往下钻，比漫无目的地翻文件高效得多。
下面四个问题，想把这张地图看透的同学点开看；只想记住主干的可以先跳过。
示例：一个 .gguf 从头到尾大致分四段，按顺序紧挨着排在同一个文件里：
逐段拆开看：① 文件头是四字节 magic "GGUF" 加一个版本号（当前为 3），后面记着"有多少个张量、多少条元数据"；② 元数据 KV 是一长串键值对——超参如 <arch>.block_count（层数）、<arch>.embedding_length（隐藏维度），词表如 tokenizer.ggml.tokens，会话模板如 tokenizer.chat_template，都塞在这一段；③ 张量信息逐个登记每个权重张量的名字、形状、类型，以及它在文件里的偏移；④ 最后才是真正的 张量数据——一块块权重数值，位置正由上面那些偏移指过去。
为什么这么设计：全部塞进一个文件，加载时直接 mmap 进内存、按偏移随用随取，不必先解压或整体拷贝（CPU 推理时近乎零拷贝；用 GPU 后端则权重还会再拷进显存）。超参数与词表都自带，引擎读完头部就知道"这是什么模型、该怎么搭计算图"，免配置文件、免 Python。把元数据排在权重前面也有讲究：引擎先读一小段头部把结构看清楚，再决定怎么映射后面那一大坨权重数据。还有个额外好处：因为是只读映射，多个进程能共享同一份权重内存，同机起多个实例时省内存又省加载时间。
源码：读写与解析在 ggml/src/gguf.cpp（gguf_kv / gguf_tensor_info 等结构）；元数据键名的常量集中定义在 gguf-py/gguf/constants.py；把这些元数据接到 llama 模型上、按 key 取超参的，是 src/llama-model-loader.cpp。
替代：更早的 GGML / GGJT 等老格式也干过同样的活，但字段零散、版本兼容差，已被 GGUF 取代（仓库里还留着一个 convert_llama_ggml_to_gguf.py 专门把老格式转过来）。
一句话：因为同一个引擎被多个项目复用——ggml 不只为 llama.cpp 服务，所以它被切成一个能单独存在的子项目。
例子：同作者的 whisper.cpp（语音转文字）等项目也直接拿 ggml 当计算引擎；它们和 llama.cpp 共享同一套张量、算子与后端代码，只是上层逻辑不同。
怎么保持同步：ggml 有自己独立的上游仓库（ggml-org/ggml），llama.cpp 里的 ggml/ 其实是它的一份镜像；仓库自带的 scripts/sync-ggml.sh 就负责把上游最新代码同步过来。于是引擎在自己的仓库里演进，各使用方（llama.cpp、whisper.cpp）再各自拉取，谁都不绑死谁。
源码：ggml/ 自带完整的 include/ 与 src/，对外的张量 / 计算图 / 后端接口是独立的一套，不依赖 src/llama-* 里的任何东西——依赖是严格单向的：src/llama-* 里用 #include "ggml.h" 调引擎，反过来 ggml 从不 include llama 的任何头文件。
好处：引擎可以独立演进（加新算子、新后端不必动 llama），也便于嵌入到任何想要本地张量计算的程序里；llama 只是它众多使用者中的一个。
一句话：common/ 是各个可执行程序共用的胶水，不是推理库本体；真正的推理逻辑住在 src/llama-*。
它管什么：命令行参数解析、把 C API 的采样接口包成更顺手的封装、日志、下载模型、聊天模板拼接……这些是"把库变成一个能用的程序"要反复写的活，抽到 common/ 让 tools/ 里每个程序都能复用。
它不管什么：加载权重、搭计算图、KV cache、真正的采样算法——这些都在 src/llama-* 里，对外只通过 include/llama.h 暴露。换句话说，删掉 common/，src/llama-* 推理库照样能编译、照样能用，只是你得自己手写一堆样板代码。
依赖方向：这条链是严格单向的——tools/ 用 common/，common/ 透过 include/llama.h 调 src/llama-*，src/llama-* 再往下压到 ggml，即 tools -> common -> llama.h -> src/llama-* -> ggml，越往右越底层、从不回头反向依赖。认准这个方向，遇到任何一个符号都知道"该去哪一层找"。
源码：参数解析看 common/arg.cpp，其余通用工具看 common/common.cpp。
一句话：顺着这张地图，给 llama.cpp 加一个新模型结构，通常只在三处落笔——一处登记、一处搭图、一处转换。
① 登记架构：在 src/llama-arch 里把新架构注册进来，并声明它用到的各类张量名字（有哪几种权重、各自叫什么）。这一步相当于在引擎的"花名册"上添个新成员。
② 搭前向图：在 src/llama-graph 提供的构件之上，把模型一次前向要做的事拼成计算图——嵌入、各层的注意力与前馈、归一化、输出。复用现成的 build_attn / build_norm 等积木，往往不必从零写算子。
③ 写转换器：在 convert_hf_to_gguf.py 里为这个模型加一段转换逻辑，把 HuggingFace 的权重与超参，按上面登记的张量名写进 .gguf。
这里只是预告：三步各自的细节，后面"模型加载""计算图"相关的课会专门展开；此刻只需记住——新增模型不是漫天改动，而是沿着登记 -> 搭图 -> 转换这条窄路走一遍。
为什么能这么省事？正是前面那套分层在兜底：算子、后端、KV cache、采样这些通用机制早已写好、且与具体模型无关，新模型只需描述"我的结构长什么样"，把现成积木重新搭一遍即可，无需重造引擎。这就是"边界清晰"在扩展性上的回报——加模型是沿既有接缝填空，而非动土重建。
common/common.hinclude/llama.hsrc/llama.cppggml/include/ggml.htools/llama-quantizegguf-py/ + convert_*.py（Python 转换脚本）src/llama-model-loadercommon/ 在整个项目里扮演什么角色？src/llama-*Open llama.cpp for the first time and a few hundred files across a dozen directories can feel overwhelming - where do you even start? In fact it is cleanly layered, every directory with one job. This lesson hands you a "campus map": learn what the top-level directories do, map them back onto the four layers from the previous lesson, then watch how a model travels from training to running - which directories it passes through and where the boundary falls. Once you have read the map you will not get lost, whether you want to run a tool or dig into the source.
Standing at the repo root, do not rush to open files; first scan the main directories by "what does this directory actually do". The table below is the map's legend:
| Directory | Role |
|---|---|
| ggml/ | The in-house tensor engine: tensors - compute graph - ops - backend scheduling; a standalone sub-project (ships its own include/ and src/) - the lowest and "hardest" layer of the whole project |
| ggml/src/ggml-cpu - ggml-cuda - ggml-metal - ggml-vulkan ... | The individual hardware backends that actually run the ops on CPU / GPU (plus hip / sycl / musa / opencl and a dozen more) |
| src/ | The llama inference library: assembles the engine into "session-level" inference - llama-model-loader - llama-graph - llama-kv-cache - llama-sampler - llama-vocab - llama-chat - llama-grammar - llama-quant ... |
| include/ | The public C API: llama.h (the only external contract); llama-cpp.h (a thin C++ wrapper + RAII) |
| common/ | Reusable helpers / glue: arg (argument parsing) - sampling (sampler wrapper) - chat - log - download - json-schema-to-grammar ...; for the programs, not the inference library itself |
| tools/ | The executable programs: llama-cli - llama-server - llama-quantize - llama-mtmd-cli (multimodal) - llama-perplexity - llama-bench ... |
| examples/ | Small example programs: simple demonstrates minimal inference in a couple hundred lines - the best "readable" entry point |
| gguf-py/ | The Python GGUF read/write library: the conversion scripts use it to write out a .gguf |
| convert_hf_to_gguf.py, etc. | Python conversion scripts (4 convert_*.py: 3 converters + 1 tokenizer-hash updater; the main one is HuggingFace -> GGUF) |
| models/ - tests/ - docs/ - grammars/ - cmake/ | Model data / tests / docs / GBNF examples / build system |
A simple rule of thumb: the lower you go, the "harder" it gets - ggml/ leans toward math and hardware, src/ toward model and session logic, tools/ and common/ toward "for people to use". The only thing ever exposed to the outside, start to finish, is the single public header include/llama.h.
The table lists many directories, but there are really only four protagonists, each locked onto one job and never crossing into another's. Get these four boundaries straight and the rest of that directory table becomes mere detail:
| This block | Does exactly one thing | Roughly lives in |
|---|---|---|
| Engine | Provide compute: define tensors, organize ops into a compute graph, schedule to the backends that actually run it | ggml/ |
| Inference lib | Assemble compute into a "session": load weights, build the graph, KV cache, sampling, tokenizing, chat templates | src/llama-* |
| Programs / glue | For people to use: the CLI, the HTTP server, the quantizer, plus the shared bits that turn a library into a program | tools/ - common/ |
| Model prep | Bring outside models in: turn HuggingFace weights into a .gguf | gguf-py/ - convert_*.py |
Why split them so hard? Because the four jobs change at completely different rhythms: backend ops chase new hardware and instruction sets; inference logic evolves with new model architectures; CLI flags and server APIs come and go with user needs; conversion scripts track upstream model formats. Weld them together and touching one means worrying you broke another. Split apart, each can evolve, be tested, and be reused on its own - add a new backend to ggml without touching a line of src/llama-*; add a new model architecture without reaching down to the low-level ops. That is the most concrete payoff of "clean boundaries".
One deliberate design is worth singling out: of the hundreds of internal files, the only thing exposed to the outside is the single header include/llama.h. Keeping the outward opening this narrow buys three things: first, change the internals freely - as long as the function signatures in llama.h stay put, however src/llama-* refactors internally, swaps data structures, or tweaks algorithms, no outside user is disturbed; second, a stable external contract, so consumers upgrade with confidence instead of chasing internal details; third, because what is exposed is a C ABI, almost any language can bind to it - Python, Go, Rust, Node and more all call the engine through this C interface, which is exactly how the many community language bindings are built. Narrow the outward opening to one small neck, and the internals earn the freedom to be refactored at will.
One aside: right next to llama.h sits llama-cpp.h, a thin convenience wrapper for C++ users - it uses RAII (smart pointers) to free llama_model / llama_context automatically, sparing you the manual free. It does not widen the exposed surface; it only wraps the same C interface more ergonomically, so "only one external contract" still holds.
Mapping those directories back onto the four-layer structure from the previous lesson makes it click:
Besides this main trunk there are two side-paths: (1) model prep - gguf-py/ plus convert_*.py (Python) turn a HuggingFace model into a .gguf file fed to the engine; (2) support - common/ (the glue that turns the library into programs) plus tests/ - docs/ - cmake/.
Why call them "side-paths"? Because an inference request only travels up and down the four-layer trunk - it never runs into these two. Model prep is done once, before you run (it emits the .gguf and steps aside), while support sits around the trunk like scaffolding: common/ spares programs from boilerplate, and tests/ - docs/ - cmake/ handle testing, docs, and the build. Tell apart "what the runtime passes through" from "what runs before or around it" and you won't mistake the scaffolding for a load-bearing wall when reading the source.
String the directories together and a model's "life" is really a straight pipeline: prepared in Python on the left, run in C++ on the right, handed over through one file in the middle. Walk it once and you will see which station each directory occupies along the whole chain:
The left half is Python (prepare), the right half is C++ (run), and the boundary between them is exactly that .gguf file in the middle.
That boundary matters: the conversion script runs once during prep, emits the .gguf, and then bows out; the runtime never touches Python and knows only this one file. So the same .gguf can feed llama-cli, llama-server, or examples/simple alike - they share the same loading and inference code.
Put this pipeline onto real commands and the common case is the three steps "convert -> quantize -> run". The block below is the minimal runnable skeleton - Python prepares on the left, C++ runs on the right, and the handover is still that .gguf in the middle:
# the full pipeline, from prep to run # 1) convert: HuggingFace model -> GGUF (Python; --outfile sets the name) python convert_hf_to_gguf.py ./my-model --outfile my-model.gguf # 16-bit float (auto by default) # 2) quantize (optional, to shrink) llama-quantize my-model.gguf my-model-Q4.gguf Q4_0 # 3) run (C++ side) llama-cli -m my-model-Q4.gguf -p "Hello"
The three steps land in three directories: step one runs in gguf-py/ + convert_*.py (Python) and spits out an FP16 .gguf; step two llama-quantize (from tools/) compresses it into a low-bit version like Q4_0, shrinking it to about a quarter of the size; step three llama-cli loads the quantized file and actually "emits text". Two things to remember: quantization is optional - if you don't care about size, just run the .gguf from step one; and steps two and three never touch Python, knowing only that one file - which is exactly what "the boundary is the .gguf" looks like once it lands on real commands.
What does convert_hf_to_gguf.py actually do in that step? It reads the config.json (hyper-parameters), the tokenizer, and the safetensors (weights) from the HuggingFace directory, renames tensors per the model architecture (transposing where needed), and writes it all out as one .gguf together with the metadata. In other words, this "loading dock" translates the outside world's motley models into the single format the engine recognizes - after which the C++ side need never care what they originally looked like.
While we are here, build some size intuition: the FP16 file from step one is about 14 GB for a 7B model; after Q4_0 in step two it drops to about 4 GB - the same command chain turns a model that "used to need a GPU" into a file that "fits in ordinary RAM". This echoes the "heavy to light" pipeline from the previous lesson, only this time landed on real commands.
Want to read the source but not sure where to start? Rather than chewing from the first file to the last, decide your goal first, then pick the matching entry point and drill down. Three common goals map to three entries:
llama-cli/llama-server running, then read the minimal call in examples/simplellama-model-loader -> llama-graph -> llama-kv-cache -> llama-samplerggml.c and the ggml-* backends - how tensors/ops/scheduling are implementedIf you want just one place to start, examples/simple is the top pick: in roughly two hundred lines it runs the whole main line - "load model -> tokenize -> decode loop -> sample -> output" - with no server or multimodal side-branches to distract you. Read it top to bottom once, then map each call back onto the four layers above - does this function belong to the inference lib or the engine, which interface in llama.h does it go through - and the whole map goes from "seen" to "walked".
Whichever path you take, remember there is only one public contract, include/llama.h: when you cannot tell which part owns some capability, go back to this header and see which functions it exposes to the outside. Find the entry first, then drill down layer by layer - far more efficient than flipping through files at random.
Four questions below - open them if you want to see the whole map clearly; skip them if you just want the main trunk.
Example: a .gguf is roughly four sections end to end, packed in order inside one file:
Section by section: (1) the header is a four-byte magic "GGUF" plus a version number (currently 3), followed by "how many tensors, how many metadata entries"; (2) the metadata KV is a long list of key-value pairs - hyper-parameters like <arch>.block_count (layer count) and <arch>.embedding_length (hidden size), the vocab like tokenizer.ggml.tokens, the chat template like tokenizer.chat_template, all sit here; (3) the tensor info records each weight tensor's name, shape, type, and its offset in the file; (4) only last comes the actual tensor data - block after block of weight values, located exactly by those offsets.
Why this design: packing everything into one file means loading just mmaps it into memory and reads on demand by offset - no unpacking or whole-file copy first (near-zero-copy for CPU inference; with a GPU backend the weights are then copied into VRAM). The hyper-parameters and vocab are built in, so once the engine reads the header it knows "what model this is and how to build the compute graph", with no config files and no Python. Putting metadata before the weights is deliberate too: the engine reads a small header first to understand the structure, then decides how to map the big blob of weight data that follows. A bonus: because the mapping is read-only, multiple processes can share the same weight memory, saving RAM and load time when you run several instances on one machine.
Source: reading/parsing lives in ggml/src/gguf.cpp (the gguf_kv / gguf_tensor_info structs); the metadata key-name constants are defined in gguf-py/gguf/constants.py; wiring that metadata onto a llama model and fetching hyper-parameters by key is src/llama-model-loader.cpp.
Alternatives: the earlier GGML / GGJT formats did the same job, but with scattered fields and poor version compatibility - now superseded by GGUF (the repo still keeps a convert_llama_ggml_to_gguf.py just to migrate the old format over).
In one line: because the same engine is reused by several projects - ggml does not serve only llama.cpp, so it is carved out as a sub-project that can stand on its own.
Example: same-author projects like whisper.cpp (speech-to-text) use ggml directly as their compute engine; they share the very same tensor, op, and backend code as llama.cpp, only the upper layer differs.
How they stay in sync: ggml has its own standalone upstream repo (ggml-org/ggml), and the ggml/ inside llama.cpp is really a mirror of it; the repo's own scripts/sync-ggml.sh is what syncs the latest upstream code over. So the engine evolves in its own repo and each consumer (llama.cpp, whisper.cpp) pulls it in separately - nobody is locked to anybody.
Source: ggml/ ships its own complete include/ and src/; its tensor / graph / backend interface is a self-contained set that depends on nothing in src/llama-* - the dependency is strictly one-way: src/llama-* does #include "ggml.h" to use the engine, while ggml never includes any llama header.
Benefit: the engine can evolve independently (new ops or backends without touching llama) and is easy to embed in any program that wants local tensor compute; llama is just one of its many users.
In one line: common/ is the glue shared by the executable programs - it is not the inference library itself; the real inference logic lives in src/llama-*.
What it handles: command-line argument parsing, wrapping the C API's sampler into something handier, logging, downloading models, assembling chat templates... the boilerplate every "turn the library into a usable program" needs, factored into common/ so each program in tools/ can reuse it.
What it does not: it does not load weights, build the compute graph, manage the KV cache, or implement the actual sampling algorithms - those all live in src/llama-* and are exposed only through include/llama.h. In other words, delete common/ and src/llama-* still compiles and still works; you would just hand-write a pile of boilerplate yourself.
Dependency direction: the chain is strictly one-way - tools/ use common/, common/ calls src/llama-* through include/llama.h, and src/llama-* presses down onto ggml, i.e. tools -> common -> llama.h -> src/llama-* -> ggml, lower the further right, never doubling back. Fix this direction in your head and any symbol tells you "which layer to look in".
Source: for argument parsing see common/arg.cpp; for the rest of the shared helpers see common/common.cpp.
In one line: following this map, adding a new model architecture to llama.cpp usually means edits in just three places - one to register, one to build the graph, one to convert.
(1) Register the architecture: in src/llama-arch, register the new architecture and declare the tensor names it uses (which weights, and what each is called). This step is like adding a new member to the engine's "roster".
(2) Build the forward graph: on top of the building blocks src/llama-graph provides, assemble what one forward pass does into a compute graph - embeddings, each layer's attention and feed-forward, normalization, output. Reusing ready-made bricks like build_attn / build_norm usually means you don't write ops from scratch.
(3) Write the converter: in convert_hf_to_gguf.py, add a conversion path for the model, writing the HuggingFace weights and hyper-parameters into a .gguf under the tensor names registered above.
This is only a preview: the details of each step are expanded in the later "model loading" and "compute graph" lessons; for now just remember - adding a model is not a sprawling change but a walk down the narrow path register -> build graph -> convert.
Why so cheap? Exactly because the layering underneath has your back: the generic machinery - ops, backends, KV cache, sampling - is already written and model-agnostic; a new model only describes "what my structure looks like" and re-assembles existing bricks, with no need to rebuild the engine. That is the payoff of "clean boundaries" for extensibility - adding a model is filling in along existing seams, not breaking ground to rebuild.
common/common.hinclude/llama.hsrc/llama.cppggml/include/ggml.htools/llama-quantizegguf-py/ + convert_*.py (Python conversion scripts)src/llama-model-loadercommon/ play in the project?src/llama-*