前面几课都在讲"是什么"；这一课讲"怎么把它编出来、怎么挑硬件后端、编完有哪些产物"。读完这一课，你就能自己从源码 build 一个带 GPU 加速的 llama.cpp， 也能看懂为什么同一份代码能在 CPU、NVIDIA、苹果、AMD 各种硬件上跑。这一课也是第二部分的收尾。
课 03 说过，一次推理会被描述成一张计算图（一堆算子：矩阵乘、softmax、rope……）。但同样一个"矩阵乘"，在 CPU 上要用 SIMD 指令写、在 NVIDIA 上要用 CUDA 写、 在苹果上要用 Metal 写——实现天差地别。如果让上层的推理逻辑去操心这些，代码会乱成一团。
ggml 的解法是定义一个统一的后端（backend）接口（ggml/include/ggml-backend.h）：上层只管"我要算这张图"，至于"在哪种硬件上、用什么指令算"， 交给具体的后端实现（ggml-cpu、ggml-cuda、ggml-metal……）。这正是课 01 反复强调的"把'算什么'和'在哪算'解耦"。
这个分层的好处是：加一种新硬件，只需新写一个后端，上层推理代码一行不用动；而编译时选哪些后端，就决定了你这份二进制"认得"哪些硬件。 所以"构建"和"后端"是同一件事的两面——构建系统的主要工作，就是按你的选择，把对应的后端代码编进来。
再补一点机制：每个后端在编进来时会向 ggml 注册自己，声明"我能算哪些算子、管哪块内存"。运行时，调度器（ggml-backend 里的 sched） 拿到计算图后，会把每个算子分派给合适的后端去算，并在 CPU 和 GPU 内存之间按需搬运数据。万一某个算子在 GPU 后端里还没实现，调度器通常能 自动回退（fallback）到 CPU 把这一步算完——所以即使某个新算子 GPU 还没支持，整张图也不至于跑不起来，只是那一步慢一点。
还要补一句："后端"不全是 GPU。CPU 本身就是一个后端；BLAS 是给 CPU 上大矩阵乘加速的库后端；苹果的 Accelerate 框架也能接进来。 所以"后端"更准确的说法是"一种把算子真正算出来的实现途径"，GPU 只是其中最受关注的一类。理解这一点，你看 ggml/src 下那一长串 ggml-cpu、ggml-cuda、ggml-blas、ggml-metal…… 目录时就不会困惑了。
llama.cpp 用 CMake 作为构建系统。从源码编译，标准流程就两步：先配置（configure），再构建（build）。
当然，最最开始还有一步别漏了：先把源码克隆下来——git clone 仓库地址、进到目录，再走下面那两步 CMake 就行。 想要某个稳定版本可以 checkout 对应的发布 tag；想跟最新进展就用默认的主分支。整个"克隆 -> 配置 -> 构建 -> 运行"四步， 就是绝大多数人上手 llama.cpp 的完整路径。
# 仅 CPU (默认) cmake -B build cmake --build build --config Release -j # 带 NVIDIA CUDA cmake -B build -DGGML_CUDA=ON cmake --build build --config Release -j
第一步 cmake -B build 是配置：CMake 会探测你的系统（有没有 CUDA 工具链、什么编译器、什么 CPU 指令集），根据你给的 -D 选项决定要编哪些后端，然后在 build/ 目录里生成真正的构建文件。第二步 cmake --build build 才是 真正编译，把源码变成库和可执行文件。-j 让它多核并行编、快很多；--config Release 表示编优化过的发布版 （而非带调试信息、慢得多的 Debug 版）。整个过程对照前面那张流程图看，就很清楚了。
顺便说：大多数人其实不用自己编。llama.cpp 官方在 GitHub Releases 提供了各平台的预编译包，下载解压即用；很多上层项目 （如 Ollama、LM Studio）也都内置了它。那什么时候才需要自己从源码编？——当你要打开某个预编译包没带的后端（比如针对你这张特定显卡的 CUDA）、 要用最新的开发版功能、或者要把库嵌进自己的程序时。这一课讲的就是这后一种"需要动手"的场景。
真正动手前，强烈建议先扫一眼仓库里的 docs/build.md：它把每个平台、每种后端的具体编译命令和注意事项都列全了 （包括 Windows、各家 GPU 的细节）。这一课给你的是"地图和直觉"，而 docs/build.md 是"逐条的操作手册"，两者配合着看，第一次编译就能少走很多弯路，遇到平台特有的问题也多半能在那里找到答案。
要不要某个 GPU 后端，就靠配置时的一个 -D 开关。常用的几个列在下面：
| CMake 选项 | 启用的硬件 / 功能 |
|---|---|
| GGML_CPU（默认 ON） | CPU 后端（自动用 AVX / NEON 等 SIMD） |
| GGML_CUDA | NVIDIA GPU |
| GGML_HIP | AMD GPU（ROCm） |
| GGML_METAL | Apple GPU（macOS 上常默认 ON） |
| GGML_VULKAN | 跨厂商 GPU（含部分集显） |
| GGML_SYCL | Intel GPU |
| GGML_BLAS | 用 BLAS 库加速大矩阵乘 |
option(GGML_CPU "ggml: enable CPU backend" ON)
option(GGML_CUDA "ggml: use CUDA" OFF)
option(GGML_METAL "ggml: use Metal" ...)
option(GGML_VULKAN "ggml: use Vulkan" OFF)
# ... HIP / SYCL / OPENCL / BLAS, 都在 ggml/CMakeLists.txt
这些开关都定义在 ggml/CMakeLists.txt 里。注意 CPU 后端默认就是开的（GGML_CPU=ON）， 所以你什么都不加，也能得到一个纯 CPU 能跑的 llama.cpp；GPU 后端则默认关闭，要哪个就显式 -D...=ON 打开。苹果设备上 Metal 通常默认开。 你也可以同时开多个后端，运行时再决定用哪个。
自己编最常见的坑，几乎都出在 GPU 工具链上。比如开了 -DGGML_CUDA=ON 却没装好 CUDA Toolkit、或者 CUDA 版本和显卡驱动对不上， 配置阶段就会报错——这其实是好事，CMake 在"配置"时就帮你把环境问题暴露出来了，省得编到一半才失败。遇到报错别慌，先看它提示缺什么： 缺 nvcc 就装 CUDA Toolkit、缺某个库就按提示装，多数问题照着错误信息走一遍就能解决。
编译完成后，所有产物都落在 build/bin 目录里，分两类：
一类是库：libggml（张量引擎）和 libllama（推理库），它们是"引擎本体"，可以被别的程序链接调用—— 课 01 说的"可嵌入"，靠的就是它们。另一类是可执行程序，就是你平时直接用的命令：llama-cli（命令行对话）、 llama-server（起一个 HTTP 服务）、llama-quantize（课 06 用过的量化工具）、llama-bench（测速）、 llama-perplexity（测质量）等等。
这些可执行程序里，最常打交道的是两个：llama-cli 适合在命令行里快速试一把或写脚本；llama-server 则会起一个常驻的 HTTP 服务，对外提供和 OpenAI 兼容的接口，适合给前端、应用或其它服务调用——你常用的各种本地大模型 App，背后往往就是它。 两者用的是同一套 libllama，只是把"入口"包装成了不同形态。
除了主角库和那几个常用程序，build/bin 里其实还会有一堆小工具和示例：llama-gguf（查看 / 操作 GGUF 文件）、 llama-tokenize（单独试分词）、各种 test-* 测试程序，以及 examples/ 下编出来的演示。平时用不到不用管， 但当你想深入某个细节（比如"这个模型到底被分成哪些张量"）时，往往能在这里找到一个正好趁手的小工具。
# 跑起来: -ngl 把多少层卸载到 GPU
./build/bin/llama-cli -m model.gguf -p "你好" -ngl 99
怎么确认 GPU 后端真的编进去、也真的用上了？最简单的办法是看启动日志。跑 llama-cli 时，它会打印检测到的设备和每层的分配情况—— 如果你看到类似 "offloaded 33/33 layers to GPU" 的字样，就说明层确实卸载到 GPU 上了。要是发现还在纯 CPU 跑，多半是 -ngl 没加、 或者那个后端根本没编进去，回到配置那一步检查 -D 选项即可。
把这一课和课 06 连起来看：你选的量化格式和你编的后端是要配合的。每个后端都为常见量化格式（如 Q4_K、Q8_0）写了专门的 解量化 + 矩阵乘内核，能直接吃量化权重、在算的时候顺手解量化。所以"选什么量化"和"用什么后端"共同决定了你的实际速度—— 这也是为什么第六部分会专门去看这些内核到底怎么写。
关于"可移植性"再多说一句，这里有个容易搞反的点。从源码自己编时，ggml 默认会打开 GGML_NATIVE（针对你这台机器的指令集做优化）， 所以默认编出来的二进制是"本机特化"的、跑得快，但不保证能拷到别的机器上跑（换台 CPU 可能直接"非法指令"崩掉）。反倒是官方预编译包 为了"谁都能用"，特意关掉 native、改用更通用且运行时自适应的指令，所以它们才是可移植的那一档。这就是"预编译求通用、自编可求性能"的取舍—— 想兼顾可移植，自己编时把 GGML_NATIVE 关掉即可。
最后给个动手的起点：想看"怎么用 libllama 写自己的程序"，仓库里的 examples/simple 是最好的入口——约两百行 C++ 就走完了加载模型、 分词、跑解码循环、输出文字的全过程，正好是课 03 那条主线的可运行版本。把它读懂、改一改，你就算真正上手 llama.cpp 的 API 了。
那为什么不干脆把所有后端都编进一个二进制、运行时谁有用谁？因为代价很大：每个 GPU 后端都拖着一大坨依赖（CUDA 要 CUDA 运行库、 Vulkan 要 Vulkan SDK……），全编进来体积暴涨、还要求目标机器装齐这些库——这恰恰违背了 llama.cpp"零依赖、轻量"的初心。所以它选择让你 按需挑选：纯 CPU 版可以小到拷哪都能跑，要加速时再单独编一个带某后端的版本。这正是本课末尾思考题的答案方向。
下面三个问题，想深究的同学点开看；只想抓主线的可以先跳过。
CPU 后端默认就开，你基本不用管它。它会自动探测并用上 CPU 的 SIMD 指令（x86 的 AVX、ARM 的 NEON 等）来加速矩阵运算——这是现代 CPU 上几乎免费的并行算力。
你还可以选装 GGML_BLAS，用成熟的 BLAS 数学库进一步加速大矩阵乘（对 prefill 阶段帮助明显）。追求极致的人会用 -march=native 让编译器针对你这台机器的指令集优化，但这样编出来的二进制就不能拷到别的机器用了——这又是一处"性能 vs 可移植"的老权衡。
因为 llama.cpp 要支持的平台和硬件太多了：Linux / macOS / Windows，CPU / CUDA / Metal / Vulkan / ROCm / SYCL……手写 Makefile 根本管不过来。
CMake 的价值在于跨平台和自动探测：它能找到你系统里装的 CUDA、判断编译器支持哪些指令、再生成对应平台的构建文件 （Linux 上是 Make 或 Ninja、Windows 上是 Visual Studio 工程）。项目早期其实有手写的 Makefile，但随着后端越来越多，现在已经统一以 CMake 为主。
编译时可以把多个后端都编进来；运行时，ggml 有一个后端"注册表"，会枚举出当前机器上实际可用的设备（比如检测到一张 NVIDIA 卡）。 -ngl 决定多少层放 GPU。
如果有多张 GPU，还能按层或按张量把模型切分到几张卡上一起算（用 --split-mode 等参数）。 所以"装哪些后端"是编译期的事，"具体用哪个、用几张卡"是运行期的事，两者分开，灵活又清晰。
到这里，第二部分的四块基础就拼齐了：课 04 讲清了模型在算什么（decoder-only、注意力、KV cache），课 05 讲清了数据怎么表示 （张量、shape / stride），课 06 讲清了权重怎么压（量化），这一课讲清了引擎怎么编、怎么挑硬件（构建与后端）。有了这四块垫底， 第三部分我们就能放心地钻进 ggml 引擎内部，去看计算图、内存池、算子这些"机器零件"到底是怎么转起来的了。可以说，第二部分是"地基"，第三部分才开始盖"主楼"。
The last few lessons were about "what things are"; this one is about "how to build it, how to pick a hardware backend, and what the outputs are". After this you can build a GPU-accelerated llama.cpp from source yourself, and you will see why one codebase runs on CPU, NVIDIA, Apple, and AMD hardware alike. This lesson also closes Part 2.
As lesson 03 noted, an inference is described as a compute graph (a pile of operators: matmul, softmax, rope...). But the same "matmul" is written with SIMD on CPU, with CUDA on NVIDIA, with Metal on Apple - wildly different implementations. If the upper inference logic had to worry about all this, the code would be a mess.
ggml's answer is a uniform backend interface (ggml/include/ggml-backend.h): the upper layer only says "compute this graph", while "on which hardware, with which instructions" is left to a concrete backend (ggml-cpu, ggml-cuda, ggml-metal...). This is exactly the "decouple 'what to compute' from 'where to compute'" that lesson 01 kept stressing.
The benefit of this layering: adding a new hardware needs only a new backend; not one line of upper inference code changes; and which backends you pick at build time decides which hardware your binary "knows". So "build" and "backend" are two sides of the same coin - the build system's main job is to compile in the backends you chose.
One more mechanism: each backend, when compiled in, registers itself with ggml, declaring "which ops I can compute, which memory I manage". At runtime the scheduler (the sched in ggml-backend) takes the graph and dispatches each op to a suitable backend, moving data between CPU and GPU memory as needed. If some op is not yet implemented in the GPU backend, the scheduler can usually fall back to CPU for that step - so even an unsupported new op will not break the whole graph, it just runs that step a bit slower.
One more note: "backend" is not only GPUs. The CPU is itself a backend; BLAS is a library backend that speeds big matmuls on CPU; Apple's Accelerate framework can plug in too. So "backend" is more precisely "a way to actually carry out the ops", with GPUs being the most-discussed kind. Grasp this and the long list of ggml-cpu, ggml-cuda, ggml-blas, ggml-metal... directories under ggml/src will not confuse you.
llama.cpp uses CMake as its build system. From source the standard flow is two steps: first configure, then build.
Of course, there is one step before all this: clone the source - git clone the repo, cd in, then run the two CMake steps below. For a stable version, checkout the matching release tag; to track the latest, use the default main branch. The whole "clone -> configure -> build -> run" four-step is how most people get started with llama.cpp.
# CPU only (default) cmake -B build cmake --build build --config Release -j # with NVIDIA CUDA cmake -B build -DGGML_CUDA=ON cmake --build build --config Release -j
Step one cmake -B build is configure: CMake probes your system (is there a CUDA toolchain, which compiler, which CPU instruction set), decides which backends to build from your -D options, and generates the real build files in build/. Step two cmake --build build is the actual compile, turning sources into libraries and executables. -j builds in parallel across cores (much faster); --config Release means an optimized release build (not the much slower, debug-info Debug build). Read it against the flow diagram above and it is clear.
By the way: most people do not build at all. The llama.cpp project ships prebuilt packages per platform on GitHub Releases - download, unzip, run; many higher-level projects (Ollama, LM Studio) bundle it too. So when do you build from source? When you need a backend the prebuilt package lacks (e.g. CUDA for your specific card), the latest dev-branch features, or to embed the library into your own program. This lesson is about that hands-on case.
Before getting hands-on, skim the repo's docs/build.md: it lists the exact build commands and caveats for every platform and backend (including Windows and each GPU's details). This lesson gives you "the map and the intuition", while docs/build.md is "the step-by-step manual"; reading both together saves a lot of first-build detours, and most platform-specific snags have an answer there.
Whether you want a given GPU backend comes down to one -D switch at configure time. The common ones:
| CMake option | hardware / feature enabled |
|---|---|
| GGML_CPU (default ON) | CPU backend (auto-uses AVX / NEON SIMD) |
| GGML_CUDA | NVIDIA GPU |
| GGML_HIP | AMD GPU (ROCm) |
| GGML_METAL | Apple GPU (often default ON on macOS) |
| GGML_VULKAN | cross-vendor GPU (incl. some integrated) |
| GGML_SYCL | Intel GPU |
| GGML_BLAS | use a BLAS library to speed big matmuls |
option(GGML_CPU "ggml: enable CPU backend" ON)
option(GGML_CUDA "ggml: use CUDA" OFF)
option(GGML_METAL "ggml: use Metal" ...)
option(GGML_VULKAN "ggml: use Vulkan" OFF)
# ... HIP / SYCL / OPENCL / BLAS, all in ggml/CMakeLists.txt
These switches all live in ggml/CMakeLists.txt. Note the CPU backend is on by default (GGML_CPU=ON), so with nothing extra you still get a CPU-runnable llama.cpp; GPU backends default off, so turn on whichever you want with an explicit -D...=ON. On Apple devices Metal is usually on by default. You can also enable several backends at once and decide which to use at runtime.
The most common pitfall when building yourself is almost always the GPU toolchain. Turning on -DGGML_CUDA=ON without a proper CUDA Toolkit, or a CUDA version mismatched with your driver, errors out at configure - which is actually good: CMake surfaces the environment problem at "configure" time, instead of failing halfway through compiling. Don't panic at an error; read what it says is missing: no nvcc means install the CUDA Toolkit, a missing library means install it as prompted - most issues resolve by following the error message.
After compiling, all outputs land in build/bin, in two kinds:
One kind is libraries: libggml (the tensor engine) and libllama (the inference library) - the "engine itself", which other programs can link against; lesson 01's "embeddable" rests on these. The other kind is executables, the commands you use directly: llama-cli (command-line chat), llama-server (starts an HTTP service), llama-quantize (the quantizer from lesson 06), llama-bench (speed), llama-perplexity (quality), and more.
Of these executables, two you will use most: llama-cli suits a quick command-line try or scripting; llama-server starts a long-running HTTP service with an OpenAI-compatible API, for front-ends, apps, or other services to call - the local-LLM apps you use are often it under the hood. Both use the same libllama, just wrapping the "entry point" in different forms.
Besides the star libraries and those common programs, build/bin also holds a pile of small tools and demos: llama-gguf (inspect / manipulate GGUF files), llama-tokenize (try tokenization alone), various test-* programs, and the demos built from examples/. You can ignore them day to day, but when you want to dig into a detail (e.g. "which tensors is this model split into") there is often a handy little tool right here.
# run it: -ngl offloads how many layers to the GPU
./build/bin/llama-cli -m model.gguf -p "Hello" -ngl 99
How do you confirm the GPU backend was really compiled in and is really being used? The simplest way is the startup log. When running llama-cli it prints the detected devices and per-layer placement - if you see something like "offloaded 33/33 layers to GPU", layers really went to the GPU. If it is still CPU-only, likely -ngl was omitted or that backend was not compiled in; go back to configure and check the -D options.
Tying this lesson to lesson 06: the quantization format you pick and the backend you build must work together. Each backend has dedicated dequant + matmul kernels for common quant formats (Q4_K, Q8_0), consuming quantized weights directly and dequantizing on the fly. So "which quantization" and "which backend" jointly decide your real-world speed - which is why Part 6 goes to look at how those kernels are actually written.
One more word on "portability", with an easy-to-get-backwards point. When you build from source, ggml turns on GGML_NATIVE by default (optimizing for your machine's instruction set), so the default binary is machine-tuned and fast, but not guaranteed to run on other machines (a different CPU may crash with "illegal instruction"). It is the official prebuilt packages that, to be "usable by everyone", turn native off and use more generic, runtime-adaptive instructions - so those are the portable ones. That is the trade: prebuilt for portability, self-build for performance - and if you want both, just turn GGML_NATIVE off when building.
A hands-on starting point: to see "how to write your own program with libllama", the repo's examples/simple is the best entry - around a couple hundred lines of C++ walk the whole path of loading a model, tokenizing, running the decode loop, and printing text, a runnable version of lesson 03's main line. Read it, tweak it, and you have truly started using the llama.cpp API.
So why not just compile all backends into one binary and let runtime pick? Because the cost is high: each GPU backend drags a big pile of dependencies (CUDA needs the CUDA runtime, Vulkan the Vulkan SDK...), so compiling them all balloons the size and demands the target machine have all those libraries - which contradicts llama.cpp's "zero-dependency, lightweight" ethos. So it lets you pick what you need: a CPU-only build can be tiny and copy-anywhere, and you build a backend-specific version separately when you want acceleration. That is the direction of this lesson's closing question.
Three questions below; open them if you want depth, skip them if you only want the main line.
The CPU backend is on by default; you mostly need not touch it. It auto-detects and uses your CPU's SIMD instructions (AVX on x86, NEON on ARM) to speed up matrix math - nearly free parallel compute on modern CPUs.
You can also opt into GGML_BLAS to further speed big matmuls with a mature BLAS library (noticeably helps prefill). The extreme route is -march=native, letting the compiler optimize for your exact instruction set - but the resulting binary cannot be copied to another machine. Another classic "performance vs portability" trade-off.
Because llama.cpp must support too many platforms and hardware: Linux / macOS / Windows, CPU / CUDA / Metal / Vulkan / ROCm / SYCL... a hand-written Makefile simply cannot keep up.
CMake's value is cross-platform and auto-detection: it finds your installed CUDA, checks which instructions the compiler supports, and generates the right build files per platform (Make or Ninja on Linux, a Visual Studio project on Windows). The project did have a hand-written Makefile early on, but as backends multiplied it has standardized on CMake.
You can compile several backends in; at runtime, ggml has a backend "registry" that enumerates the devices actually available on the machine (e.g. it detects an NVIDIA card). -ngl decides how many layers go on the GPU.
With multiple GPUs, you can split the model across cards by layer or by tensor (via parameters like --split-mode). So "which backends to compile" is a build-time matter, "which to use and across how many cards" a runtime matter - kept separate, flexible and clear.
And with that, Part 2's four foundations are complete: lesson 04 clarified what the model computes (decoder-only, attention, KV cache), lesson 05 how data is represented (tensors, shape/stride), lesson 06 how weights are compressed (quantization), and this one how the engine is built and how hardware is chosen (build & backends). With these four underneath, Part 3 can confidently dive into the ggml engine to see how the compute graph, memory pool, and operators - the "machine parts" - actually turn. Part 2 is the "foundation"; Part 3 starts building the "main floors".