🦙 llama.cpp 图解教程llama.cpp Visual Guide 共 40 课 · 9 个部分40 lessons · 9 parts
从零开始 · 面向完全新手From scratch · for complete beginners

用图解理解整个 llama.cpp 项目Understand the whole llama.cpp project, visually

这套教程带你层层深入:先建立宏观全景,再学会使用, 然后深入 ggml 引擎llama 推理内部,最后直抵底层内核。每课配真实源码对应、图解与设计亮点。 A layered tour: build the big picture first, learn to use it, then dive into the ggml engine and llama inference internals, down to the low-level kernels. Every lesson maps to real source, with diagrams and design insights.

宏观理解Big picture 细节 / 源码Details / source 生活类比Analogy 关键要点Key points

📌 对照 llama.cpp 仓库真实源码核实 · 源码引用以“文件 + 符号名”为主(行号随上游更新而变)📌 Verified against the real llama.cpp source; references cite file + symbol (line numbers drift upstream)

第一部分 · 宏观全景Part 1 · The Big Picture
01llama.cpp 是什么What is llama.cpp解决什么问题 · 零依赖哲学What problem it solves; zero-dep philosophy 02项目全景地图The project mapggml / src·llama / common / tools / 转换脚本ggml / src·llama / common / tools / converters 03一次推理的生命周期Lifecycle of one inferenceprompt -> 分词 -> 计算图 -> logits -> 采样 -> tokenprompt -> tokenize -> graph -> logits -> sample -> token
第二部分 · 前置基础Part 2 · Foundations
04大模型推理基础LLM inference fundamentalsdecoder-only · 因果掩码 · 自回归 · KV cache 为何精确decoder-only; causal mask; autoregression; why the KV cache is exact 05张量是什么What is a tensorshape/stride/行优先 · ggml_tensor 字段 · view 零拷贝shape/stride/row-major; ggml_tensor fields; zero-copy views 06量化入门Quantization, intuitively为什么量化 · 块量化 · Q4_0/Q8_0/K-quant 一览why quantize; block quantization; Q4_0/Q8_0/K-quant tour 07构建系统与后端Build system & backendsCMake 两步走 · 后端选项 · 产物在 build/bin · -ngltwo-step CMake; backend options; build/bin outputs; -ngl
第三部分 · ggml 引擎Part 3 · The ggml engine
08ggml 核心对象ggml core objectsggml_context · 内存池 arena · no-malloc bump 分配ggml_context; the memory-pool arena; no-malloc bump allocation 09计算图:惰性构建The compute graph: lazy build先建图后执行 · op/src 反向指针 · nodes vs leafsbuild-then-run; op/src back-pointers; nodes vs leafs 10图的执行与调度Graph execution & schedulingbackend 执行 · sched 多后端调度 · ggml-alloc 内存复用backend compute; multi-backend sched; ggml-alloc memory reuse 11核心算子Core operatorsmul_mat 形状推导 · rms_norm/rope/soft_max_ext · CPU 计算mul_mat shapes; rms_norm/rope/soft_max_ext; CPU compute 12量化格式细节Quantization formats in detailblock 字节布局 · super-block K-quant · 解量化 · type_traitsblock byte layout; super-block K-quant; dequant; type_traits 13GGUF 文件格式The GGUF file formatmagic/version · metadata KV · tensor info · 对齐 · mmapmagic/version; metadata KV; tensor info; alignment; mmap
第四部分 · llama 推理内部Part 4 · Inside llama inference
14模型加载Model loadingllama_model_loader · GGUF metadata/张量清单 · mmap · 分片llama_model_loader; GGUF metadata/tensor map; mmap; splits 15架构与超参Architecture & hyperparametersllm_arch · llama_hparams · LLM_TENSOR_NAMES 命名约定llm_arch; llama_hparams; LLM_TENSOR_NAMES naming 16构建计算图Building the compute graphbuild_graph 派发 · build_attn/build_ffn 积木 · src/models · ggml_cgraphbuild_graph dispatch; build_attn/build_ffn blocks; src/models; ggml_cgraph 17上下文与会话Context & sessionllama_context · cparams · memory/sched/logits · llama_decodellama_context; cparams; memory/sched/logits; llama_decode 18批处理Batchingllama_batch · pos/seq_id/logits 标志 · llama_batch_allocr -> ubatchllama_batch; pos/seq_id/logits flags; llama_batch_allocr -> ubatch 19KV cacheThe KV cachellama_kv_cache · cell(pos/seq_id) · 上下文移位 · 多序列 · 变体llama_kv_cache; cell(pos/seq_id); context shift; multi-seq; variants 20词表Vocabularyllama_vocab · tokenize/detokenize · SPM/BPE/WPM · 字节回退llama_vocab; tokenize/detokenize; SPM/BPE/WPM; byte fallback 21采样Samplingllama_sampler · 采样链 · top-k/top-p/温度 · greedy vs distllama_sampler; sampler chain; top-k/top-p/temp; greedy vs dist 22对话模板Chat templatesllm_chat_template · 内建 vs Jinja · ChatML/Llama/Gemmallm_chat_template; built-in vs Jinja; ChatML/Llama/Gemma 23语法约束Grammar (GBNF)GBNF · llama_grammar · 掩码采样(-inf) · root 规则GBNF; llama_grammar; mask sampling(-inf); root rule 24LoRA 适配器LoRA adaptersLoRA 低秩增量 scale·B·A · llama_set_adapters_lora · 控制向量LoRA low-rank delta scale*B*A; llama_set_adapters_lora; control vectors
第五部分 · 公共 API 与工具Part 5 · Public API & tools
25C API 总览The C APIinclude/llama.h:句柄、调用序列、C++ RAII 包装include/llama.h: handles, call sequence, the C++ RAII wrappers 26common 工具层The common layercommon:参数解析、采样包装、下载缓存——各 tool 的共享胶水common: arg parsing, sampler wrapper, downloads - the shared glue for every tool 27llama-clillama-cli命令行/交互外壳:跑在共享引擎上的生成主循环the CLI/interactive shell: a generation loop on the shared engine 28llama-serverllama-serverHTTP + OpenAI 兼容 + slot 连续批处理:把引擎变成服务HTTP + OpenAI-compatible + slot continuous batching: the engine as a service 29quantize 工具The quantize toolllama-quantize + imatrix:把模型压小,并用重要性矩阵保质量llama-quantize + imatrix: shrink the model, keep quality via an importance matrix 30评测与基准Evaluation & benchmarksperplexity 量质量、llama-bench 量速度:选模型/量化档的两把尺子perplexity for quality, llama-bench for speed: the two rulers for choosing models/quants
第六部分 · 底层内核Part 6 · Low-level kernels
31CPU 后端The CPU backendggml-cpu:从标量到 SIMD、量化点积、多线程ggml-cpu: scalar to SIMD, quantized dot product, multithreading 32CUDA 后端The CUDA backendggml-cuda:线程网格、分块矩阵乘、显存层级ggml-cuda: thread grid, tiled matmul, the memory hierarchy 33后端调度Backends & dispatchggml-backend:抽象层、注册与动态加载、调度、其它后端一览ggml-backend: the abstraction, registry & dynamic load, scheduling, the backend zoo
第七部分 · 进阶专题Part 7 · Advanced topics
34投机解码Speculative decodingdraft model / n-gram 提候选 · target 并行验证 · 接受率draft model / n-gram propose, target verifies in parallel, acceptance rate 35MoE 专家混合Mixture of expertsrouter 门控 -> top-k 选专家 -> ggml_mul_mat_id 稀疏算 -> 加权合并router gating -> top-k experts -> ggml_mul_mat_id sparse compute -> weighted combine 36多模态Multimodalmtmd 管线: 图像 -> clip(ViT) -> projector(mmproj) -> embedding -> 与文本交织mtmd pipeline: image -> clip(ViT) -> projector(mmproj) -> embeddings -> interleave with text 37状态空间模型State-space modelsMamba/RWKV: 递推状态替代 KV cache · ggml_ssm_conv/scan · O(1) 显存Mamba/RWKV: recurrent state instead of KV cache; ggml_ssm_conv/scan; O(1) memory
第八部分 · 实战与贡献Part 8 · Practice & contributing
38从 HF 转换模型Converting HF modelsconvert_hf_to_gguf.py 薄 CLI + conversion 包 + gguf-py 写盘字节布局convert_hf_to_gguf.py thin CLI + conversion package + gguf-py byte layout 39编译·调试·贡献Build & contributeCMake 多后端构建 · ctest/test-backend-ops · CONTRIBUTING 的 AI 政策 · clang-formatCMake multi-backend build, ctest/test-backend-ops, CONTRIBUTING AI policy, clang-format
第九部分 · 速查Part 9 · Quick reference
40术语表·索引Glossary & index全书 40 课术语一句话查 + 概念依赖图 + 点链接跳到对应课40-lesson glossary one-liners + concept dependency map + jump links
没有匹配的课程,换个关键词试试。No matching lessons, try another keyword.