common 工具层The common layer

Last lesson (L25) we met the C API in include/llama.h - it really can drive the whole machine, but using it is rather "verbose": filling one llama_batch means setting token/pos/seq_id field by field; building a sampler chain means adding it node by node with llama_sampler_chain_add; parsing the command line into params and pulling a model off the network - every tool writes all of this from scratch. Those repeated chores are exactly what the common/ layer packages away for you.

common/ is llama.cpp's built-in "shared glue layer": it wraps those scattered C calls into a set of handy C++ helpers, so bundled tools like llama-cli and llama-server all stand on its shoulders instead of each writing their own boilerplate. But one thing must be clear up front: common is not the public API - it carries no ABI-stability promise like llama.h does, it is just an internal convenience library for the project's own tools; third-party language bindings should write against llama.h directly rather than depend on common.

So what chores does this layer actually package? Roughly four kinds: parse the command line into one big config, initialize the model and context from that config in one shot, wrap the sampler chain and grammar constraint into one handy sampler, and download and cache the model file when you hand it only a repo name. This lesson walks those four things in turn, then spends a little time making clear "why common is not the public API", plus two small helpers used every day yet rarely mentioned on their own - logging and terminal coloring.

One-stop config: common_params

At common's core is one big config struct, common_params (see common/common.h). The model path, the prompt, how many tokens to generate, how large the context is, plus a whole nested block of sampling settings - nearly every "knob" is packed into this one struct. It also carries a set of sane defaults, so you only override the few things you actually care about and leave the rest to run as-is. So a tool no longer passes a fistful of scattered parameters around; it just revolves around this single common_params: fill it by reference (common_params &) while parsing the command line, then pull the whole thing out at init time - one object threaded end to end, sparing you layers of pass-through.

// one big struct holds every knob (simplified from common/common.h)
struct common_params {
    std::string  prompt;              // -p  the prompt
    int32_t      n_predict;           // -n  how many tokens to generate
    int32_t      n_ctx;               // -c  context window size
    common_params_sampling sampling;  // nested: samplers / temp / grammar
    common_params_model    model;     // model.path / model.hf_repo ...
    // ... dozens more fields
};

Note the two nested sub-structs: sampling (that is common_params_sampling, holding the sampler-chain order, temperature, grammar, and so on) and model (that is common_params_model, holding the local model.path and the HF model.hf_repo). Tucking related knobs into their own little structs keeps common_params from turning into one big stew, and lets modules like sampling and download each take just what they need.

With the config in hand, how does it become runnable objects? In two steps. First call common_init() once for global init (start the logging system, print build info) - this step is model-independent and runs once per program; the real work is done by common_init_from_params(params), which takes the filled common_params and produces, in one shot, a common_init_result holding the loaded model, the built context, and the configured sampler (taken via .model() / .context() / .sampler()). The two steps are split because global init must happen before any logging, or early errors could be swallowed silently.

The point of this step is that L25's long, easy-to-misfire sequence - llama_model_load_from_file to load, llama_init_from_model to build the context, then the sampler chain node by node - is folded entirely inside common_init_from_params. It also looks after errors and cleanup along the way: on a failed load it returns an empty result so you notice early, and on success the common_init_result "owns" those objects and frees them together on destruction, so you need not free each one by hand. A tool's main() is much cleaner for it: fill common_params, call init once, and the trio is ready, sparing you a long stretch of boilerplate. This echoes L17's cparams and L14's model loading - none of those low-level steps are gone, they are just tucked into this glue layer.

How the command line becomes config

Where do the fields in common_params come from? Mostly from the command line. common/arg.{h,cpp} describes each command-line option as a common_arg object: it records the option's names (a short -m and a long --model at once), a line of help text, and a callback that "writes the value into common_params". The help text of every common_arg is also gathered automatically into the table you see when you type --help, sparing a separate document that always drifts out of sync. At declaration you can further tune behavior with chainable builders - .set_examples() limits which tools it belongs to, .set_env() lets it read from an environment variable, .set_excludes() rules certain tools out.

// declare the "-m / --model" option (simplified from common/arg.cpp)
add_opt(common_arg(
    {"-m", "--model"}, "FNAME", "model path to load",
    [](common_params & params, const std::string & value) {
        params.model.path = value;          // the value goes into common_params
    }
).set_examples({LLAMA_EXAMPLE_COMMON}).set_env("LLAMA_ARG_MODEL"));

// parse the whole argv into params; the 4th arg picks which tool's option set
common_params_parse(argc, argv, params, LLAMA_EXAMPLE_CLI);

What actually "pours" the command line into the struct is common_params_parse(argc, argv, params, ex): it walks argv, finds the matching common_arg by name, and calls each callback to write into params. The 4th argument ex is an enum llama_example (like LLAMA_EXAMPLE_CLI, LLAMA_EXAMPLE_SERVER, LLAMA_EXAMPLE_COMMON) that decides which options this parse recognizes - one mechanism, so it can expose different parameter subsets to different tools: cli has cli's options, server has server's, with no clashes, while the ones marked COMMON are the shared basics everyone gets. When an environment variable and the command line both supply a value, the command line wins - so setting defaults via env in CI and overriding on the command line ad hoc feels natural.

Words alone are not concrete enough. Below we follow one minimal command line, -m model.gguf -p "Hi" -n 16 --temp 0.7, and watch it turn step by step into common_params fields, then get handed off to produce the trio:

The sampler wrapper: common_sampler

As for sampling, L21 covered how its underlying form is a llama_sampler chain, and L23 covered GBNF grammar constraints. These two were separate by nature: the chain picks words by probability, the grammar enforces "only legal words allowed". common wraps them into one object, common_sampler (see common/sampling.{h,cpp}): the sampler chain plus the grammar bundled together, exposing only one handle outward, so each tool need not fuss over details like "apply the grammar before or after sampling". common_sampler_init(model, params.sampling) builds the chain node by node, following the order listed in params.sampling.samplers (a list of common_sampler_type, like COMMON_SAMPLER_TYPE_TOP_K / TOP_P / TEMPERATURE).

Using it is tidier: common_sampler_sample(gsmpl, ctx, idx, grammar_first) does "take logits, apply the grammar mask, run the whole chain, pick a token" in one call; afterward common_sampler_accept(...) feeds the token back to update repetition penalties and grammar state. llama-cli and llama-server both use this layer, not the raw llama_sampler_* from L21. The diagram draws only a few nodes of the default chain (the real default also includes penalties, dry, min_p, and more), but the logic of the order is the same: sieve out unwanted candidates layer by layer, then draw one by distribution at the end.

// cli/server use this layer, not raw llama_sampler_* (simplified from common/sampling.h)
common_sampler * smpl = common_sampler_init(model, params.sampling);
// ... each step ...
llama_token id = common_sampler_sample(smpl, ctx, -1);   // logits->grammar->sample in one call
common_sampler_accept(smpl, id, /*is_generated=*/ true); // feedback: update penalties and grammar state

Note this is a loop you run every step: for each generated token you common_sampler_sample to pick one, then common_sampler_accept to feed it back, over and over until an end token or the planned length is reached. That accept step matters - the repetition penalty relies on it to remember "which words already appeared", and the grammar state relies on it to advance to the next legal position. The grammar_first argument controls whether the "grammar mask" applies before or after the other samplers: the default is fine most of the time, and you only adjust it when the grammar is strict yet you still want temperature and friends to act first.

Download and cache

There is one more repeated chore: getting the model file itself. common/download.{h,cpp} lets you write -hf user/repo:tag to pull a model straight from Hugging Face (a public model-hosting site), sparing the two-step dance of "download by hand, then point -m at it". The first step is common_download_split_repo_tag("repo:tag"), which splits the repo:tag form into a repo name and a tag - the tag often picks a particular quantization (like Q4_K_M); then it downloads the file from HF using that info and builds a local cache. For a big model split into several shards (multiple .gguf parts) it fetches all the parts together too.

The key step on this pipeline is to check the cache first: before any real network download, it looks at whether the file already exists locally and matches the remote version, and if so it skips the download entirely. So the "HF download" box above really only runs on the very first run; later runs are short-circuited by the cache.

The cache is the key: a model downloads only the first time it is used, and once it lands in the local cache directory, every later start hits it directly and opens instantly. The cache defaults to ~/.cache/llama.cpp/ (overridable via the LLAMA_CACHE environment variable); common_list_cached_models() lists the already-cached models; and the download itself is driven by common_download_callback (with three callbacks on_start / on_update / on_done, paired with common_download_progress) that powers the progress bar. This cache is also shared across tools: when cli, server, and bench point at the same repo, they all hit the same local copy rather than each downloading their own.

Deep dive: the boundary and two small helpers

Finally, two folds for two often-asked points: why we keep stressing that "common is not the public API", and those unglamorous little utilities used every day (logging and the terminal). One is about the "boundary" - which layer you should actually depend on - and the other about the "feel" - who quietly helps you while debugging - and together they round off the lesson.