GGUF
Also known as: GGML successor
Self-contained binary format for quantized LLMs; the standard for llama.cpp / Ollama / LM Studio.
A binary file format for quantized LLM weights popularized by llama.cpp. Bundles weights, vocab, hyperparams, and metadata in one file optimized for CPU and Apple Silicon inference.
In practice
If you've used Ollama you've used GGUF. Production interviews probe its trade-offs vs safetensors and provider APIs.
Related topics
Questions that mention this term
Related terms
API LLM
An LLM accessed through a provider API: pay per token, get the frontier model, hand over ops.
Beam Search
Keep the K best partial sequences at each step; deterministic, breadth-first decoding.
FlashAttention
A memory-aware attention kernel that's 2-4x faster than vanilla, with identical math.
Greedy Decoding
At each step, pick the single highest-probability token. Fast and deterministic, but often loops.
Grouped-Query Attention (GQA)
Compromise between MHA and MQA: query heads share KV heads in groups, cutting KV cache by 4-8x.
Knowledge Distillation
Train a small student model to match a big teacher's outputs: cheap, fast inference with most of the quality.