25 side-by-side comparisons of the concepts interviewers ask about most. Understand the tradeoffs, know when to use what, and never confuse the two again.
Prompt engineering shapes a single turn; reach for agents only when a task genuinely needs tools, state, and multiple steps. Good prompts are a prerequisite either way.
API providers give you the best models with zero ops. Self-hosting gives you data control, custom fine-tunes, and predictable cost past a break-even volume, usually 10 to 50 million tokens per day.
Bi-encoders precompute embeddings and search fast; cross-encoders score query and document together for higher quality. Production usually runs bi-encoders for retrieval and cross-encoders as a reranker on top.
BPE merges frequent pairs, WordPiece picks merges by likelihood, SentencePiece is a training framework that treats input as raw bytes (or unicode) and can produce BPE or unigram tokenizers.
CoT reasons in a line; ToT explores multiple branches, evaluates them, and keeps the promising ones. ToT costs many more tokens but wins on hard search problems like puzzles.
Fixed-size chunking is simple; semantic chunks at meaning boundaries; recursive respects the document's own structure. Recursive is the modern default for anything hierarchical (headings, code).
DDP replicates the model per GPU; FSDP and ZeRO shard it, freeing memory so bigger models fit. ZeRO is a spec (stages 1-3); FSDP is a fully-shardable PyTorch implementation of the same idea.
Distillation shrinks the architecture (fewer parameters); quantization shrinks each parameter's bit count. They attack different axes of size and stack cleanly. QLoRA is quantization plus adapter fine-tuning.
DPO turns RLHF into supervised learning on preference pairs, no reward model needed. PPO still wins on some tasks but is trickier to tune.
Encoder-only understands text (BERT); decoder-only generates it (GPT, Llama); encoder-decoder maps input to output (T5, translation). Most modern LLMs are decoder-only.
Greedy is fastest but repetitive; beam is best for closed-ended tasks like translation; sampling with temperature and top-p is the default for open-ended generation.
Hallucination is a correctness problem and context window a capacity one; in RAG you tune retrieval to solve both at once, ground the answer without overflowing the prompt.
In-context learning adapts at inference by putting examples in the prompt; fine-tuning adapts by updating weights. Reach for prompting first; fine-tune when you cannot get there.
Evaluation measures quality; RLHF improves it. They are partners, not rivals, you need evaluation to even know whether RLHF worked.
Long context lets the model see everything; RAG selects what it should see. Long context is convenient but not free: cost scales, and quality falls off past the middle of the window.
LoRA reaches 90-99% of full fine-tuning quality for a fraction of the compute and storage; use full fine-tuning only when that last margin is genuinely worth it.
Native function calling is the fastest way to give one provider's model tools; MCP standardizes those tools so they work across any client without a rewrite.
Offline eval runs against fixed datasets before deployment; online eval measures real users after deployment. You need both, and neither is a substitute for the other.
KV cache reuses attention within a single generation; prompt caching reuses that cache across separate requests that share a prefix. Both attack the same waste from different angles.
Reach for RAG when knowledge changes or must be cited; fine-tune when you need the model to change how it behaves, not what it knows.
ReAct interleaves thought and action turn by turn; plan-and-execute drafts the whole plan up front, then executes it. Plan-and-execute is cheaper on long tasks; ReAct adapts better to surprises.
Retrievers fetch a shortlist fast; rerankers score the shortlist accurately. Production RAG uses both. A bi-encoder retrieves the top 100, a cross-encoder reranks to the top 5.
Attention parallelizes training and reaches long-range dependencies in one hop, which is why transformers displaced RNNs for nearly all sequence modeling.
Use semantic search when users phrase things their own way; keep keyword search when exact terms like codes and IDs must match. Most production systems run both.
Zero-shot asks the model to do a task cold; few-shot shows examples; chain-of-thought asks the model to reason step by step. They compose: few-shot chain-of-thought is often stronger than either alone.