Compare AI Concepts
25 side-by-side comparisons of the concepts interviewers ask about most. Understand the tradeoffs, know when to use what, and never confuse the two again.
DDP vs FSDP vs ZeRO
DDP replicates the model per GPU; FSDP and ZeRO shard it, freeing memory so bigger models fit. ZeRO is a spec (stages 1-3); FSDP is a fully-shardable PyTorch implementation of the same idea.
DPO vs PPO
DPO turns RLHF into supervised learning on preference pairs, no reward model needed. PPO still wins on some tasks but is trickier to tune.
In-context learning vs Fine-tuning
In-context learning adapts at inference by putting examples in the prompt; fine-tuning adapts by updating weights. Reach for prompting first; fine-tune when you cannot get there.
LoRA vs Full Fine-tuning
LoRA reaches 90-99% of full fine-tuning quality for a fraction of the compute and storage; use full fine-tuning only when that last margin is genuinely worth it.