Short Answer

CoT distillation and STaR: what made rejection-sampled reasoning FT effective?

How does QLoRA fit a 65B fine-tune onto a single 48GB GPU?

Why does QLoRA use NF4 instead of uniform INT4?

Why has prefix/prompt tuning largely lost to LoRA in production?

ORPO: what does it combine into one stage, and why is that attractive?

Explain bf16's advantage over fp16 and what gradient checkpointing trades

Why 'fine-tune on our company docs' usually fails: and what to do instead

Merge LoRA into the base vs keep it as a swappable adapter: when do you pick which?

Membership inference on fine-tuned models. Why are small FT datasets vulnerable?

Pick a LoRA rank for a domain task and justify why r=512 isn't a free upgrade

Why is LoRA's B initialised to zero and A to Gaussian. What breaks if both are Gaussian?

Explain why a rank-r B·A update captures most of full fine-tuning's signal

Can you stack two LoRA adapters? What's the failure mode?

What does LoRA's α/r ratio control, and why is α = 2r a common default?

Explain LIMA's Superficial Alignment Hypothesis and what it implies for your data budget

When would you pick KTO over DPO?

IA3 vs LoRA: what does IA3 parameterise, and what's the param-count win?

Why can fine-tuning on domain facts INCREASE hallucination?

Estimate peak VRAM for a 7B full-FT run in bf16 + Adam

Why does a small FT dramatically improve strict-JSON output?

Design the post-FT eval suite for a customer-support fine-tune

DPO: what's the closed-form trick that lets it skip the reward model?

DPO failure modes: large β, small β, noisy preferences

What does DoRA decompose, and why does it beat LoRA at the same rank?

FT-based distillation vs logit distillation: and what works with closed APIs