Order these instruction datasets from highest to lowest quality per example
- 1LIMA (Meta, 2023): 1000 hand-curated high-quality examples covering many tasks
- 2Alpaca (Stanford): 52k examples generated by GPT-3.5 via self-instruct, lightly filtered
- 3Dolly v2 (Databricks): ~15k human-written instruction/response pairs by Databricks employees
- 4OpenAssistant (LAION): community-contributed, multi-turn, human-curated conversations
- 5FLAN v2 (Google): millions of templated examples mapping classic NLP datasets to instruction format
Rank by how directly humans shaped each example: LIMA (hand-curated) > OpenAssistant > Dolly v2 > FLAN v2 (templated) > Alpaca (LLM-generated).
Imagine grading homework by how much care went into each answer, not how many answers there are. LIMA is a thousand essays each polished by an expert, so every example teaches a lot. OpenAssistant is real volunteers writing and rating conversations, still very human. Dolly is staff writing answers on the clock, good but rushed. FLAN reshapes old quiz datasets into one shape, so it is clean but repetitive and narrow per example. Alpaca had an older chatbot write its own homework with light checking, so each example is the cheapest and noisiest. More polish per item means more learning per item, even if the pile is smaller.
Detailed answer & concept explanation~7 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: name the per-example axis + walk the authorship ladder LIMA to Alpaca + explain why FLAN ranks low despite scale + flag the coverage regime where the ranking flips.
| Dataset | How produced | Per-example quality |
|---|---|---|
| LIMA | Hand-curated by researchers (1k items) | Highest, every example polished |
| OpenAssistant | Volunteers write and rate dialogues | High but contributor variance |
| Dolly v2 | Employees write ~15k pairs on the job | Solid human, light curation |
| FLAN v2 | Templated from NLP benchmarks (millions) | Clean but narrow and repetitive |
| Alpaca | GPT-3.5 self-instruct, light filtering | Lowest, noisy and shallow |
Real products, models, and research that use this idea.
- LIMA fine-tuned a 65B Llama base on just 1000 examples and matched far larger instruction sets in human preference tests, popularizing the curation-first thesis.
- OpenAssistant's human-rated conversation trees seeded the preference data behind several open chat models and remain a reference SFT corpus on Hugging Face.
- Alpaca's self-instruct recipe was so cheap that teams now generate synthetic SFT data with May-2026 flagship models like GPT-5.5 and Claude Opus 4.7, then filter hard before training.
- FLAN v2 templates still anchor broad task-coverage mixes used to pretrain instruction behavior in open models like Llama 4 before a curated polish stage.
- Modern stacks built with Axolotl or LLaMA-Factory mix a large FLAN-style base with a small LIMA-style curated set to get both coverage and per-example density.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you empirically rank two SFT datasets if you could only run a few fine-tunes?
QWhy does self-instruct data like Alpaca lower quality even after filtering?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Conflating dataset size with quality. Alpaca has 52x more examples than LIMA, yet ranks last per example because LLM self-generation injects noise that hand curation removes.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.