5FLAN v2 (Google): millions of templated examples mapping classic NLP datasets to instruction format
This question is about **quality per example**, not total size. One LIMA example teaches the model more than one Alpaca example, even though Alpaca has 50 times as many.
The ranking tracks how directly humans shaped each example. At the top sits LIMA: a thousand examples hand-curated by researchers, every one polished. Its whole thesis is that a small clean set beats a large noisy one.
Next comes OpenAssistant, real volunteers writing and rating multi-turn conversations. Then Dolly v2, written by Databricks employees on the job, human but produced faster and with less curation.
Lower down is FLAN v2. It is clean, but it is built by pouring old benchmark datasets into templates, so examples are repetitive and narrow in what each one teaches. Last is Alpaca: an older model generated its own instructions with only light filtering, so each example carries the most noise.
The lesson: a model learns the average quality of what it sees, so curation per example beats raw volume.
The correct order is LIMA, then OpenAssistant, then Dolly v2, then FLAN v2, then Alpaca. The single axis is **quality per example**, which tracks how directly and carefully humans authored each item.
LIMA leads by design. It is a thousand examples, hand-curated to maximize per-example signal. OpenAssistant follows: volunteers wrote and ranked real multi-turn dialogues, so quality is high but more variable. Dolly v2 sits next, written by employees, genuinely human but produced quickly with light curation.
FLAN v2 ranks below the human-written sets despite being clean. It is templated from existing NLP benchmarks, so each example is narrow and many are near-duplicates. Volume is huge, but per-example novelty is low. Alpaca is last because an older model generated it with only light filtering, so each example carries the most noise and factual drift.
The trap is equating size with quality. Alpaca has 52x more examples than LIMA, yet ranks last per item. A model absorbs the average quality of its data, so curation per example wins.
Quick reference
TL;DR
Rank by how directly humans shaped each example: LIMA (hand-curated) > OpenAssistant > Dolly v2 > FLAN v2 (templated) > Alpaca (LLM-generated).
Imagine grading homework by how much care went into each answer, not how many answers there are. LIMA is a thousand essays each polished by an expert, so every example teaches a lot. OpenAssistant is real volunteers writing and rating conversations, still very human. Dolly is staff writing answers on the clock, good but rushed. FLAN reshapes old quiz datasets into one shape, so it is clean but repetitive and narrow per example. Alpaca had an older chatbot write its own homework with light checking, so each example is the cheapest and noisiest. More polish per item means more learning per item, even if the pile is smaller.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Your training budget is very large and you optimize for broad task coverage Aggregate coverage matters more than per-example density, so FLAN-style volume can outperform tiny curated sets.
You need a specific narrow capability the curated set never covers A small general set like LIMA may simply lack examples of your target task, so coverage trumps polish.
You are bootstrapping with no labeling budget at all Self-instruct generation like Alpaca is cheap and fast, useful before any human curation pipeline exists.
You are doing pure format adaptation rather than reasoning Templated FLAN-style data teaches output shape efficiently, where per-example reasoning depth is not the goal.
Quick scan · 2–4 min
Everything important, quickly.
4 min: name the per-example axis + walk the authorship ladder LIMA to Alpaca + explain why FLAN ranks low despite scale + flag the coverage regime where the ranking flips.
Side-by-side
Dataset
How produced
Per-example quality
LIMA
Hand-curated by researchers (1k items)
Highest, every example polished
OpenAssistant
Volunteers write and rate dialogues
High but contributor variance
Dolly v2
Employees write ~15k pairs on the job
Solid human, light curation
FLAN v2
Templated from NLP benchmarks (millions)
Clean but narrow and repetitive
Alpaca
GPT-3.5 self-instruct, light filtering
Lowest, noisy and shallow
Real-world examples
Real products, models, and research that use this idea.
1LIMA fine-tuned a 65B Llama base on just 1000 examples and matched far larger instruction sets in human preference tests, popularizing the curation-first thesis.
2OpenAssistant's human-rated conversation trees seeded the preference data behind several open chat models and remain a reference SFT corpus on Hugging Face.
3Alpaca's self-instruct recipe was so cheap that teams now generate synthetic SFT data with May-2026 flagship models like GPT-5.5 and Claude Opus 4.7, then filter hard before training.
4FLAN v2 templates still anchor broad task-coverage mixes used to pretrain instruction behavior in open models like Llama 4 before a curated polish stage.
5Modern stacks built with Axolotl or LLaMA-Factory mix a large FLAN-style base with a small LIMA-style curated set to get both coverage and per-example density.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you empirically rank two SFT datasets if you could only run a few fine-tunes?
Sign in or go Pro to see the approach for this follow-up.
QWhy does self-instruct data like Alpaca lower quality even after filtering?
Sign in or go Pro to see the approach for this follow-up.
QWhen would you deliberately mix a templated FLAN set with a curated LIMA set?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Conflating dataset size with quality. Alpaca has 52x more examples than LIMA, yet ranks last per example because LLM self-generation injects noise that hand curation removes.