Tulu-style instruction tuning trades volume for diversity: which axes of diversity matter most?

The correct choices are **A, B, and E**. **A (task diversity)**: covering many distinct skills, summarisation, classification, code, reasoning, extraction, translation, is the single strongest predictor of post fine-tuning general capability. A 1,000-example set that touches 50 task types consistently beats a 10,000-example set that touches only 5. **B (output-format diversity)**: models that have practised producing JSON, code blocks, lists, prose, and tables during SFT remain fluent in all of them at inference. Models trained only on one output shape regress when asked to produce others. **E (prompt-template diversity)**: varying how the same task is framed (different phrasings, system prompts, instruction structures) teaches the model that the task identity lives in the request semantics, not in a single surface form. The other options do not belong. **C (token-count diversity)** is fine to have for balanced batching but is a hygiene concern, not a quality driver. **D (tokenizer-version diversity)** is anti-recommended; tokenizer drift is a bug to pin down, not a regularisation strategy. **F (hardware diversity)** is irrelevant; the trained weights do not encode information about which GPU produced them. The underlying principle is the LIMA / Tulu line: past a quality floor of a few thousand carefully curated examples, additional gains come from breadth on the dimensions that matter, not from raw volume on dimensions that do not.

**The right choices are A, B, and E**, the three axes that align with the LIMA / Tulu line on instruction tuning. **Option A is task diversity.** Coverage across distinct skill categories is the strongest single predictor of post-FT general capability. A curated 1,000-example set covering 50 task types reliably outperforms a 10,000-example set covering 5. Each new task type adds a dimension to the manifold the model has to learn, and breadth transfers better than depth on any single slice. **Option B is output-format diversity.** Models that practised JSON, code, prose, lists, and tables during SFT remain fluent in all of them at inference. Models trained on a single shape regress measurably when asked to produce others. The forgetting is not absolute but is sharp enough to matter for production. **Option E is prompt-template diversity.** Varying how the same task is expressed under several phrasings and system-prompt styles teaches the model that task identity lives in the request semantics, not in a surface pattern. A model trained on one template locks in to it and breaks when users phrase things differently. The three rejected options each fail for a different reason. **C (token-count diversity)** is a batching hygiene concern; wide length distributions complicate efficient packing but are not a quality driver. **D (tokenizer-version diversity)** is anti-recommended; tokenizer changes are bugs to eliminate, not regularisation, and mixing versions produces inconsistent special-token handling. **F (hardware diversity)** is irrelevant; numerical differences between GPUs average out and the trained weights do not reflect which silicon produced them. The meta-principle: past a modest quality threshold, the marginal value of more examples is small compared to better coverage on the breadth axes that shape what the model can do.