Tulu instruction tuning family: what is it and who publishes it?

Flashcard·Easy·4.0 · 0·~30s·Asked atPatronus Pinterest Salesforce·Relevant atDatabricks

Attempt it

Tulu is an open instruction-tuned model family from Allen AI (AI2), together with the data mixture used to train it. The dataset blends high-quality instruction sources, FLAN, OpenAssistant, ShareGPT, code, math, and curated chain-of-thought data, into a single SFT (and, in later versions, DPO/PPO) recipe.

The headline is openness: the model weights, the exact mixture composition, the per-source filtering rules, and the training script are all published. Practitioners use Tulu in two ways: (1) as a ready to use general SFT base, and (2) as a reference data MIX whose composition can be borrowed when assembling a custom instruction-tuning corpus. Successor versions (Tulu 2, Tulu 3) progressively added preference tuning on top of SFT.

Your answer

**Tulu** is two things bundled together: an open instruction-tuned model family and the carefully documented dataset mixture used to train it. Both come from Allen AI, also known as AI2, a non-profit research lab. The model side is straightforward. You can download Tulu weights, run them, fine-tune them further, and use them in production under their open licence. They are competitive with other open instruction-tuned models in their parameter range. The data side is the more interesting part. AI2 publishes the exact mixture of sources that went into training: which instruction datasets, in what proportions, with which filtering rules applied. Sources typically include FLAN, OpenAssistant, ShareGPT, code instruction data, math problems, and curated chain-of-thought examples. The recipe is the artifact people actually study. Later versions, Tulu 2 and Tulu 3, extended the recipe to include preference tuning with DPO or PPO on top of the SFT base. Each version is published with its full training pipeline and ablation results. Many teams building custom fine-tunes use Tulu's recipe as a starting blueprint, copying the source mix and then adapting it for their domain.

**Tulu** is Allen AI's open instruction-tuned model family, paired with the full dataset mixture and training recipe used to produce it. The name applies to both the model and the data blend. The model is competitive open weights, useful as a general SFT base or as a starting point for further fine-tuning. The data mixture is the headline artifact. It combines several instruction sources, including FLAN, OpenAssistant, ShareGPT, code instructions, math problems, and chain-of-thought data, in published ratios with documented filtering rules. The training script and ablation studies are also public, so the recipe is reproducible end to end. The openness is what makes Tulu unusual. Most strong instruction-tuned models publish the weights but not the data mix or the filtering details. Tulu publishes everything, which lets practitioners study how each source contributes, copy the recipe for their own runs, or swap in domain-specific sources while keeping the general-chat slice constant. Successor versions extended the original SFT-only recipe with preference tuning. Tulu 2 added DPO; Tulu 3 explored a broader mix of alignment methods. Each version's release includes the same level of recipe transparency. For a fine-tuning practitioner, Tulu is most useful as a reference blueprint when designing a custom SFT mixture.

TL;DR

Tulu is Allen AI's fully open instruction-tuned model family plus its training data mixture. The headline value is the published recipe: weights, data ratios, filters, and scripts are all available to copy or adapt.

Memory aid

Easy to grasp

Picture two cookbooks. One tells you which dish to order at a restaurant; the other tells you the exact recipe, the supplier of each ingredient, the oven temperature, and the timing. Most large language models are the first kind: you get the finished dish without learning how it was made. Tulu is the second kind. It hands you the finished dish too, but it also hands you every step that produced it, so you can cook the same dish at home or change one ingredient and see what happens. That openness is why practitioners cite it when designing their own instruction-tuning recipes.

Key concepts

Instruction tuning Open models Data mixture

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Tulu is the canonical example of a fully open instruction-tuning project. Allen AI (AI2) publishes not just the model weights but the entire data mixture, the per-source filtering rules, the training script, and ablation studies showing how each source contributes. That level of openness is unusual in a field where most strong instruction-tuned models release only weights.

The value to fine-tuning practitioners is twofold. The model itself is competitive open weights, useful as a base or comparison point. The recipe, the mixture composition plus filtering and training procedure, is the more impactful artifact because it lets other teams replicate, study, and adapt the approach rather than guess at how it was built.

This deep dive covers what Tulu actually is, what the mixture contains, why recipe transparency changes how the community designs fine-tunes, the version progression from SFT-only to preference-tuned, and how teams typically borrow from Tulu when assembling their own recipes.

What Tulu refers to

The Tulu name applies to two related artifacts published together.

The first is the model family. Tulu, Tulu 2, and Tulu 3 are open instruction-tuned models at various parameter scales (typically 7B and 70B variants, tracking the underlying base model). The weights are downloadable from Hugging Face under the licences AI2 specifies for each release. You can run them, fine-tune them, and use them as drop-in chat models within the licence terms.

The second is the dataset mixture and training recipe. This is the more cited artifact. It includes the per-source token or row counts, the filtering rules applied to each source (deduplication, quality scoring, length bounds), and the training configuration (optimiser, learning rate schedule, hyperparameters, masking convention).

These two are intentionally bundled. AI2 publishes them together so the model is reproducible: anyone with the base model and the published mixture should be able to train an equivalent Tulu themselves, given enough compute. That reproducibility is the point.

The base model is usually a Llama or OLMo variant. AI2 publishes OLMo as a fully open pretrained base, and Tulu is often the instruction-tuning recipe applied on top. The combination is a complete open-source LLM pipeline: open pretraining data, open pretraining recipe, open instruction-tuning data, open instruction-tuning recipe.

What the mixture typically contains

Why open recipes change the field

The version progression

How practitioners actually use Tulu

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Allen AI's Tulu 3 release in early 2025 published the full data mixture, training script, and per-source ablation results for community use.
The Tulu mixture is cited in design docs for several 2026 domain-specific fine-tunes that borrow the general-chat slice while substituting their own domain data.

What an interviewer would ask next. Try answering before peeking at the approach.

QPick one source in the Tulu mixture, explain what it contributes, and predict what would change if you removed it.

For example, FLAN contributes task diversity and zero-shot instruction-following breadth. Removing it would narrow the model's task coverage and likely drop performance on instruction benchmarks like MMLU and BBH.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating Tulu as just a model family without recognising the data-mixture artifact, which is often the more useful piece for teams building their own SFT recipes.

60 second bullets to scan on the way to the call.

Who publishes the Tulu family and under what licence
The two artifacts the Tulu name refers to: model weights and data mixture

Primary sources. Browse if you want the original framing.