What is the Tulu instruction tuning family, and who publishes it?

Flashcard·Easy·4.0 · 0·~30s·Asked atPatronus Pinterest Salesforce·Relevant atDatabricks

Attempt it

Tulu is an open instruction-tuned model family from Allen AI (AI2), together with the data mixture used to train it. The dataset blends high-quality instruction sources (FLAN, OpenAssistant, ShareGPT, code, math, and curated chain-of-thought data) into a single SFT (and, in later versions, DPO/PPO) recipe.

The headline is openness: the model weights, the exact mixture composition, the per-source filtering rules, and the training script are all published. Practitioners use Tulu in two ways: (1) as a ready to use general SFT base, and (2) as a reference data MIX whose composition can be borrowed when assembling a custom instruction-tuning corpus. Tulu 2 added DPO preference tuning on top of SFT, and Tulu 3 kept DPO and added reinforcement learning with verifiable rewards (RLVR).

Your answer

**Tulu** is two things bundled together: an open instruction-tuned (trained to follow requests) model family and the carefully documented dataset mixture used to train it. Both come from Allen AI, also known as AI2, a non-profit research lab. The model side is straightforward. You can download Tulu weights, run them, fine-tune them further, and use them in production under their open licence. They are competitive with other open instruction-tuned models in their parameter range. The data side matters more. AI2 publishes the exact mixture of sources that went into training: which instruction datasets, in what proportions, with which filtering rules applied. Sources typically include FLAN, OpenAssistant, ShareGPT, code instruction data, math problems, and curated chain-of-thought examples. The recipe is the artifact people study. Later versions, Tulu 2 and Tulu 3, extended the recipe to include preference tuning with DPO or PPO on top of the SFT (supervised fine-tuning) base. Each version is published with its full training pipeline and ablation results. Many teams building custom fine-tunes use Tulu's recipe as a starting blueprint, copying the source mix and then adapting it for their domain.

**Tulu** is Allen AI's open instruction-tuned model family, paired with the full dataset mixture and training setup used to produce it. The name applies to both the model and the data blend. The model is competitive open weights, useful as a general SFT base or as a starting point for further fine-tuning. The data mixture is the main artifact. It combines several instruction sources, including FLAN, OpenAssistant, ShareGPT, code instructions, math problems, and chain-of-thought data, in published ratios with documented filtering rules. The training script and ablation studies are also public, so the recipe is reproducible end to end. The openness is what makes Tulu unusual. Most strong instruction-tuned models publish the weights but not the data mix or the filtering details. Tulu publishes everything. That lets practitioners study how each source contributes, copy the recipe for their own runs, or swap in domain-specific sources while keeping the general chat slice constant. Successor versions extended the original SFT-only recipe with preference tuning. Tulu 2 added DPO. Tulu 3 explored a broader mix of alignment methods. Each version's release includes the same level of recipe transparency. For a fine-tuning practitioner, Tulu is most useful as a reference blueprint when designing a custom SFT mixture.

TL;DR

Tulu is Allen AI's fully open instruction-tuned model family plus its training data mixture. The headline value is the published method: weights, data ratios, filters, and scripts are all available to copy or adapt.

Memory aid

Easy to grasp

Picture two cookbooks. One tells you which dish to order at a restaurant. The other tells you the exact recipe, the supplier of each ingredient, the oven temperature, and the timing. Most large language models are the first kind: you get the finished dish without learning how it was made. Tulu is the second kind. It hands you the finished dish too, but it also hands you every step that produced it. So you can cook the same dish at home, or change one ingredient and see what happens. That openness is why practitioners cite it when designing their own instruction-tuning setups.

Key concepts

Instruction tuning Open models Data mixture

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Tulu is the best-known example of a fully open instruction-tuning project. Allen AI (AI2) publishes more than the model weights. It also publishes the entire data mixture, the per-source filtering rules, the training script, and ablation studies showing how each source contributes. That level of openness is unusual in a field where most strong instruction-tuned models release only weights.

The value to fine-tuning practitioners is twofold. The model itself is competitive open weights, useful as a base or comparison point. The recipe (the mixture composition plus filtering and training procedure) is the more impactful artifact. It lets other teams replicate, study, and adapt the approach rather than guess at how it was built.

What Tulu refers to

The Tulu name applies to two related artifacts published together.

The first is the model family. Tulu, Tulu 2, and Tulu 3 are open instruction-tuned models at various parameter scales. The scales track the underlying base model, for example 7B, 13B and 70B for Tulu 2, and 8B, 70B and 405B for Tulu 3. The weights are downloadable from Hugging Face under the licences AI2 specifies for each release. You can run them, fine-tune them, and use them as drop-in chat models within the licence terms.

The second is the dataset mixture and training setup. This is the more cited artifact. It includes the per-source token or row counts, the filtering rules applied to each source (deduplication, quality scoring, length bounds), and the training configuration (optimiser, learning rate schedule, hyperparameters, masking convention).

These two are intentionally bundled. AI2 publishes them together so the model is reproducible. Anyone with the base model and the published mixture should be able to train an equivalent Tulu themselves, given enough compute. That reproducibility is the point.

The base model is usually a Llama or OLMo variant. AI2 publishes OLMo as a fully open pretrained base, and Tulu is often the instruction-tuning setup applied on top. The combination is a complete open-source LLM pipeline: open pretraining data, open pretraining setup, open instruction-tuning data, open instruction-tuning setup.

What the mixture typically contains

Why open recipes change the field

The version progression

How practitioners use Tulu

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Allen AI's Tulu 3 release in November 2024 published the full data mixture, training script, and per-source ablation results for community use.
The Tulu 3 SFT mixture is published on Hugging Face (allenai/tulu-3-sft-mixture). Domain teams can borrow its general-chat slice while substituting their own domain data.

What an interviewer would ask next. Try answering before peeking at the approach.

QPick one source in the Tulu mixture, explain what it contributes, and predict what would change if you removed it.

For example, FLAN contributes task diversity and zero-shot instruction-following breadth. Removing it would narrow the model's task coverage and likely drop performance on instruction benchmarks like MMLU and BBH.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating Tulu as just a model family without recognising the data-mixture artifact. That artifact is often the more useful piece for teams building their own SFT setups.

60 second bullets to scan on the way to the call.

Who publishes the Tulu family and under what licence
The two artifacts the Tulu name refers to: model weights and data mixture

Primary sources. Browse if you want the original framing.