Order the steps in building a 5k-pair DPO dataset on helpfulness.

**The correct order is 0, 1, 2, 3, 4, 5.** Each step needs the output of the previous step to exist, so the sequence is largely forced. First, you need prompts. These come from production traffic or a proxy that matches what users actually ask. Picking prompts later would mean generating candidates before knowing what to generate for. Second, you generate candidate responses for each prompt. Multiple candidates per prompt at a non-zero temperature give you real diversity to choose between. Third, you label the candidates. Humans or a strong judge model rank them, producing chosen and rejected pairs. The labelling step needs candidates to exist first. Fourth, you deduplicate. Many candidates from the same model are near-identical; pairs where chosen and rejected are essentially the same text teach the model nothing useful. Fifth, you filter for quality gap. Pairs where the chosen response is only marginally better than the rejected one carry low signal; keeping them dilutes the training loss with near-random labels. Sixth, you split and store. Train, dev, and holdout splits go to the trainer in the standard format, with the random seed frozen for reproducibility. The two filtering passes specifically belong between labelling and splitting because both filters need the labels to operate.

**The order is 0, 1, 2, 3, 4, 5.** The sequence is dictated by data dependencies: each step needs the artifact produced by the previous step. Step 0 sources prompts from real traffic or a representative proxy. This anchors the preference signal to what users actually ask, not to academic prompt distributions. Step 1 generates 4 to 8 candidates per prompt from the current SFT model at temperature around 1.0. Multiple candidates and a non-trivial sampling temperature are both load-bearing; without diversity you get pairs that are too similar to learn from. Step 2 labels the candidates. Humans or a strong judge model produce chosen and rejected pairs. The labelling needs candidates to exist first, and the size of the labelling budget often determines how many prompts you can include. Steps 3 and 4 are the two filtering passes, both placed after labelling because both need the labels. Step 3 drops near-identical text pairs, which arise when the model produces close variants of the same answer. Step 4 drops low-confidence pairs where the chosen and rejected responses are essentially tied; these pairs carry near-random labels and harm training. Step 5 splits the cleaned data into train, dev, and holdout sets, stores them in the canonical {prompt, chosen, rejected} format, and freezes the generation seed so the dataset can be reproduced or extended consistently. The filtering passes are placed between labelling and splitting because earlier would be impossible (no labels yet) and later would corrupt the splits with noise.