When would you pick KTO over DPO?

**DPO** and **KTO** both teach a model human preferences without training a separate reward model. The big difference is the shape of the data each one eats. DPO needs pairs. For every prompt you must supply two responses, a chosen one and a rejected one. The model learns to push probability toward the chosen answer and away from the rejected one. KTO needs no pairs. Each row is a single response with one binary label: desirable or undesirable. That maps directly onto thumbs-up and thumbs-down ratings from a real app, where each user sees one answer and rates it. The name comes from Kahneman and Tversky's prospect theory, the idea that people feel losses more sharply than equal gains. KTO bakes that asymmetry into its loss function. Pick KTO when your feedback is naturally single-response, lopsided (far more good than bad, or the reverse), or a bit noisy. Pick DPO when you genuinely have clean A versus B comparisons to learn from.

**DPO** and **KTO** both skip the explicit reward model, but they require different data shapes. DPO consumes pairwise preferences. Each prompt carries a chosen response and a rejected response. KTO consumes binary labels instead. Each response is tagged desirable or undesirable on its own, with no partner needed. This matches production telemetry directly. A thumbs-up or thumbs-down button gives one rating per single response, never a clean pair. The loss draws on Kahneman-Tversky prospect theory. It applies a value function over the log-probability ratio against the reference model. Separate slopes weight losses more heavily than equal gains, mirroring human loss aversion. Pick KTO in three cases. First, when feedback is genuinely unpaired and forcing pairs means inventing or discarding data. Second, when classes are imbalanced, since KTO tolerates any ratio of desirable to undesirable. Third, when labels are noisy, where the prospect-theory shape resists label flips better than the Bradley-Terry loss under DPO. The trade is signal quality. When you do have natural pairs, like A/B response logs, DPO's direct comparison gives a slightly cleaner gradient. KTO is the pragmatic winner when real feedback is single-response, asymmetric, or messy.