Which labeling UI detail guards against position bias?

MCQ·Medium·4.0 · 0·~1 min·Asked atGroq Notion Zoho

Attempt it

**Answer: A.** Per-comparison randomization is the standard guard against position bias. The problem: human raters (and LLM-as-judge models) systematically favor one position. Often it is the first or left-side response. If the candidate model always occupies the same slot, every label is contaminated in the same direction. The reward model trained on those labels inherits the bias. Randomizing which side each response appears on for every comparison converts the systematic bias into noise that averages out across the dataset. Logging the position lets you measure residual bias afterward. The other options make things worse. Fixed placement is exactly the bug. Showing reward model (RM) scores anchors annotators on the existing model's judgment. That defeats the purpose of independent human input and can amplify RM mistakes. Time-based skipping discards the hardest, most informative comparisons (close calls take longer to judge). It leaves you with only the easy ones, which are less useful for training.

**Answer: A.** Per-comparison randomization is the standard guard against position bias in preference labeling UIs. **The mechanism.** Both human raters and LLM-as-judge models systematically favor one position, typically the first or left-side response. If the candidate (or newest) model always occupies the same slot, every label gets a small consistent push in the same direction. The reward model trained on those labels inherits the bias and the policy optimized against the RM exploits it. Randomizing which side each response appears on, per comparison and per rater, converts the systematic bias into random noise. Across many comparisons the noise averages to zero in expectation, while the genuine quality signal remains. Logging the displayed position alongside the label lets you audit the setup. Each model should appear on each side about half the time. If not, the randomization is broken. The win rate by displayed position shows how strong the raters' bias is. **Why the other options fail.** - **Fixed left for the newest policy.** This is exactly the bug. Consistency for raters costs you systematic bias on every label. - **Show RM scores.** This anchors annotators on the existing model's judgment, defeating the purpose of getting independent human input. It also amplifies RM errors because annotators tend to defer to displayed scores under cognitive load. - **Time-based skipping.** Close-call comparisons take longer because they are hard. Skipping them discards the most informative pairs and leaves you with easy comparisons that add little signal.

TL;DR

Per-comparison randomization of which side each response appears on converts position bias from a systematic distortion into noise that averages out across the dataset.

Memory aid

Easy to grasp

Imagine two pies in a contest and the judges stand on one side of the room. If pie A is always on the left, the left-side judges naturally lean toward it. The contest is biased in pie A's favor before anyone tastes anything. The fix is to randomly swap which pie sits on which table for every judge. The favored side still gets picked too often. But now half the time it is pie A and half the time it is pie B, so the bias becomes noise that cancels out. Showing the judges the scores from a previous contest would defeat the purpose. Fixing the pie locations is exactly the bug. Skipping the hard comparisons just throws out the most useful data.

Key concepts

RLHF

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Position bias is one of the most studied and most underestimated sources of contamination in preference-data collection. It is small per comparison. But it accumulates systematically over thousands of labels and propagates from labelers to the reward model to the policy. The correct option is A: randomize which side each response appears on for every comparison. That breaks the systematic part of the bias.

A common UI design mistake is fixing the candidate model to a consistent slot for rater convenience. That convenience costs you a contaminated reward model, which is exactly the artifact downstream RL will exploit.

Why position bias is structural

Position bias has multiple sources that all push in the same direction. Cognitive anchoring causes humans to weight the first option encountered more heavily. Left to right reading cultures process the left-side response first by default. Layout habits can add to the effect.

The net effect is that, when quality is equal, raters often pick the left or first response more than half the time. The size of the effect varies by task and rater.

LLM judges show the same bias. Even when explicitly told to avoid position bias in the prompt, judges measurably favor one position in side by side comparisons. In the MT-Bench study (Zheng et al. 2023), GPT-4 gave the same verdict after the two responses were swapped in only 65 percent of pairs. Claude-v1 favored the first position in 75 percent. MT-Bench therefore judges each pair in both orders, and AlpacaEval randomizes which output appears first.

The per-comparison bias is small, but it compounds. Suppose a reward model is trained on 100,000 labels with a 10 percent systematic position bias on the candidate model side. It learns to favor the candidate by an amount roughly proportional to that bias. The policy trained against that RM then exploits the bias, producing responses that are not actually better, just structured in ways the biased RM prefers.

Why per-comparison randomization works

Why each wrong option is a recognizable mistake

Operational discipline and how this lives in production

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

AlpacaEval randomizes which output its LLM judge sees first, so position bias becomes noise instead of a push toward one model
Arena-Hard judges each pair twice with the positions swapped, to cancel the judge's position bias

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you audit residual position bias after randomization is in place?

From the position log, check that each model appears on each side about half the time. A skew means randomization is broken. Then track the win rate by displayed position: it measures the raters' bias, and drift can point to a secondary bias (length, formatting) acting through position.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Fixing the candidate model to a consistent slot for rater convenience. That is exactly what creates systematic position-bias contamination of every label.

60 second bullets to scan on the way to the call.

Why position bias is structural in both human and LLM judging
How per-comparison randomization converts systematic bias into noise

Primary sources. Browse if you want the original framing.