Describe LLM09 overreliance and one concrete UX mitigation

Flashcard·Easy·4.0 · 0·~30s·Asked atContextual Ai Roblox Zed

Attempt it

Overreliance is the failure where users accept LLM output as authoritative without verification, they sign the legal brief, ship the code, or take the medical advice the model produced. A concrete UX mitigation: do not present the answer as a verdict. Show retrieved sources next to each claim, mark uncertainty (confidence band or 'I am not sure about X'), and require an affirmative click before the answer is used for any irreversible downstream action. Friction calibrated to stakes is the design lever.

Your answer

OWASP LLM09 is overreliance, the failure where users accept LLM output as authoritative without checking it. The model produced a legal brief, the lawyer signed it without reading. The model produced code, the developer shipped it without testing. The model produced medical advice, the patient took it without consulting a clinician. The failure is on the human side, not the model side. The model might be wrong some fraction of the time (hallucinations, outdated information, plausible but incorrect reasoning), but overreliance is about what the user does with the output. If users verify, occasional model errors are caught. If users do not verify, every model error reaches the downstream consequence. The concrete UX mitigation that does not require a smarter model has three parts. Show sources next to each claim, so the user can verify with one click. Mark uncertainty explicitly, confidence bands, 'I am not sure about X', or 'check this with a clinician'. Add friction proportional to the stakes: a one-click accept for an email draft, a multi-step confirmation for an irreversible action. The lever is design, not model capability.

**Overreliance is a user-behaviour failure, not a model failure.** OWASP LLM09 names the case where users accept the LLM's output as authoritative, they sign, ship, prescribe, decide, without verifying the model was correct. The model might be wrong some fraction of the time. That is unavoidable on the current generation of systems. The downstream impact depends on whether anyone caught the error before it mattered. Overreliance is the gap between 'the model emitted something wrong' and 'a human acted on it'. The concrete UX mitigation is friction calibrated to stakes, expressed through three design moves. **Show sources** next to each claim so verification is one click away, RAG citations, web search references, or the model's own retrieved evidence. **Mark uncertainty** explicitly: confidence bands, 'I am not sure about X', or category-specific warnings like 'this is general information, not medical advice'. **Add affirmative confirmation** before any irreversible action, a refund, a code deploy, a contract signature. The amount of friction scales with the cost of being wrong. None of this requires a smarter model. It requires a product design that refuses to present LLM output as a verdict. The interview signal is recognising that overreliance is a design problem and that the lever is in the product surface, not in the model weights.

TL;DR

Overreliance is users accepting LLM output as authoritative; the mitigation lives in UX, sources next to claims, uncertainty cues, and friction calibrated to stakes.

Memory aid

Easy to grasp

Picture a confident friend who gives you driving directions without checking a map. Most of the time they are right, but sometimes they confidently send you the wrong way. The fix is not to make the friend smarter, it is to put a map on your phone, show the route's confidence ('estimated, traffic data 10 minutes old'), and ask 'are you sure?' before turning. The user is still in charge, but the friend is no longer a verdict. Overreliance happens when product design lets users skip the map step. The mitigation is putting the map step back in.

Key concepts

Safety and Guardrails

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

OWASP LLM09 is the entry in the Top 10 for LLM applications that catches teams off-guard most often. The first eight entries are about model-side and infrastructure-side failures: injection, insecure outputs, training data poisoning. LLM09 is different, the failure is in the user, not the model.

Overreliance is the gap between 'the model emitted something' and 'a human acted on it'. The model might be wrong some fraction of the time, but the downstream harm only materialises if no one verifies before acting. That makes the mitigation a product-design problem, and the lever is in the UI.

What overreliance is, and what it is not

Overreliance is the user behaviour pattern of accepting LLM output as authoritative without verifying it. It is not the same thing as the model being wrong, that is hallucination or factual error. Overreliance is what happens after the wrong output is produced, when no human catches it before the consequence lands.

The canonical 2023 example is the New York lawyer who filed a brief citing six case names the LLM had hallucinated. The hallucination was the model's failure; the overreliance was the lawyer not verifying the cases existed. The court fined the lawyer, not the model.

The two failures interact but are separate. A model with zero hallucinations would make overreliance harmless. A user who verifies every output would make hallucinations recoverable. Production systems live between those extremes, and the design question is how to nudge users toward the verification behaviour the system needs.

Why this is a product-design problem, not a model problem

Three UX moves that mitigate overreliance

Calibrating friction to stakes

Measuring overreliance before the public incident

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Perplexity's inline citation model, each claim links to a source the user can verify in one click, mitigating overreliance by design.
GitHub Copilot Workspace (2026) requires explicit user confirmation before applying multi-file refactors, calibrating friction to the action's blast radius.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you measure overreliance in production analytics?

Talk about source-link click-through rates, post-acceptance correction rates, time to accept on first-shown outputs, and structured user feedback on whether the output was used as-is, edited, or discarded.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating overreliance as a model problem to be solved by better training. The user-side accept without verification behaviour is the gap, and design is the lever.

60 second bullets to scan on the way to the call.

Define overreliance and contrast it with hallucination
Name three UX moves that mitigate overreliance without changing the model

Primary sources. Browse if you want the original framing.