Three UX patterns that keep reasoning model latency acceptable in production
Reasoning models can take many seconds or minutes before returning a final answer. Name practical UX patterns that keep the experience acceptable without abandoning slow thinking models entirely.
Stream progress, route easy queries to fast models, and use async jobs with clear expectations so multi-second reasoning feels intentional.
Waiting for a slow chef is tolerable if you see prep happening, if simple orders go to the fast grill, and if huge banquets get a pager instead of blocking the counter. Reasoning APIs need the same: show activity, do not send every question down the slow path, and offload monster tasks to background jobs.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Reasoning quality often costs latency. Interviewers want product engineering answers — not just bigger clusters — for keeping users engaged during slow thinks.
We cover three durable UX patterns plus operational guardrails.
Streaming partial progress
Blank UI during 20–60s generation feels broken. Server-sent events carrying phased status — "parsing problem," "testing hypothesis" — reassure users. Some products stream abbreviated thinking bullets rather than full CoT.
First-byte and first-progress SLIs should be monitored separately from total completion time.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI o-series APIs expose reasoning effort knobs and streaming for long requests.
- Perplexity routes simple lookups to fast models while deep research runs async.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you build a difficulty router for reasoning vs chat?
Features: token length, domain tags, user tier, classifier confidence, fallback on timeout.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Relying only on faster hardware without UX patterns — users still churn on blank screens during 30s thinks.
60 second bullets to scan on the way to the call.
Stream partial progress during thinking
Route easy queries to non-reasoning models
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.