**Context bloat means the agent is paying for tokens that no longer help it answer.** The defining property is asymmetric: input grows, output does not. So the cleanest indicator is exactly that asymmetry.
Think about why. Bloat usually creeps in from one of three places: retrieval starts pulling more chunks, conversation history starts being kept verbatim instead of summarized, or tool results accumulate without eviction. All three add input tokens. None of them change what the user asked for or what the agent needs to say in response, so output tokens stay roughly constant.
The other options each have their own problems. **Falling judge quality** is real but lagging. By the time the judge score moves a noticeable amount, bloat has already been established and quality has degraded. The token signal moves first. **Rising tool error rate** can correlate with bloat in agent loops, but the dominant causes are flaky tools, schema changes, and provider outages. **User session length** correlates with engagement, with feature releases, and with churn, too many other causes to be a clean bloat signal.
Watch input to output ratio per call. That single dashboard catches bloat before users notice the symptom.
**Bloat is, by construction, an input-side problem.** More input tokens for the same work. The cleanest dashboard signal is therefore a per-call input to output ratio that drifts upward over time while the output side stays flat.
The other options are tempting but lag or confound. Judge-model quality scores are the eventual symptom, not the leading indicator; by the time a 1-2 point drop is statistically detectable, the bloat is already two or three weeks old and the user-facing regression is in flight. Tool error rate is mostly driven by external causes (API changes, rate limits, schema drift) and only weakly tracks context size. User session length is a product metric, not a context metric; it moves with feature releases and seasonality.
The input to output ratio works because it inverts the question. Instead of asking 'are answers worse?' (slow to answer), it asks 'are we doing more work to produce the same answer?' (fast to answer). Bloat caused by a misconfigured retriever, a missing summarization step, or an eviction bug all show up the same way: input climbs, output does not.
In production, instrument input tokens, output tokens, and the ratio per call. Alert on the ratio trending up over a 7-day window. Pair it with a per-component breakdown (system, retrieval, history, tool results) so the alert points at the leaky slot.
Quick reference
TL;DR
Rising input tokens per call with flat output tokens is the earliest and cleanest fingerprint of context bloat; quality regressions lag this signal by days or weeks.
Imagine running a restaurant where the chef writes the same kind of orders every shift but the prep cooks keep wheeling more and more carts of vegetables into the kitchen. Same orders out, more ingredients in. That mismatch is the early warning that something upstream is over-prepping. You see it on the inventory report long before customers complain that dinner is slow or wrong. The same is true for an agent. The input-token meter rises while the output meter stays flat. By the time customer complaints arrive, the over-prepping has been going on for weeks; the meters told the story first.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Workloads where output token distribution is itself drifting (e.g., a feature change that asks for longer answers) The denominator of the ratio is no longer stable, so a rising ratio could be either bloat or output shrinking by design. Re-baseline after the feature change before reading the signal.
Single-turn classifier-style calls where input and output are both tiny The signal to noise on the ratio is poor at small magnitudes; absolute input tokens per call is a cleaner monitor in this regime.
Early prototypes without enough call volume for a rolling baseline Without a stable baseline, a 'rising ratio' alert fires on normal startup variance; defer the monitor until traffic supports a 7-day rolling window.
Quick scan · 2–4 min
Everything important, quickly.
2 min: define bloat as an input-side problem, justify input to output ratio as the leading indicator, explain why quality / errors / session length lag or confound, and name the per-slot decomposition pattern.
Real-world examples
Real products, models, and research that use this idea.
1LangSmith, Langfuse, and Phoenix all expose per-trace input and output token counts; computing the ratio per route is a standard 2026 SRE dashboard.
2OpenAI's usage dashboard breaks input tokens into cached and uncached, which makes leaks in the non-cached prefix immediately visible.
3Anthropic's prompt-caching telemetry exposes cache_creation_input_tokens versus cache_read_input_tokens; bloat in the non-cached part shows up as cache_creation rising.
4Datadog LLM Observability in 2026 ships a default 'bloat' panel that tracks the input/output ratio over time as part of its onboarding template.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decompose the input-token signal to localize which slot is leaking?
Sign in or go Pro to see the approach for this follow-up.
QWhat is the right alert threshold and window for an input to output ratio monitor?
Sign in or go Pro to see the approach for this follow-up.
QHow does prompt caching change the bloat signal?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Waiting for the judge-model quality score to fall before acting. That signal moves last and noisily; the input to output ratio moves first and cleanly.