Pick the most reliable trigger for refreshing a rolling conversation summary

A rolling summary needs a trigger that decides when to refresh. Four candidate triggers sound reasonable in isolation, but only one is reliable in production. **Every turn.** This wastes most of its work. Many turns add only a few tokens, and re-summarizing the same content every turn burns LLM calls for no gain. **User reminder.** Relying on the user to say "please remember this" is fragile. Most users never do, and by the time someone thinks to mention it, the relevant context may already be out of the window. **Downstream symptom (refusal or low confidence).** By the time the model refuses or hedges, the summary refresh is already late. The bot has already failed to answer the user. **Soft token threshold.** When the live conversation tail crosses, say, 70 percent of the reserved window, the summarizer fires. This is the reliable choice. It is proactive (fires before the budget is exceeded), cheap (does not run on every turn), and observable (the trigger is a number, not a behavior signal). The soft threshold is the same shape as budget watermarks elsewhere in the system: cache eviction high-water marks, autoscaling triggers, queue backpressure. Borrow the pattern.

A reliable summary-refresh trigger has to be **proactive** (fires before the budget overflows), **cheap** (does not run when no work is needed), and **observable** (mechanical to monitor and tune). A soft token threshold on the live conversation tail meets all three; the other options each fail at least one. **Every-turn refresh** fails on cheapness. Many turns add only a handful of tokens, and re-summarizing the same content on every turn means most summarizer calls do no useful work. Total compute and latency overhead scale linearly with conversation length for no quality benefit. **User-reminder refresh** fails on proactivity and reliability. Most users never explicitly ask the model to remember anything. The few who do tend to do so after they have already noticed something was forgotten, which means the context they want preserved is already gone. The trigger fires after the failure, not before. **Downstream-symptom refresh** (refusal, low confidence) also fails on proactivity. Refusals and low confidence are end of pipeline observations. By the time they appear in production, the user has already gotten a bad response. The summary refresh is downstream of the failure, not upstream. **Soft-threshold refresh** fires when the live conversation tail crosses a fraction of the reserved window (commonly 65-80 percent). The trigger is proactive (fires before overflow), cheap (only when the budget pressure is real), and observable (the threshold is a number, easy to monitor and tune). It is the same pattern as cache eviction high-water marks, autoscaler triggers, or queue backpressure: a soft early signal that gives the system time to act before a hard limit is hit. Production systems pair the soft threshold with a hard ceiling: if the soft refresh fails or is async-pending, the hard ceiling forces a synchronous refresh to prevent overflow. The soft threshold does almost all the work; the hard ceiling is overflow insurance.