Which proxy signals are valid indicators of potential quality drift in a production LLM system?

**Quality drift** is when a production LLM system silently gets worse over time even though nobody changed the model on purpose. The inputs shift, a vendor pushes a new model version, a prompt edit lands, or retrieval data goes stale. You need cheap signals that flag the slide before users complain loudly. Four of the options are valid proxies. Declining thumbs-up or explicit feedback means users are less happy with the outputs. A rising rate of follow-up clarification questions means answers are unclear or incomplete. A refusal rate that jumps off its baseline signals a safety-tuning change or an attack spike. Falling LLM-judge scores on a frozen canary set directly measure quality on a controlled set. Two options are tempting but wrong. A drop in average response length does not mean lower quality. A tighter, shorter answer is often better. And serving latency rising past the 95th percentile is an infrastructure concern, not an output-quality one. A slow but correct answer has not drifted in quality. The practical rule: watch satisfaction signals plus a frozen canary eval, and keep length and latency on a separate operational dashboard.

**Quality drift** means a production system degrades without a deliberate change. You monitor it with cheap proxy signals plus a controlled eval. Four options qualify. Declining feedback rate and rising follow-up rate are user-side proxies. Both track whether outputs satisfy intent. Refusal rate deviating from baseline is a behavioral proxy. It catches safety-tuning shifts or adversarial spikes. Falling judge scores on a frozen canary set are the direct measure. The canary is fixed, so any movement is a model or pipeline change, not input shift. The two distractors confuse system metrics with quality. Response length is not quality. A prompt refactor can make answers shorter AND better. Serving latency is infrastructure. A slow answer can still be perfectly correct. Senior interviewers probe one nuance. Proxy signals are confounded by input distribution shift. More follow-ups may mean harder questions arrived, not worse answers. So you split the signal. A frozen canary isolates model and pipeline drift. Live proxies capture both input shift and quality drift together. You triangulate. When canary scores hold steady but live proxies degrade, your inputs changed, not your model. When the canary itself slides, the model or its version drifted.