A teammate randomized system prompts per user and expects the prompt cache to still help, what's wrong?

**Prompt caches** save money by storing the model's internal state for the leading part of a prompt that many requests share. When a new request arrives, the provider checks whether its prompt starts with the same bytes as a cached entry. If yes, it skips redoing the work for that part. The check is exact. The provider walks tokens from position zero and stops at the first token that differs. Whatever lives after that mismatch is not compared at all, because there is no point. The plan in the question puts a per-user salt at the very top of the prompt. The first token is different for every user. So the cache walk stops at position one for every request, even though the next two thousand tokens are identical across users. The cache hit rate is effectively zero, not ninety percent. The second mistake is the idea that providers match prompts by meaning. They do not. Anthropic, OpenAI, and Google all key the cache on exact bytes. The fix is simple: move the salt out of the cacheable prefix and put it in the user message section, after the shared system prompt has been matched.

**Two compounding errors.** The argument assumes prompt caches do fuzzy matching and assumes a small leading difference is forgivable. Neither is true. First error, exact prefix match. The cache walks tokens from position zero and stops at the first mismatch. A six-token salt at the head of the prompt means the walk stops at token one. Everything after the salt, no matter how identical, is never compared. The hit rate is approximately zero, not ninety percent. Second error, the semantic-similarity claim. No major provider does semantic prompt caching. Anthropic, OpenAI, and Google all compare exact tokens because the cache stores deterministic KV-cache state. A fuzzy match would not be valid against a non-matching prefix. The fix is structural. Canonicalize the cacheable prefix: model identity, task instructions, tool schemas, few-shot examples. Inject all per-user data after the cache breakpoint, in the user-message section, so the system-prompt prefix stays byte-identical across users. On Anthropic this is the cache_control marker; on OpenAI the prefix is implicit but the same discipline applies. Done correctly, the hit rate climbs from zero to whatever the prefix length warrants, often eighty to ninety-five percent on a chatbot or agent loop. The diagnostic question for any prompt-caching design is the same: what are the first N tokens, and are they truly the same bytes across requests? If anything varies in those tokens, the cache is dead.