Find the flaw: this semantic answer cache started serving answers to the wrong questions

The flaw is the number in the threshold check: `cosine(q_vec, hit.vec) > 0.75`. For a **semantic answer cache**, 0.75 is far too loose. Here is why it breaks. The cache reuses a stored answer whenever a new query's embedding is close to a cached one. But two questions can share most of their topic words and still want completely different answers. "How do I reset my password" and "how do I change my password" sit well above 0.75 in cosine similarity, yet one is about recovery and one is about a routine update. When they collide, the cache confidently returns the wrong answer. That is worse than a cache miss, because the user gets a fluent, plausible response that does not address their actual question. The fix is to raise the threshold to something tight, around 0.95, so only near-identical paraphrases produce a hit. A miss just runs the pipeline; a false hit lies to the user.

The bug is the cosine threshold: `> 0.75` is far too loose for a **semantic answer cache**. The right value is tight, typically around 0.95 or higher. The reasoning is about precision versus recall. In retrieval, a generous threshold is fine. You fetch several candidates and let reranking and the LLM sort them out, so a loose bar costs you nothing but a few extra chunks. A cache is the opposite. A hit commits to one stored answer with no second stage. So the cache needs high precision: a false hit serves a wrong answer with full confidence, which is strictly worse than missing and running the pipeline. At 0.75, distinct questions that merely share topic vocabulary collide. "How do I reset my password" and "how do I change my password" land above 0.75 but want different answers. The cache returns the cached one and the user is misled. Raising the bar to ~0.95 means only genuine paraphrases — "what's my refund window" versus "how long do I have to get a refund" — produce a hit. Beyond the threshold, a production semantic cache also needs to exclude dynamic or risky intents, anything personalized or time-sensitive. It must also invalidate entries when the underlying index changes, so stale answers do not persist.