Multiple users complain that your AI assistant 'ignores the documents I uploaded and just makes things up'. Before involving an ML engineer, which pipeline layer should you investigate first?

The complaint is 'the AI ignored my documents and made things up'. The instinct is to blame the search part, but that's usually wrong here. A RAG system has two separable jobs. First, retrieval finds the right document chunks and places them in the prompt. Second, the model reads those chunks and writes an answer. The user's words point at the second job, not the first. The layer to check first is the **augmentation prompt**, the instruction that wraps the retrieved chunks before the model sees them. If that prompt never says 'answer only from the context below, cite the chunk you used, and refuse if it's not enough', the model treats the chunks as optional hints. It blends them with what it learned in training, and when the chunks feel thin it guesses. The giveaway is that the chunks are usually right there in the prompt; the model just wasn't told to stick to them. That makes the fix a small prompt edit, far cheaper than swapping the embedding model or retraining anything. Check the prompt, confirm the chunks were retrieved, then escalate only if grounding directives are already present.

Investigate the **augmentation prompt** first. The complaint 'ignored my docs and made things up' is the classic signature of missing grounding directives, not broken retrieval. Split the pipeline into two failure classes. Retrieval-missing means the relevant chunks never made it into the prompt; you'd see this as low context recall, and the fix lives in embeddings, chunking, or the vector index. LLM-ignoring means the chunks ARE in the prompt but the model didn't anchor to them; the fix lives in the prompt. The one diagnostic that separates them: log the actual prompt sent to the model. If the right chunks are present and the model still hallucinated, retrieval is fine and the augmentation layer is the culprit. A grounding correct prompt does three things. It instructs the model to answer only from the supplied context and to cite which chunk each claim came from. It also tells the model to refuse or say 'I don't know' when the context is insufficient. Without these, the model defaults to parametric memory whenever the context is weak. The fix is often a three-line system-prompt change, which is why it's the right first stop, before touching the model version or the vector database.