What belongs in the trace of every LLM call?

An LLM call isn't like a normal API call. The same input can produce different outputs, the quality depends heavily on the exact prompt, and each call costs money and takes a variable amount of time. So your trace has to capture enough to reproduce and explain any single call after the fact. That means three groups of fields. First, the **full prompt** you actually sent — system prompt plus all the context — and the model's complete output. Second, the model name and version, the input and output token counts, and the cost you computed from them. Third, the latency (time to first token and total) and a request or trace ID so you can correlate this call with logs, user reports, and downstream effects. The two wrong options drop the prompt to save space. Since outputs are prompt-dependent and non-deterministic, throwing the prompt away means you can never reproduce a bad answer.

The job of an LLM trace is to let you fully reconstruct and diagnose any call long after it happened. Because outputs are stochastic and prompt-dependent, and because cost and latency vary per call, you log a specific bundle. The **prompt and output** are non-negotiable: the exact assembled prompt — system instructions, retrieved context, user message — and the model's complete response. Without the input you cannot reproduce the output; with a paraphrased or summarized prompt you reproduce something else. Next, model name and version, because provider model updates silently shift behavior, and a trace that doesn't pin the version can't be re-run. Then token counts and computed cost, which drive billing, budgets, and routing decisions. Finally, latency split into time to first token and total, plus a correlation ID that threads this call through your other logs and any user report. The distractors both omit the prompt — one to "keep storage small," one by calling prompts an implementation detail. Both are fatal. Storage is cheap relative to the cost of an unreproducible production incident, and the prompt is the single most diagnostic field you have. Capturing only the final answer is logging the symptom and discarding the cause.