**Audit logs** answer one question: can a regulator, a security reviewer, or a court reconstruct exactly what happened in this interaction. The fields you log are chosen to make that reconstruction possible without creating new compliance problems.
The right answers cover three properties. **Reproducibility**: model id and version, system-prompt hash, and decoding parameters let a reviewer rerun the request months later and get a comparable output. **Traceability**: classifier id and version plus the rule that fired tells the reviewer which safety decision was made and why. **Attribution**: the user's IP and a tokenised session id let the reviewer tie the event to a real but redacted identity.
Two wrong answers are footguns. Storing raw PII in plaintext violates GDPR, HIPAA, and most other data-protection regimes; the fix is tokenisation or hashing with the raw data held in a separate, access-controlled vault. Storing free-text engineer speculation pollutes the log with opinions that have legal weight in litigation but no investigatory value; if an engineer needs to record a hypothesis, that belongs in an incident-review document, not in the audit log.
Tool-call outcomes, every tool the agent tried to call and whether the orchestrator allowed, denied, or executed it, are what regulators care most about, because tool calls are where the model meets real-world side effects.
**An audit-ready guardrail log is structurally different from a debug log.** Debug logs are for engineers chasing a single bug and may contain free text, full payloads, and stack traces. Audit logs are structured, redacted, retention-bounded, and designed to survive cross-examination by a regulator months or years later.
**Reproducibility fields.** The model id and version, a hash of the exact system prompt used, and the decoding parameters (temperature, top-p, max tokens, seed where supported) let a reviewer rerun the request against the same model snapshot. Versioning of system prompts matters because prompts change weekly in production; logging the hash plus a content-addressed pointer to the full text in a separate store gives both compactness and reconstructability.
**Traceability fields.** Every classifier that ran on the request leaves a record: classifier id (Lakera Guard, Llama Guard 4, OpenAI Moderation, the orchestrator's own rules), classifier version, raw score, and the policy rule that fired if any. This makes safety decisions auditable rule by rule and is the evidence base that the EU AI Act Article 12 logging requirements and NIST AI RMF Measure function expect.
**Tool-call outcomes.** Each tool the model attempted, the arguments (with PII tokenised), the orchestrator decision (allowed, denied, executed), and the result class. Regulators care about side effects, and tool calls are where side effects live.
**Attribution with redaction.** User IP, tokenised session id linked to an authenticated identity, with all PII fields tokenised or hashed and the raw data held in a separate vault under stricter access control. This is the GDPR-compatible posture: the audit log proves an event happened to a real person without storing the personal data alongside it.
The two trap options, raw PII in plaintext, free-text engineer speculation, fail an audit.
Quick reference
TL;DR
Audit logs need reproducibility, attribution, and traceability, model and prompt hash, classifier verdicts, tool outcomes, and tokenised identity.
An audit log is like the flight recorder on a plane. It should let an investigator months later reconstruct what happened: which plane, which version of the software, which weather, which actions the pilot took. It is not a place for the pilot to write their feelings about why the flight went sideways, and it is definitely not a place to record the passengers' personal data in plaintext. Same with LLM audit logs. Record the facts that let a reviewer reconstruct the run, attribute it to a real (but tokenised) user, and trace every safety decision back to a rule and a classifier version. Speculation and raw PII both get the log thrown out by an auditor.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Throwaway prototypes processing synthetic data with no real users There is no real PII or regulatory exposure; a simpler debug log is adequate until production launch.
Pure internal tooling used only by engineers debugging the system itself Operational debug logs serve a different purpose and can be retained briefly without the audit-log structure, though even internal tools that touch customer data should follow the redaction rules.
Batch pipelines with no real-time tool calls and no user-facing interaction The tool-call outcome category collapses, the attribution model changes (batches are run by a service principal not a user), and the schema simplifies; reproducibility and traceability still apply.
Quick scan · 2–4 min
Everything important, quickly.
3 min: three properties of an audit log (reproducibility, traceability, attribution); the core fields in each property; why plaintext PII and free-text engineer speculation are the two anti-patterns; how the schema maps to EU AI Act Article 12 and NIST AI RMF Measure.
Real-world examples
Real products, models, and research that use this idea.
1The EU AI Act Article 12 logging requirements for high-risk AI systems specifically expect reproducibility and traceability fields of the kind described.
2AWS Bedrock Guardrails and Azure AI Content Safety emit structured policy-decision events with classifier id, score, and rule fields that map directly to audit-log requirements.
3Anthropic's Claude API and OpenAI's responses API both include request id, model id, and usage data that feed reproducibility fields; teams add the system-prompt hash and rule chain at the orchestrator layer.
4ISO 42001 (AI management systems) requires evidence of safety controls and decisions, with redaction and retention policies that align to GDPR and the EU AI Act.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you design the retention policy difference between the main audit log and the PII vault?
Sign in or go Pro to see the approach for this follow-up.
QWhat is the right structure for a tool-call audit-log record so a regulator can answer 'did this agent ever email an external recipient' in a single query?
Sign in or go Pro to see the approach for this follow-up.
QHow do you log a content-policy decision in a way that an auditor can rerun and verify months later when the classifier has been updated?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating the audit log as a debug log. Debug logs collect free text and full payloads for engineers. Audit logs are structured, redacted, and designed to survive cross-examination by a regulator.