An LLM driven action call (e.g. one that books a flight or sends an email) times out, so the client retries. Explain the danger and how idempotency keys make retries safe.
A timeout doesn't tell you if the side effect happened, so retrying a non-idempotent action can double-send. An idempotency key lets the downstream recognize a repeat and replay the first result instead of re-executing.
Imagine mailing a letter and the postbox swallows it without a confirmation beep. Did it go in or not? If you drop a second copy just in case, your friend might get two identical letters. The fix is to number each letter with a unique ticket. The post office keeps a list of tickets it has already handled. When a duplicate ticket arrives, it doesn't mail a second copy — it just tells you 'already done, here's what happened the first time.' That ticket is the idempotency key, and it makes resending safe no matter how many times you panic and retry.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Retries are the most natural reliability reflex in distributed systems: something failed, try again. The reflex is correct for transient failures, but it hides a sharp edge that only shows up when the operation changes something in the real world. Sending an email, charging a card, and booking a seat are not like reading a value — doing them twice is not the same as doing them once.
The trap is the timeout. Engineers reason about a timeout as if it were a clean failure, a signal that the work didn't happen. It isn't. A timeout only says the response didn't arrive in the window you allowed. The request may have sailed through, completed the side effect, and had its acknowledgement dropped on the return trip. Failure and lost-success look identical from the client's seat.
This deep dive builds the idea from delivery semantics up: why exactly-once is not free, how an idempotency key reconstructs exactly-once effects on top of at least once delivery, the two implementation details that people get wrong (key reuse and atomic commit), and why LLM agents turn this from a backend footnote into a core design concern for any tool that touches the outside world.
Why a timeout tells you nothing useful
Picture the path of a single request. The client sends it, the server receives it, performs the side effect, and sends back a 200. Now imagine the acknowledgement is lost — a dropped packet, a load-balancer reset, a GC pause that blows past the client's timeout.
From the client's perspective, two completely different worlds are indistinguishable. In world one, the request never reached the server and nothing happened. In world two, the request fully succeeded and only the receipt was lost. The client sees the same thing in both: a timeout.
This is the whole danger in one sentence. The retry decision has to be made under genuine uncertainty about whether the side effect already occurred. If you treat the timeout as 'it failed' and the truth was world two, your retry duplicates the action.
Notice that simply lengthening the timeout doesn't fix this — it just moves the boundary. A slow but successful request can always exceed any finite timeout, and the moment you give up waiting, you're back to the same ambiguity. The problem isn't the duration; it's that the network can't promise you the response will arrive at all. You need a mechanism that makes the retry safe regardless of which world you're in.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
def send_email_tool(idempotency_key, to, body):
# check and claim the key atomically
existing = store.get(idempotency_key)
if existing is not None:
return existing # replay the original result
result = email_provider.send(to=to, body=body) # the side effect
store.put(idempotency_key, result, atomic_with=result) # commit together
return result
# key is derived ONCE from the logical action, reused across retries
key = f"send_email:{user_id}:{message_hash}"
send_email_tool(key, to=user_email, body=draft)| Aspect | Naive retry | Retry with idempotency key |
|---|---|---|
| Side effect on duplicate | Executes again (two emails) | Skipped; original result replayed |
| Effect guarantee | At-least-once (duplicates possible) | Exactly-once effect |
| State after timeout | Unknown and dangerous | Safe to repeat freely |
| What the server stores | Nothing | Key plus result, committed atomically |
Real products, models, and research that use this idea.
- Stripe's API uses an Idempotency-Key header so a retried charge creates the payment once and returns the original result on repeats.
- AWS SQS and Kafka exactly-once semantics rely on producer dedupe IDs / sequence numbers to suppress duplicate side effects.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhere should the idempotency key be generated — the client, the orchestrator, or the tool — and why?
Generate it at the layer that owns the logical operation's identity so the same key survives every retry; deriving it deterministically from the action beats a fresh UUID per attempt.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Assuming a timeout means the action failed and is safe to retry, when the request may have completed server-side and only the response was lost.
60 second bullets to scan on the way to the call.
Why a timeout leaves the side-effect outcome unknown
How a blind retry causes double-execution of a non-idempotent action
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.