How should a long-running agent task persist state so it can resume after a crash or deployment?

MCQ·Medium·4.0 · 0·~1 min·Relevant atAnthropic

Attempt it

A long agent task is fragile in the obvious way. The worker can crash, the pod can be evicted, a deploy can roll the process, a timeout can trigger a kill. If the task state lives only in process memory, every one of these events loses the work. The right pattern is **checkpointing**. After every step that changes state, the agent writes a checkpoint to a durable store. The checkpoint contains everything needed to continue: the full message history, the current step index, the results of any tool calls, and the next planned action. The store has to be durable and shared. Postgres, Redis with persistence enabled, or S3 are all reasonable. The key is the task id, so any worker can look up any task. When a worker picks up a task, it loads the latest checkpoint and continues from there. The other options fail in different ways. In-memory state vanishes on any restart. Local disk dies when the pod is replaced and is not visible to other workers. Snapshotting the model weights is meaningless because the model is loaded by the runtime, not stored per task. Frameworks like LangGraph ship a checkpointer abstraction backed by Postgres or similar. In production you almost always use the framework's built-in checkpointer rather than rolling your own.

The pattern is **durable checkpointing keyed by task id**, written after every state-mutating step. The checkpoint contents are the minimum needed to resume cleanly: the full message history, the current step index, the results of tool calls already executed, and the next planned action. The store is something that survives any single worker dying: Postgres, Redis with append-only persistence, or S3 with object versioning. LangGraph's `Checkpointer` plus Postgres is the canonical reference implementation in 2026. On resume, a worker loads the checkpoint for the task id, reconstitutes the agent state, and continues from the next planned step. The same checkpoint key serves both the resume case (after a crash) and the hand-off case (a different worker picks up the task). There is no special crash-recovery code path; resume is just load and continue. The other options each fail a specific test. In-memory state does not survive the typical pod lifetime, which is minutes, not the 30 minutes a long task can take. Local disk survives a process restart but not a pod replacement and is not shared with other workers. Model-weight snapshots confuse model state (loaded by the runtime) with task state (the only thing that varies per request). For tasks longer than a typical pod lifetime, anything other than durable checkpointing is unsafe.

Quick reference

TL;DR

Checkpoint after every state-mutating step to a durable store keyed by task id, with full message history, step index, and tool results, so any worker can resume.

Memory aid

Easy to grasp

Imagine writing a long story in chapters. Every time you finish a chapter, you save the whole story so far to a folder on a server, not just on your laptop. If your laptop dies, you can pick up any other laptop, open the folder, see exactly where you left off, and write the next chapter. An agent task is the same. After every step it takes, it writes down the entire story so far, the step number it is on, what tools it called and what they returned, and what it plans to do next. The notes live in a database that survives any single computer crashing. Any worker can pick up any task by reading the notes and continuing.

Key concepts

Checkpointing Durable State LangGraph

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

Start with why in-memory fails: pod lifetime is minutes, agent task is 30 minutes. State the canonical pattern: durable checkpoint after every state-mutating step keyed by task id, payload is messages plus step index plus tool results plus next action. Name LangGraph plus Postgres as the reference stack. Cover idempotent tool calls, then contrast plain checkpointing with workflow engines like Temporal and Inngest for cases where the agent is one node in a longer pipeline.

Real-world examples

Real products, models, and research that use this idea.

LangGraph ships a `Checkpointer` interface with Postgres, SQLite, and in-memory backends; Postgres is the canonical production choice for long-running agent tasks.
Temporal's durable workflow engine is widely used as the orchestration layer for multi-step agent pipelines at scale, providing exactly-once semantics on tool calls.
Inngest provides a similar durable-workflow primitive aimed specifically at LLM agents, with built-in retries and step-level state persistence.
Anthropic's documentation on building effective agents calls out durable state as a precondition for production multi-step tasks, recommending Postgres or equivalent over in-memory storage.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhere exactly do you write the checkpoint: before the tool call, after the tool call, or both?

QHow do you handle a checkpoint that has been poisoned by a bad reasoning step five steps ago?

QIs durable checkpointing enough, or do you also need a workflow engine like Temporal?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Holding agent state in process memory or on local disk, so any pod restart, deployment, or crash loses progress and forces the task to start over.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.