Flashcard: what is an 'agent runtime' and what does it do?

Flashcard·Easy·4.0 · 0·~30s

Attempt it

Your answer

**The agent runtime** is the code that sits around the language model and turns it into an agent. The model itself only produces text and forgets everything the moment a call ends. Every other operational concern lives in the runtime. Concretely, the runtime does five things. It runs the loop: call the model, parse the response, execute any tool calls, append the results, and call the model again. It owns the tool registry: the set of functions the model can ask for, plus their schemas. It manages state: the conversation history, intermediate results, and any persistent memory between turns. It enforces stopping conditions: the step limit, the cost cap, the timeout, the final-answer detection. It emits traces and logs so you can debug what the agent actually did. The runtime is the part you write or import from a framework. LangGraph, CrewAI, AutoGen, and the OpenAI Agents SDK are popular agent runtimes in 2026. Each gives you a slightly different way to express the loop (state graph, role-play crew, multi-agent conversation, function-calling SDK), but underneath they all do the same five things. The phrase to remember is: the model is the brain; the runtime is the body. Without a runtime, an LLM call is just a chat completion.

**The agent runtime** is the orchestration layer around the LLM. It owns the loop (call model, parse output, execute tool, append result, repeat), the tool registry, the conversation state and memory, the stopping conditions, and the observability hooks. This is the deterministic code half of an agent. The model is non-deterministic and stateless across calls; the runtime is the stable code you wrote that connects model decisions to real effects. When the model emits a tool-use block, the runtime parses it, validates the arguments against the tool's schema, executes the function, captures the result (including errors), and sends it back as the next observation. When the model emits a final answer with no tool call, the runtime detects that and exits the loop. Every operational lever lives in the runtime. Step budgets, cost caps, wall-clock timeouts, retry policies, idempotency keys, permission checks, and trace spans are all runtime concerns. The model cannot enforce any of these on itself because it has no view of the budget and no ability to refuse a call. Frameworks in 2026 each pick a different topology. LangGraph models the loop as a typed state graph. CrewAI models a crew of role-playing agents collaborating. AutoGen models agents as participants in a multi-party conversation. The OpenAI Agents SDK and Anthropic's MCP-aware tooling expose tool calling as a primitive and let you build the loop directly. Underneath, all four are runtimes implementing the same observe reason act repeat pattern.

Quick reference

TL;DR

The agent runtime is the deterministic code around the LLM that drives the loop, executes tools, manages state, enforces budgets, and emits traces. LangGraph, CrewAI, AutoGen, and the OpenAI Agents SDK are runtimes.

Memory aid

Easy to grasp

Picture a chess engine paired with a robot arm. The engine picks the move, but the arm is what actually slides the piece across the board, hits the clock, and writes down the move on the scoresheet. The engine alone is just numbers on a screen; the arm alone has no idea what to play. The agent runtime is the robot arm and the scoresheet and the clock and the rulebook, all rolled into one. The LLM only ever says what should happen next. The runtime is the part that actually makes it happen, keeps notes, watches the time, and decides when the game is over. Frameworks like LangGraph and CrewAI ship as ready-made runtimes so you do not have to build the arm yourself.

Key concepts

Agent Runtime Orchestration Agent Loop

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 min: define the runtime as the deterministic layer around the LLM, list the five responsibilities (loop, tools, state, budgets, observability), explain why each must live there rather than in the model, and name two or three production runtimes with their distinguishing topologies.

Real-world examples

Real products, models, and research that use this idea.

LangGraph: the runtime is a typed state graph; you declare nodes for the model and each tool, and the framework drives the loop with explicit edges and recursion limits.
CrewAI: the runtime coordinates multiple role-playing agents (planner, researcher, writer) with built-in delegation and shared memory.
AutoGen: the runtime treats agents as participants in a conversation, with message passing as the control primitive and a group-chat manager driving turn order.
OpenAI Agents SDK and Anthropic's MCP-aware tooling: SDK-level runtimes that expose tool calling, handoffs, and tracing as building blocks for custom loops.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QIf the model is stateless across calls, how does an agent remember what it did three turns ago?

QWhy are permission checks placed in the runtime rather than the system prompt?

QWhat changes when you swap LangGraph for CrewAI for the same task?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Calling the runtime 'just glue' or 'plumbing'. The runtime owns budgets, state, retries, validation, and observability; most agent reliability work happens in the harness, not the prompt.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.