Zenaique

Design how to detect an agent stuck in a loop from traces alone

Flashcard·Hard·4.0 · 0·~30s·Asked atBytedanceDifyDoordash
Attempt it
TL;DR

Hash (tool_name, normalized_args) on every tool span, flag a run when the last K hashes inside one agent run match or near-match without progress, and surface a loop_detected attribute.

Memory aid
Sign in to see the mnemonic that makes this stick.
Easy to grasp

Imagine watching a friend look for their keys. If you see them lift the same cushion three times in thirty seconds and still not find the keys, you would say they are stuck in a loop. An AI agent that calls tools is the same. Every time it calls a tool, you write down a short fingerprint of which tool it called and roughly what it asked for. Then you watch the last few fingerprints. If three or four in a row look the same and nothing has changed in the world (no answer, no new file, no next step), the agent is going in circles. You raise a little flag on the trace that says loop detected, and a dashboard later shows you which tools and which prompts cause this most often, so you can fix the root cause.

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Agent loops are one of the most common production failure modes in LLM agent systems. The model picks a tool, the tool returns something ambiguous, the model picks the same tool again with almost the same arguments, and the cycle repeats until a recursion_limit or token budget kills the run. From traces alone (no special framework support, no instrumented model), you can detect this cheaply with a hash and a sliding window.

This deep dive covers the signature design, the two detection rules (exact-repeat and near-repeat), how to surface and act on the signal at runtime, and how the same primitive generalizes to planner loops.

Designing the per-span signature

Every tool-call span gets one new attribute: tool_call.signature. The value is a hash of (tool_name, normalized_arguments). The hash function does not matter (sha256 is fine); the normalization is the part that decides whether the detector works.

Normalization rules that matter

  • Strip request ids, timestamps, and any auto-incremented field. If the agent's tool client passes a fresh request_id every call, every signature is unique and the detector goes silent.
  • Canonicalize JSON. Sort dict keys, lowercase string values where case is not meaningful, collapse whitespace, round floats to a stable precision.
  • Truncate or hash long free text. A 4000-character query string is almost always slightly different on every call due to whitespace or trailing context; truncate to 512 chars or hash the prefix.
  • Keep tool_name first-class. Do not fold tool_name into the args hash; you want to group dashboards by tool name later.

The signature is cheap to compute (a few microseconds per call) and is the single attribute the rest of the detector relies on.

Two complementary detection rules
Surfacing and acting on the signal
Generalizing to planner loops and tuning
Sign in to unlock the full deep dive.

Situations where this technique stops working.

Sign in to see when this approach fails.

2–4 min · Everything important, quickly.

Sign in to see the quick scan of the deep dive.
RuleCostFalse positivesCatches
Exact-repeat over last K signaturesVery cheapLowIdentical tool + args repeated
Near-repeat via Jaccard on argument tokensCheapMediumSame tool, slightly tweaked args each call
Plan-text hash repeatCheapLowPlanner loops where plan repeats but calls differ
Elapsed-time threshold aloneFreeHighOnly catches long runs, not loops

Real products, models, and research that use this idea.

  • LangGraph ships a built-in recursion_limit on graph runs as a hard floor; teams add Langfuse or Arize Phoenix span-level loop detection on top for the soft signal that fires before the hard limit trips.
  • OpenAI's Agents SDK and the Anthropic agent traces both expose tool-call spans cleanly enough that a 30-line signature hasher plus a sliding-window rule is the typical first implementation.
Sign in to see more production examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow do you tune K and the Jaccard threshold without a labeled dataset of looping runs?
A

Mine historic traces for runs that ended in recursion_limit or a long then failed pattern; treat those as positives and tune K and threshold on that retroactive set.

3 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating each tool call in isolation instead of comparing it against the recent history within the same agent run, so the loop is only obvious in hindsight.

Sign in to see all red flags and common mistakes.

60 second bullets to scan on the way to the call.

  • What goes into the per-span signature and why each field is normalized

  • How the sliding-window detector decides a run is looping

Sign in to unlock the revision sheet.

Primary sources. Browse if you want the original framing.

Similar questions

Same topic, related formats. Practice these next.

4 curated
Next question
Describe how end user thumbs up/down should flow back onto a trace
Flashcard·Easy