Pick the right signal that your hierarchical summary depth is too shallow

MCQ·Medium·4.0 · 0·~1 min·Asked atAnthropic OpenAI Microsoft

Attempt it

Hierarchical summarization uses multiple levels of compression. Level-1 summarizes recent old turns. Level-2 summarizes older level-1 blocks. Level-3 (if present) summarizes older level-2 blocks. Each level absorbs history from the level below it, and the deepest level holds the most distant era of the conversation in heavily compressed form. "Too shallow" means you do not have enough levels for the conversation length. The diagnostic fingerprint of too-shallow depth is **the bottom level filling up and starting to evict live verbatim turns**. The mechanism: as the conversation grows, level-1 accumulates older summaries. With no deeper level to drain into, level-1 grows past its token budget. The system then has two bad choices: either let level-1 grow into the verbatim slot (pushing recent turns out) or truncate level-1 (losing older content entirely). Both are symptoms of the same root cause, not enough depth. The fix is to add a deeper level. Older level-1 blocks get compressed into a level-2 summary; level-1 has room again; verbatim turns are no longer being pushed out. The other options describe different problems. Option 1 (forgetting recent turns) is a too-small verbatim slot, not a depth problem. Option 2 (factual errors) is a summarizer quality issue. Option 3 (concise output) is unrelated.

**The correct diagnostic for too-shallow hierarchical summary depth is the bottom level overflowing and pushing recent verbatim turns out.** In a hierarchical scheme, each level has a finite budget. Level-1 summarizes the older history that no longer fits as verbatim turns. Level-2 (if present) summarizes the older history that no longer fits in level-1. The conversation grows; older content drains downward through the levels. If there is nowhere for it to drain, the bottom level fills up and the system runs out of room. When there is no deeper level, the failure manifests at the boundary between summary and verbatim. The system has to choose: let the bottom summary spill into the verbatim slot (evicting recent turns) or truncate the bottom summary (losing older content). Production agents typically let the verbatim slot shrink because the summary is treated as protected once written. The visible symptom is that recent verbatim turns get evicted earlier than they should, which feels like an attention problem to the user but is structurally a depth problem. The fix is to add a deeper level: compress older level-1 blocks into a level-2 summary, freeing level-1 to hold more of the medium-history content while older history still survives in compressed form. **Why the other options are wrong.** - **Option 1 (forgetting recent turns)**: that is a too-small verbatim slot, not a depth issue. Adding levels does not help because the levels exist to compress older content, not to make room for recent turns. - **Option 2 (occasional factual error)**: that is a summarizer quality issue (model choice, drift, prompt design). Adding levels does not fix model errors. - **Option 3 (concise output)**: unrelated to structural depth; output conciseness is about prompting and model behavior.

Quick reference

TL;DR

The diagnostic fingerprint of too-shallow depth is the bottom summary level overflowing and pushing recent verbatim turns out; adding a deeper level absorbs older content and frees the working area.

Memory aid

Easy to grasp

Picture organizing a year of paper notes on a desk that only fits a small stack. If you only have one box for everything older than the desk stack, that box fills up fast and starts spilling onto the desk, pushing today's notes off. The fix is to add a deeper layer of filing: a year folder above the box. Old box contents move into the year folder; the box has room again; the desk stays clear for today's work. Summary depth works the same way. One level absorbs recent history; deeper levels absorb older eras; the verbatim turns stay protected.

Key concepts

Context engineering Agents Retrieval-augmented generation

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

3 minutes: the draining-system view, the overflow signature, how to distinguish depth from verbatim sizing, when to add a level.

Real-world examples

Real products, models, and research that use this idea.

Letta (formerly MemGPT) ships explicit core versus archival memory levels and exposes them as a tunable depth surface; adding levels is a configuration change.
LangGraph's checkpoint-aware state management supports hierarchical state with separate compression policies per level.
Anthropic's Claude Opus 4.7 long running agent guide recommends 2-3 summary levels for multi-hour sessions and instrumenting per-level token counts.
AutoGen's GroupChatManager exposes summary-depth controls for multi-agent conversations that can run for many turns.
Cognition's Devin and similar long-horizon coding agents use hierarchical summaries to span multi-day sessions without losing context.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does drift mitigation change for deeper levels?

QHow would you decide between adding a level versus enlarging level-1?

QWhat happens when the agent needs to retrieve a specific fact buried in level-3?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Confusing too-shallow depth (bottom level overflows) with too-small verbatim slot (recent turns get forgotten). They look similar to a user but have different fixes; reading the production signal correctly matters.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.