Context engineering becomes parameterised by tenant: same persona and system prompt, but top-k, summary depth, rerank, and model class scale with the plan, so free stays profitable and enterprise gets what it pays for.
Think of an all you can eat restaurant that has two memberships. The basic membership gets a small plate, no dessert station, and house-brand soft drinks. The premium membership gets a big plate, the dessert station, and the imported drinks. The kitchen is the same. The recipes are the same. The waiter is the same. What changes is how much each member can put on the plate and which stations they can visit. If you gave everyone the big plate the restaurant would lose money on the basic members. If you gave everyone the small plate the premium members would be paying for things they cannot use. The agent works the same way. The persona is the kitchen. The plate size and the dessert station are the tenant-tuned context knobs.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
When the same agent serves a hobbyist on a free plan and a Fortune 500 customer on an enterprise contract, the per call economics are very different. The free user generates near-zero revenue; the enterprise contract may pay six figures a year. If you assemble the same context for both, you either lose money on the free tier or you give the enterprise tier less than they paid for.
Cost gating is the discipline of letting the tenant's plan flow into the context-assembly pipeline as a budget envelope. Persona, safety and the tool registry stay the same. Retrieval volume, summary depth, model class and working context size all scale with the plan.
In most retrieval-heavy LLM products, input tokens are the largest cost driver. They are spent at assembly time, not billing time. You cannot recover margin in invoicing if you have already spent it in tokens. The right place to defend unit economics is in the assembler.
What stays constant versus what scales
The first move in designing tier-aware context engineering is to decide what is policy (constant) and what is parameter (tier-scaled).
Constant across tiers: system prompt and persona, output schema, safety and policy rails, the tool registry. Behavior should be tier-independent. The agent introduces itself the same way to everyone, refuses the same out of scope requests, and follows the same brand voice. Tier-flavoring the persona ('cheap version is rude, premium version is friendly') destroys trust and inverts the conversion funnel. Your free-tier users are the ones you most want to persuade to pay.
Scaled with tier: working context budget, top-k retrieval size, reranker on/off, query expansion on/off, summary depth, model class, persistent memory horizon. Each of these has a direct cost lever. Doubling top-k roughly doubles the retrieved tokens. Turning on a commercial reranker adds a per-query cost on top of vector search. Model class is a large multiplier too: at list prices, Claude Sonnet 4.6 to Opus 4.7 is about 1.7x per input token, and Haiku 4.5 to Opus 4.7 is 5x. The total cost per call envelope is the multiplication of all these factors. The right envelope for a tier is whatever keeps the per user gross margin positive.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Knob | Free tier | Pro tier | Enterprise tier |
|---|---|---|---|
| Working context budget | 4-8k tokens | 16-24k tokens | 32-64k tokens |
| Top-k retrieval | 3-5 | 8-12 | 15-25 |
| Reranker | Off | Local / cheap | Cohere rerank-v4.0 / Voyage rerank-3 |
| Summary depth | Single rolling | 2-level | 3-level + semantic |
| Model class | Small model (e.g. Haiku 4.5) | Mid-tier model (e.g. Sonnet 4.6) | Flagship model |
| Persistent memory | Session-only | Week-long | Persistent with admin controls |
Real products, models, and research that use this idea.
- ChatGPT's Free, Plus and Pro plans differ in model access and usage limits while the assistant's behavior stays the same, which is the same constant-persona, scaled-budget split.
- Hosted rerank APIs such as Cohere rerank-v4.0 are billed per search on top of vector retrieval, which is why many products enable reranking only on paid tiers.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you keep the free tier's quality high enough to drive conversions while staying profitable?
Invest in retrieval precision rather than retrieval volume. A free tier with top-k 3 and a strong reranker can outperform a top-k 10 with no rerank, at lower cost. The unit economics get better as the retrieval quality goes up, not as the budget goes up.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Running the same context policy for every tenant and trying to recover margin with billing math. The fix is at the assembly layer, not the invoice.
60 second bullets to scan on the way to the call.
Which knobs stay constant across tenant tiers and which scale
Why margin must be defended at the context-assembly layer rather than in billing
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.