How many few-shot examples is the right number, and what tradeoff governs that count?

Flashcard·Medium·4.0 · 0·~30s·Asked atAnthropic OpenAI Cohere

Attempt it

Each example you add improves task adherence (the model has more demonstrations of the pattern to follow) and costs tokens (both the example text and the proportional shrinkage of every other context slot). The curve flattens fast: 0 to 3 examples buys most of the lift, 3 to 6 buys diminishing returns, beyond 6 the cost-benefit usually inverts. The right count is the smallest number that demonstrates every distinct case you actually care about (one for each output shape, one for each edge case the model gets wrong without an example). Twelve is almost always too many, that is a sign the prompt is being used as a substitute for fine-tuning.

Your answer

Few-shot examples are demonstrations of the desired input-output pattern, placed in the prompt before the actual user input. They are a powerful tool because they show the model what to do without describing it. The tradeoff is straightforward in shape but easy to get wrong. **The lift curve flattens fast.** Going from zero examples to one typically buys a large jump in task adherence. One to three buys more. Three to six buys diminishing returns. Past six, on most tasks, you are barely moving the needle. **The cost curve does not flatten.** Each example costs tokens for the example itself plus tokens that other slots in the context (system prompt, retrieved chunks, conversation history) now cannot use. You pay full price for every example, on every call. **The right count is coverage-based, not volume-based.** Ask: what are the distinct output shapes I need to demonstrate? What edge cases does the model get wrong without an example? Use one example per case. If you have three distinct cases, three examples suffice. If you have twelve, you probably do not actually have twelve distinct cases, you have one case demonstrated twelve different ways. When you genuinely need twelve examples to cover the cases, that is a signal that you should fine-tune.

**Few-shot examples** trade input-token cost for task adherence, and the curve governing that trade is steeper at the start and flatter past 3-6 examples. The lift mechanism is in-context learning. Each demonstration gives the model a concrete pattern to imitate: format, tone, edge-case handling, output structure. The first example does most of the work because it converts an abstract instruction into a concrete shape. The second and third add coverage of variations the first did not show. Past that, additional examples mostly demonstrate cases similar to ones already covered. The cost mechanism is linear input-token billing. Each example costs the bytes of its prompt plus its expected output, and those bytes squeeze every other slot in context: shorter retrieval, shorter conversation history, less room for the user input. Twelve examples on a typical 500-token per example task is 6K tokens of overhead on every call, before retrieval or user input even shows up. The right framing is coverage-based. List the distinct output shapes you need to demonstrate. List the edge cases the model gets wrong without an example. Use one example per case. Three or four well-chosen examples usually outperform twelve that overlap. If you genuinely need a dozen distinct cases to cover the task, the prompt is doing the work fine-tuning is built for, and a fine-tuned model on the same base will give you better quality at lower per-call cost.

Quick reference

TL;DR

Few-shot lift flattens after 3-6 examples while token cost stays linear; the right count is the smallest set that covers every distinct output shape and known edge case.

Memory aid

Easy to grasp

Picture teaching a new hire how to format a report. Show them one good example and they get the rough shape. Show them three covering the variations they will see and they get it solidly. Show them twelve and you are now training, not onboarding. The fourth through twelfth example each adds less than the one before, but the time you spend reading them through together does not shrink. At some point you are no longer teaching the format. You are doing the new hire's first month of work for them, on the clock, in the meeting. A prompt with twelve few-shot examples has the same shape.

Key concepts

Context engineering Prompt engineering Fine-tuning

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

4 minutes: lift curve shape, cost shape, the squeeze effect, coverage-based sizing, the fine-tuning crossover, ordering matters, when caching changes the math.

Real-world examples

Real products, models, and research that use this idea.

OpenAI's 2026 prompt engineering guide for GPT-5.5 explicitly notes diminishing returns past 4-5 examples on most tasks and points to fine-tuning for richer pattern coverage.
Anthropic's Claude Opus 4.7 documentation recommends 2-4 well-chosen examples for structured-output tasks and prompt caching for the example block.
Hugging Face PEFT and the OpenAI fine-tuning API both target the 'twelve examples is too many' inflection point with LoRA workflows that need 100-500 labeled samples.
DSPy treats prompts including few-shot examples as compiled artifacts; the optimizer routinely selects 2-4 examples from a candidate pool, confirming the empirical knee.
Llama 4 Maverick instruction-tuned variants show the same shape: marginal lift past 4 examples on structured-output benchmarks like BIG-Bench Hard.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does example ordering affect output, and which example matters most?

QHow does prompt caching change the cost equation for few-shot?

QAt what dataset size does fine-tuning beat few-shot economically?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Adding examples until quality stops improving without checking whether each example covers a distinct case. Three carefully chosen examples that span the actual output shapes usually beats twelve that all demonstrate the same case slightly differently.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.