How effective is the soft directive 'be brief' as an output-length constraint compared to the hard max_tokens parameter? · Zenaique

Attempt it

Quick reference

TL;DR

Soft directives like 'be brief' are surprisingly weak (~30-50% non-compliance); max_tokens is hard but truncates mid-sentence. Combine both and eval the output-length distribution.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Picture asking a chatty friend to keep their answer short. They mean well, but they get carried away and ramble for a full minute anyway. Now picture setting a kitchen timer for 30 seconds; the timer cuts them off, but often mid-word. The answer is to do both: ask for brevity AND set the timer, with the timer set generously enough that they finish their sentence on normal questions but get cut off on truly runaway ones. Then check a recording of ten real conversations to see if the strategy is actually working.

Key concepts

Output length control max_tokens parameter Soft vs hard constraints Prompt evaluation

Detailed answer & concept explanation

~4 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

4 min: why soft directives leak + how max_tokens truncates + the soft+hard combo + structured output guarantee + the length-distribution eval pattern.

Real-world examples

Real products, models, and research that use this idea.

Voice assistants like Alexa and Google Assistant cap responses tightly because text to speech latency scales with length; they layer prompt directives with hard caps and tight evaluation.
GitHub Copilot Chat uses soft brevity directives in its system prompt and a generous max_tokens, then eval-checks response length against UX guardrails.
Customer-support copilots running on Claude Opus 4.7 and GPT-5.5 typically pair 'answer in under 3 sentences' with max_tokens around 300 to allow graceful endings.
Production teams running OpenAI Structured Outputs or Anthropic's tool-use schema avoid raw max_tokens entirely for JSON returns; the schema constrains decoding and guarantees a complete object.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy do instruction-tuned models systematically over-produce length even when told to be brief?

Sign in or go Pro to see the approach for this follow-up.

QHow would you guarantee a JSON response fits within a strict size budget?

Sign in or go Pro to see the approach for this follow-up.

QWhat's the right way to evaluate length compliance in production?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Relying on 'be brief' alone and being surprised when production outputs run 3x longer than asked; the directive shapes intent, not behavior, and 30-50% of responses ignore it.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.