How effective is the soft directive 'be brief' as an output-length constraint compared to the hard max_tokens parameter?

**Two tools** control how long an LLM's reply is. Each has a problem if you use it alone. The **soft directive** is a sentence in the prompt: 'answer in under 100 words', 'be brief', 'one sentence only'. These work some of the time, but modern models are trained to be helpful, and 'helpful' often means 'thorough'. When the model thinks the question deserves a longer answer, it just writes one and ignores the directive. Production data typically shows 30-50% non-compliance on length asks. The **hard cap** is the `max_tokens` parameter on the API call. Whatever you set, the model literally cannot exceed it. The problem is the cutoff is mechanical: when the token count hits the limit, the model stops, even if it is mid-word. So a too-tight `max_tokens` produces responses that look broken. The pattern that works is both at once. Use a soft directive to shape the model toward the target length. Use max_tokens as a generous safety cap so runaway generations are bounded without truncating normal ones mid-sentence. Then eval the actual output-length distribution to confirm it lands where you want.

**Soft directives are surprisingly weak.** 'Be brief' or 'answer in under 100 words' gets ignored 30-50% of the time on production data, because modern instruction-tuned models bias toward thoroughness. The directive shapes intent; the model still decides. **max_tokens is a hard cap** that the model physically cannot exceed. It works, but it truncates mechanically: when the cap is reached, generation stops, even mid-word or mid-JSON-object. Set max_tokens too low and you ship responses that look broken. The production answer is to combine both. Use a soft directive ('answer in under 100 words') to steer the model's natural length toward the target. Set max_tokens generously above that target (say 200 for a 100-word ask) so it only triggers on true runaways. Then eval the actual output-length distribution on a real sample of traffic. If the p95 is bumping against max_tokens, the directive is being ignored more than you think and you need a stricter prompt or a different model. If outputs are clustering well below the cap, the cap is doing its job as a safety net and the directive is steering. The combination is the discipline. Neither tool is enough alone.