What's the highest leverage cost optimization for a production LLM app spending $10k+/month on tokens, BEFORE prompt level token trimming?

The biggest cost lever for most LLM (large language model) apps is not how many tokens (word pieces) you send. It is which model you send them to. Flagship models are often five to fifteen times more expensive per token than the small models in the same family, such as Claude Haiku or GPT-5-mini. Many tasks in a production app do not need a flagship. Examples are simple classification, short extraction, routine generation, and summarization of clean text. A small model handles these at the same quality you need. The routing pattern: classify each incoming request by complexity, send simple ones to the cheap tier, escalate to the flagship only when the task needs it. When most traffic is simple, this can cut cost by fifty to eighty percent with acceptable quality. Prompt trimming usually buys ten to thirty percent at best, because the per token price stays the same. Start with the price tier, then come back to the prompt. That is option B. A (trimming) gives a smaller gain. C (switching language) is impractical and gains little. D (lower max_tokens) only caps the longest outputs.

The answer is B, model routing. Cost optimization in a production LLM app has a clear order of operations: model routing first, prompt caching second, trimming and decoding params third. Token-by-token trimming comes last. Cost is multiplicative. A flagship model often costs five to fifteen times more per token than its smaller sibling. If you can route half your traffic to the cheap tier, you cut total spend by close to half before you touch a single word of the prompt. Compare that to prompt trimming, which works inside a single price tier and typically buys ten to thirty percent. The routing layer is itself a small engineering surface. A lightweight classifier or even a heuristic (length, format, presence of code) decides the tier. Hard cases escalate; easy cases stay cheap. Production stacks add a confidence check: if the cheap tier flags low confidence, the system retries the request on the flagship. The economics still win because escalation is rare. Once routing is in place, the prompt trimming and cache hit work become real and worth doing. But the first dollars come from the price tier, not the prompt length. Option A, trimming, works inside one price tier, so its gain is smaller. Option C, switching to a more token dense language, is impractical and the gain is bounded. Option D, lowering max_tokens, caps only the longest outputs and leaves typical output length unchanged.