Qwen 3.5 is available both via hosted APIs (Alibaba Cloud, OpenRouter, Together) and as open weights you can self-serve. Explain the cost crossover: name the dominant fixed-vs-variable factor, the utilization knob that decides which side wins, and give an order-of-magnitude break-even in tokens per day.
Free · 2 AI evals / day
**The crossover is fixed-vs-variable cost.** The hosted API charges per token, strictly variable. You pay nothing when you serve nothing. Self-hosting rents the GPU by the hour, strictly fixed. You pay the same bill whether you serve 1 token or 1 billion in those hours.
This creates a crossover point. At low traffic, the API is cheaper because the self-host bill is mostly paying for an idle GPU. At high traffic, self-host is cheaper because the fixed bill gets amortized across many tokens.
**The deciding knob is GPU utilization.** A rented H100 at $2.50/hr costs $60/day regardless. If it serves 5M tokens that day, per-million cost is $12. If it serves 50M tokens, per-million cost is $1.20. The hosted Qwen API costs roughly $0.50-$1.00 per million tokens depending on tier.
**Order of magnitude.** On Qwen 3.5 (a 32B-class model) running FP8 on one H100 with continuous batching, peak output is several hundred tokens/sec. At 50-70% sustained utilization across the day, you serve roughly 2-5M tokens/day. That's the break-even neighborhood, below it the API is cheaper; well above it self-host wins.
**Don't forget ops cost.** Hosted API absorbs scaling, monitoring, weight upgrades, on-call, and reliability engineering. For a small team that overhead is often worth more than the per-token price gap.
**The structural fact** is that hosted APIs price tokens (variable) while self-hosting prices GPU-hours (fixed). The hosted bill scales with traffic; the self-host bill is constant per hour regardless of traffic. That structural mismatch creates a crossover point determined by how heavily you load the GPU.
**GPU utilization** is the deciding knob. A rented H100 at $2.50/hr costs $60/day whether you serve 1 token or 1 billion tokens in that 24-hour window. At full saturation a 32B-class model in FP8 with continuous batching sustains a few hundred output tokens/sec per H100, call it 600 tok/s for a clean number. That's ~52M tokens/day if the GPU ran flat-out 24/7.
Real utilization is much lower. Default unoptimized serving sees 20-30% sustained utilization across a day (idle nights, low off-peak hours, request gaps between bursts). A well-tuned batch server with steady traffic can sustain 50-70%, occasionally 80%+. Above that you start hitting head-of-line blocking and queue growth.
**Break-even.** Per-million cost on a $60/day GPU = $60 / (daily tokens / 1M). To beat a hosted Qwen API at $0.50/M, you need daily tokens > 120M, far above what one H100 can sustain. To beat $1.00/M you need 60M tokens/day. At realistic utilization of 50-70%, an H100 serves 25-35M tokens/day, between the $1.00/M and $0.50/M break-evens. Self-hosting is cost-competitive in the upper-middle of that range and clearly wins at very high sustained volume.
**Ops cost.** Hosted absorbs scaling, failovers, weight version management, multi-region availability, on-call coverage, and the ongoing eng investment to keep all of that running. For a small team the ops cost can dominate, easily 3-5 cents per million tokens worth of engineer-time. That tilts the call further toward hosted for any traffic level where the compute-cost gap is narrow.
Picture two ways to get drinks at a wedding: pay $5 per glass at the bar, or rent your own bartender for $1000 a day. If only 10 people show up and each has 2 drinks, the bar is way cheaper at $100. If 500 people show up and each has 4 drinks, the bartender is way cheaper because you've spread their flat fee across 2,000 drinks. The crossover is somewhere around 200 drinks. The bartender also has hidden costs: you have to manage their schedule, find a backup if they get sick, and make sure they show up. For a small wedding the convenience of the bar might be worth a slightly higher per-drink price.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Very low or bursty traffic with no sustained volume Self-host's fixed cost can't be amortized; the GPU bills its full hour whether you serve 0 or 5,000 tokens.
Workloads that need a frontier model not available as open weights Self-host isn't an option for closed-weight frontier models (GPT-5, Claude Opus); hosted is the only choice.
Compliance or data residency requirements only the hosted provider can satisfy Self-hosting on your infrastructure can be the only path for HIPAA, SOC 2 Type II in some configurations, or sovereignty regulations, flipping the question entirely.
Teams without GPU-serving engineering depth Self-hosting at production quality is non-trivial, vLLM tuning, autoscaling, KV-cache management, monitoring. Hosted absorbs all of this for the same compute cost.
Quick scan · 2–4 min
Everything important, quickly.
3 min: fixed-vs-variable cost framing + utilization as the deciding knob + the break-even math with realistic numbers + the role of ops cost + when spot pricing and multi-GPU change the picture.
Real-world examples
Real products, models, and research that use this idea.
1Together AI, Fireworks, and Alibaba Cloud's Bailian platform all host Qwen 3.5 at roughly $0.40-$0.60/M tokens for the 32B tier in 2026.
2Cursor and Anysphere migrated heavy code-completion traffic to self-hosted models when sustained utilization crossed the break-even threshold.
3Replicate and RunPod expose H100 instances at $2-3/hr; the math in this question matches their published serving guides.
4OpenRouter aggregates multiple Qwen 3.5 providers and lets teams A/B test hosted vs DIY economics on real traffic before committing.
5Many startups in 2025-2026 publicly reported staying on hosted Qwen/Llama APIs through Series A, switching to self-host only after sustained 10M+ token/day volume.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does spot pricing on the H100 change the break-even?
Sign in or go Pro to see the approach for this follow-up.
QWhy is the H100's per-million-token cost a hyperbola, not a line, in tokens-per-day?
Sign in or go Pro to see the approach for this follow-up.
QWhen does the multi-region availability story flip the calculus?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Computing self-host cost using peak token-rate as if the GPU runs at 100% for 24 hours. Real production utilization is 20-30% by default and 50-70% on well-tuned batch servers; using peak makes self-host look 3-5x more attractive than it actually is.