Qwen 3.5: when does running your own GPU beat paying the hosted API per token?

**The crossover is fixed-vs-variable cost.** The hosted API charges per token, strictly variable. You pay nothing when you serve nothing. Self-hosting rents the GPU by the hour, strictly fixed. You pay the same bill whether you serve 1 token or 1 billion in those hours. This creates a crossover point. At low traffic, the API is cheaper because the self-host bill is mostly paying for an idle GPU. At high traffic, self-host is cheaper because the fixed bill gets amortized across many tokens. **The deciding knob is GPU utilization.** A rented H100 at $2.50/hr costs $60/day regardless. If it serves 5M tokens that day, per-million cost is $12. If it serves 50M tokens, per-million cost is $1.20. The hosted Qwen API costs roughly $0.50-$1.00 per million tokens depending on tier. **Order of magnitude.** On Qwen 3.5 (a 32B-class model) running FP8 on one H100 with continuous batching, peak output is several hundred tokens/sec. At 50-70% sustained utilization across the day, you serve roughly 2-5M tokens/day. That's the break-even neighborhood, below it the API is cheaper; well above it self-host wins. **Don't forget ops cost.** Hosted API absorbs scaling, monitoring, weight upgrades, on-call, and reliability engineering. For a small team that overhead is often worth more than the per-token price gap.

**The structural fact** is that hosted APIs price tokens (variable) while self-hosting prices GPU-hours (fixed). The hosted bill scales with traffic; the self-host bill is constant per hour regardless of traffic. That structural mismatch creates a crossover point determined by how heavily you load the GPU. **GPU utilization** is the deciding knob. A rented H100 at $2.50/hr costs $60/day whether you serve 1 token or 1 billion tokens in that 24-hour window. At full saturation a 32B-class model in FP8 with continuous batching sustains a few hundred output tokens/sec per H100, call it 600 tok/s for a clean number. That's ~52M tokens/day if the GPU ran flat-out 24/7. Real utilization is much lower. Default unoptimized serving sees 20-30% sustained utilization across a day (idle nights, low off-peak hours, request gaps between bursts). A well-tuned batch server with steady traffic can sustain 50-70%, occasionally 80%+. Above that you start hitting head-of-line blocking and queue growth. **Break-even.** Per-million cost on a $60/day GPU = $60 / (daily tokens / 1M). To beat a hosted Qwen API at $0.50/M, you need daily tokens > 120M, far above what one H100 can sustain. To beat $1.00/M you need 60M tokens/day. At realistic utilization of 50-70%, an H100 serves 25-35M tokens/day, between the $1.00/M and $0.50/M break-evens. Self-hosting is cost-competitive in the upper-middle of that range and clearly wins at very high sustained volume. **Ops cost.** Hosted absorbs scaling, failovers, weight version management, multi-region availability, on-call coverage, and the ongoing eng investment to keep all of that running. For a small team the ops cost can dominate, easily 3-5 cents per million tokens worth of engineer-time. That tilts the call further toward hosted for any traffic level where the compute-cost gap is narrow.