Predict the hardware cost vs gross margin for serving $15/M output tokens on H100

**Unit economics** for token serving comes down to one comparison: what it costs you to make a million output tokens, versus what you sell them for. The setup gives a rental price of $5 per hour for an H100 and a decode rate of 150 tokens per second. First, find the supply per hour. At 150 tokens per second, one hour of 3600 seconds produces 150 times 3600, which is 540,000 tokens. So a single H100 makes about 0.54 million tokens every hour it runs. Next, turn that into cost per million. You paid $5 for that hour and got 0.54 million tokens. Dividing $5 by 0.54 gives roughly $9.26 to produce one million output tokens. Now compare against the price. You sell at $15 per million, so your gross margin is 15 minus 9.26, about $5.74 per million. As a percentage that is 5.74 divided by 15, roughly 38 percent. The takeaway: the margin is healthy but not huge, and the fastest way to grow it is to make each GPU emit more tokens per second, not to hunt for slightly cheaper rentals.

**Cost per token** is set by how much GPU time you rent divided by how many tokens that time produces. Start with supply. At 150 tokens per second, one hour yields 150 times 3600, which is 540,000 tokens per H100. Cost per token follows directly: $$\text{cost/token} = \frac{\$/\text{hr}}{\text{tok/s} \cdot 3600}$$ Plugging in gives $5 divided by 540,000, about $9.26 per million tokens. You sell at $15 per million. Gross margin is 15 minus 9.26, roughly $5.74 per million. As a fraction that is 5.74 over 15, near 38 percent. The structural insight matters more than the digits. Cost per token scales inversely with throughput, so doubling tokens per second halves cost and roughly doubles margin at a fixed price. Rental price moves cost only linearly. That is why serving teams chase batching, speculative decoding, and quantization. Each one raises tokens per GPU-second, which compounds straight into the bottom line.