FLOPs vs FLOP/s: fill in the counts versus rate distinction.

**FLOPs** with a lowercase s at the end is a count: the number of floating-point operations a piece of work requires. It has no time component. A 70B model forward pass on one token costs about 140 billion FLOPs. **FLOP/s**, with the slash, is a rate: floating-point operations per second. It describes how fast a chip can compute. An H100 GPU has a peak of about 990 trillion FLOP/s in BF16. That is its compute throughput, not its job size. The two come together when you ask how long a job will take. Time equals FLOPs divided by FLOP/s, in theory. In practice the answer is often higher because the GPU spends time waiting on memory, not actually computing. For LLM inference, the prefill phase processes the whole prompt in one parallel pass and does a lot of math per byte read, so it tends to be compute-bound. The decode phase produces one token at a time and reads the entire model weights for each step, so it tends to be bandwidth-bound. Knowing which phase is which tells you where to spend optimisation effort.

**FLOPs** is the plural-noun count of floating-point operations. **FLOP/s** is the per-second rate. The lowercase s in FLOPs is the plural; the slash in FLOP/s is the division by time. The naming is unfortunate but the distinction is critical. When you say a Llama 3.1 70B forward pass costs about 140 GFLOPs per token, you are stating a job size. When you say an H100 SXM runs at 990 TFLOP/s in BF16, you are stating a hardware throughput. Divide one by the other and you get a theoretical wall-clock estimate. 140 GFLOPs divided by 990 TFLOP/s gives roughly 0.14 milliseconds for the compute alone. But that assumes the data is in cache. In reality the GPU must read 140 GB of weights from HBM, which takes about 42 ms on the chip's 3.35 TB/s of bandwidth. The compute finishes in a millisecond; the memory read takes 40+ ms. The kernel is bandwidth-bound. This is the prefill versus decode split. Prefill processes hundreds of tokens in one matmul, amortising the weight read across many FLOPs, and lands compute-bound. Decode at batch 1 does one token per matmul, paying the full weight read for very little arithmetic, and lands bandwidth-bound. The arithmetic intensity, FLOPs per byte, is what determines which side of that boundary you sit on.