Spot the errors in this tensor-core utilization claim

**Tensor cores** are fixed-function hardware blocks on the GPU that do one thing extremely fast: small matrix-multiply-accumulate operations. They are not general purpose. To engage them, your matmul dimensions must be aligned to specific multiples, and your data must be in a dtype the hardware generation actually supports. The passage gets four things wrong. First, you do not hit peak throughput at arbitrary shapes. Hopper wants dimensions that are multiples of 16 for FP16 and BF16, and 32 for FP8 and INT8. Misaligned shapes quietly fall back to a slower path. Second, tensor cores do not adapt to any shape or dtype. Each generation supports a fixed menu of combinations; anything off the menu drops to the slower CUDA cores. Third, padding a small matmul up to alignment is usually faster, not wasteful. The few extra operations from padding are far cheaper than the fallback slow path you avoid. Fourth, FP16 and INT8 do not share a peak. Lower precision means higher throughput: roughly INT8 doubles FP16, and FP8 doubles INT8 again on Hopper.

**Tensor cores** are fixed-function multiply-accumulate units, not flexible ALUs. They engage only for supported shape and dtype combinations. Everything else falls back to general CUDA cores at a fraction of the rate. The passage stacks four errors. Hopper needs dimensions aligned to multiples of 16 for FP16 and BF16, and 32 for FP8 and INT8. Misaligned shapes silently miss peak. Padding up to alignment is usually a win. The extra operations cost less than the fallback. Lower precision raises the peak, so INT8 roughly doubles FP16 and FP8 doubles INT8. They are not equal. The deeper trap is utilization. A common claim is that high GPU utilization in nvidia-smi means tensor cores are saturated. That is wrong. The busy counter reports whether a kernel is resident, not whether the math units do useful work. During decode each step is memory bound. The GPU streams the whole KV cache and weights from HBM for one token. Tensor cores sit idle waiting on bandwidth while the busy light reads high. The honest metric is achieved FLOP per second against the hardware peak, often expressed as MFU. Decode MFU is frequently under ten percent even when utilization shows ninety-plus.