H100

At batch 16 and 4k context with FP8 weights, which model still fits on one 80 GB H100?

Pick the H100 workload where spot pricing is genuinely safe to deploy on

Sketch a Llama 3.1 70B self-host versus hosted API break even on the back of an envelope

A MIG slice on an H100, what is it and what is it good for?

Identify the NVIDIA H100: what generation, what memory, what tensor core formats?

Same LoRA recipe on H100 vs A100: what precision should each team default to?

LoRA vs full FT on 7B + 10k examples: order of magnitude cost comparison

Apply the roofline model to LLM inference and identify where decode and prefill sit on it.

Walk through the byte accounting that proves a single batch decode step is bandwidth bound on H100.

Predict the bandwidth vs compute latency of a single Llama-70B decode step on H100

Predict the critical batch size where decode crosses from memory bound to compute bound on H100