The correct answer is B. Both A100 and H100 have hardware support for bf16, which is the right default for almost every LoRA recipe in 2026.
bf16 has the same dynamic range as fp32. That means gradients do not underflow to zero, so you do not need any loss-scaling tricks. It runs on tensor cores at full speed on both chips. The result is fast, stable training without extra plumbing.
fp16 also runs on tensor cores, but it has a narrow dynamic range. To stop gradients from vanishing, frameworks add a dynamic loss scaler that constantly adjusts a multiplier. It works, but it is one more moving part that can fail loudly on a tricky run.
fp8 is real and fast on H100, but it ships with caveats. The numerics are tight enough that untuned recipes hit outlier issues, optimizer state needs careful precision, and kernels need per-tensor scaling. That is an optimization you turn on after you have a working bf16 baseline, not a default.
**Option B** is right. bf16 is the default on both Ampere (A100) and Hopper (H100) because it gives you tensor-core throughput without any of the operational complexity that fp16 or fp8 add.
bf16 keeps fp32's exponent width, so its dynamic range matches fp32 and gradients do not underflow. That eliminates the loss-scaler dance fp16 requires: no dynamic scaling factor to manage, no skip-step logic when the scaler overshoots, no surprise NaNs after a learning-rate change. The recipe just works.
fp16 still runs on tensor cores but needs that loss scaler. It costs nothing in throughput, but it is one more thing that can break, and on a new recipe the failure mode is silent gradient underflow that looks like a stuck loss curve.
fp8 on H100 is the genuine optimization, and it can lift throughput meaningfully on the right workload. But it requires per-tensor scaling, careful treatment of outlier activations, optimizer state in higher precision, and framework support that has matured but is still recipe-dependent. The right pattern is: prove a bf16 baseline works end to end, then switch to fp8 with full numerics validation. Defaulting to fp8 cold is a way to spend a week chasing instabilities.
Quick reference
TL;DR
Both A100 and H100 default to bf16; it hits tensor cores, matches fp32 dynamic range, and needs no loss scaler. fp8 is an opt-in only after numerics are validated.
Picture two photocopiers, an older one and a newer one. Both can copy at the same reliable mid-quality setting that almost never jams. The newer one also has an experimental ultra-fast mode that sometimes smudges if the paper is unusual. The safe default for both is the reliable mid-quality setting, and the experimental mode on the new copier is something you only enable after you have run a stack through cleanly and checked every page. Picking the experimental mode just because the new copier supports it would mean troubleshooting jams instead of getting work done. The same logic holds for precision on training hardware: pick what works on both, switch up only after you have a stable baseline.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Debugging a brand-new recipe that is producing NaNs you cannot trace Drop to fp32 temporarily to rule out numerics; bf16 is the default but fp32 is the bisection tool.
Targeting maximum throughput on Hopper for a validated workload fp8 with Transformer Engine can lift training throughput meaningfully once the per-tensor scaling and loss-curve parity are confirmed.
Running on consumer GPUs that lack bf16 tensor cores Some older or budget cards force fp16; accept the loss-scaler complexity and verify it engaged before judging stability.
Training a model where activation outliers are known to cause fp8 instability Smoothing techniques or per-channel scaling may help, but staying in bf16 is often the faster path to a working baseline.
Quick scan · 2–4 min
Everything important, quickly.
5 min: name bf16 as the default on both chips, contrast with fp16 loss scaling and fp8 numerics, then sketch when fp8 is worth the upgrade and what validation it requires.
Side-by-side
Precision
A100 support
H100 support
Operational notes
fp32
Yes (no tensor cores)
Yes (no tensor cores)
Stable but ~2x slower; only for debugging
fp16
Tensor cores
Tensor cores
Needs dynamic loss scaling; narrow dynamic range
bf16
Tensor cores
Tensor cores
Matches fp32 range; no loss scaler; safe default
fp8 (E4M3 / E5M2)
No native support
Tensor cores
Per-tensor scaling, outlier handling; opt-in only
Real-world examples
Real products, models, and research that use this idea.
1Hugging Face TRL and Transformers default the SFTTrainer to bf16 on both A100 and H100 SKUs, with fp8 gated behind explicit Transformer Engine integration.
2Unsloth and Axolotl LoRA recipes ship bf16 as the standard precision flag across open models like Llama 4 Maverick, Qwen 3.5, and DeepSeek V4.
3NVIDIA's Transformer Engine documentation positions fp8 as an opt-in performance mode and ships a validation recipe that compares loss curves against a bf16 baseline.
4Cloud fine-tuning vendors such as Together and Fireworks expose bf16 by default and surface fp8 as a workload-specific accelerator with explicit numerics caveats.
5Practitioners report consistent throughput wins from fp8 on Hopper for pretraining-scale workloads, with the same recipe needing per-tensor scaling tuned per model family.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does bf16 avoid the dynamic loss scaler that fp16 needs?
Sign in or go Pro to see the approach for this follow-up.
QWhat changes when you move a validated bf16 recipe to fp8 on H100?
Sign in or go Pro to see the approach for this follow-up.
QWhy do you keep optimizer state in bf16 or fp32 even when forward and backward run in fp8?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Reaching for fp16 on A100 or fp8 on H100 as the obvious default. fp16 needs loss scaling and fp8 needs per-tensor calibration; bf16 sidesteps both on either chip.