Spot H100 capacity is roughly half on-demand price but can be preempted with seconds of notice, so it fits only stateless, latency-tolerant, checkpoint-resumable batch jobs.
Imagine a parking lot that offers half-price spots, but the manager can ask you to leave in two minutes if a full-price customer shows up. If you are running into the store for a quick errand and your car is empty, that deal is fine. If you are mid-surgery in an ambulance parked there, losing the spot is a disaster. Spot GPU pricing works the same way. Cloud providers sell unused H100 capacity at a steep discount with the right to reclaim it at any moment. Batch jobs that can pause and pick up from a saved point handle the eviction gracefully. Live customer traffic does not, because every preempted second is a user staring at a blank screen.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Spot pricing is one of the largest cost levers in cloud GPU economics, often quoted as 50 to 70 percent off on-demand. For H100 capacity in 2026 that is the difference between roughly 1.50 dollars an hour and 4 dollars an hour. The question every serving team eventually asks is which workloads can ride that discount safely, and which absolutely cannot.
The answer turns on a small number of operational facts about how spot capacity works, what eviction actually looks like in practice, and which classes of inference workload have the ability to absorb a preemption without anyone noticing. The four MCQ options span the spectrum from clearly safe (overnight batch with checkpointing) to clearly catastrophic (sole replica behind a live SLO), and the reasoning that distinguishes them generalises to any decision about where to deploy a GPU workload.
This deep dive walks through the spot pricing model, the eviction contract, the two-question test for spot-safety, the architectural patterns for mixing tiers, and the specific failure modes that snare teams who treat spot as a generic cost knob. The goal by the end is to be able to look at a workload description, classify it correctly in under a minute, and explain the reasoning to a finance partner who only sees the dollars-per-hour difference.
What spot capacity actually is
Spot capacity is unused on-demand capacity that the cloud provider sells at a discount with the right to reclaim. The provider's incentive is to keep utilisation high; the customer's incentive is to pay less. Both sides accept that the relationship can be torn up at the provider's discretion.
For H100 in 2026 the discount is typically 50 to 70 percent. AWS p5 spot instances trade around 8 to 15 dollars per hour for an 8-GPU node (so ~1 to 2 dollars per H100), compared with 25 to 40 dollars per hour on-demand. GCP A3 and Lambda 1-Click follow similar patterns. The exact ratio shifts hourly with regional supply and demand; during major launches or training waves, spot can dry up entirely.
The legal contract is uniform across providers. The instance is yours until the provider sends an eviction notice, then you have a window (typically two minutes on AWS, 30 seconds on GCP) to wind down before the underlying VM is terminated. The notice arrives via instance metadata or a cloud event; the application is responsible for catching it and draining cleanly. After the window closes the GPU is gone.
That contract makes spot a fundamentally batch-shaped product. Any workload that cannot complete or hand off cleanly within the eviction window pays a penalty proportional to how much state was in flight. Batch jobs with checkpoints lose minutes; live serving requests lose users.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- AWS EC2 p5 (H100) spot pricing sits around 50 to 70 percent below on-demand, used widely for overnight training and batch eval at large labs in 2026.
- Lambda Labs and CoreWeave offer interruptible H100 capacity primarily for fine-tuning and dataset labelling workloads with built-in checkpointing.
What an interviewer would ask next. Try answering before peeking at the approach.
QCould you serve customer traffic on spot at all, given a robust failover layer?
Yes, but only as elastic surge above an on-demand floor sized for the SLO. The pattern is: on-demand carries the steady-state and SLO commitment, spot replicas register into the load balancer when capacity is available, and an eviction handler drains in-flight requests before the instance dies. Most teams find the engineering cost only pays back at very large fleets.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating spot as a generic cost lever and pointing it at customer-facing traffic. The discount is real, but the preemption probability eats the SLO long before the savings show up on the bill.
60 second bullets to scan on the way to the call.
Rough spot versus on-demand price ratio for H100 in 2026
The two conditions a workload must meet to be spot-safe
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.