Pick the H100 workload where spot pricing is genuinely safe to deploy on

MCQ·Medium·4.0 · 0·~1 min·Relevant atDatabricks NVIDIA

Attempt it

Cloud providers sell GPU capacity in two main flavours. **On-demand** instances are guaranteed for as long as you pay; spot instances cost much less but can be reclaimed by the provider at short notice when someone else wants on-demand capacity. For H100 GPUs in 2026, spot pricing typically runs around 1.50 to 2.50 dollars per hour, compared to 3 to 5 dollars per hour on-demand depending on cloud and region. That is a real 40 to 70 percent discount. The catch is the eviction. The provider can give you anywhere from a few seconds to a few minutes of warning, then take the GPU back. If your workload was serving a live customer in that moment, the request fails and the user sees an error. The rule of thumb is simple. Spot is safe when (a) the job is stateless or can resume from a checkpoint, and (b) nobody is waiting in real time. Overnight batch evals, dataset labeling, offline embedding generation, and training jobs with frequent checkpoints all fit. Live chat serving, per-user pinned sessions, and launch-window traffic do not. Option B in this question is the textbook fit: it can resume from a checkpoint and nobody is watching a screen waiting for it.

**Spot H100s** cost roughly half of on-demand, around 1.50 to 2.50 dollars per hour versus 3 to 5 dollars, in exchange for the cloud's right to preempt the instance with seconds to minutes of warning. That discount only pays off for workloads that can absorb a preemption without anyone noticing. Two conditions matter: the job is stateless or checkpoint-resumable, and there is no live latency budget. Option B, a nightly batch eval that resumes from a checkpoint, hits both. The eviction just costs a few minutes of wasted work on the next launch. The other three options each fail one or both checks. Option A puts the sole replica of a customer-facing chat behind a live SLO; a single preemption is an outage. Option C pins per-user LoRA state and active sessions to the GPU; eviction drops conversations mid-token. Option D commits to 99.9 percent availability during a launch window where preemption probability blows the target without a pre-warmed on-demand failover. The broader principle: spot pricing is a batch and offline tool. The moment a workload carries either a live SLO or per-request state that cannot be re-hydrated quickly, it belongs on on-demand, reserved, or provisioned-throughput capacity. Real deployments often use both, with spot soaking up offline work and on-demand carrying the user-facing fleet.

Quick reference

TL;DR

Spot H100 capacity is roughly half on-demand price but can be preempted with seconds of notice, so it fits only stateless, latency-tolerant, checkpoint-resumable batch jobs.

Memory aid

Easy to grasp

Imagine a parking lot that offers half-price spots, but the manager can ask you to leave in two minutes if a full-price customer shows up. If you are running into the store for a quick errand and your car is empty, that deal is fine. If you are mid-surgery in an ambulance parked there, losing the spot is a disaster. Spot GPU pricing works the same way. Cloud providers sell unused H100 capacity at a steep discount with the right to reclaim it at any moment. Batch jobs that can pause and pick up from a saved point handle the eviction gracefully. Live customer traffic does not, because every preempted second is a user staring at a blank screen.

Key concepts

Spot pricing Availability SLOs Checkpoint and resume

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

4 min: spot economics on H100 + the two conditions for spot-safety + why each wrong option fails + mixed-fleet architectural patterns + batch API monetisation.

Real-world examples

Real products, models, and research that use this idea.

AWS EC2 p5 (H100) spot pricing sits around 50 to 70 percent below on-demand, used widely for overnight training and batch eval at large labs in 2026.
Lambda Labs and CoreWeave offer interruptible H100 capacity primarily for fine-tuning and dataset labelling workloads with built-in checkpointing.
Anthropic and OpenAI batch APIs run on heavily discounted internal spot tiers, exposing a 24-hour SLA that hides eviction recovery from callers.
DeepSeek and Mistral training runs on spot H100 clusters routinely, using frequent checkpoint-and-restart loops to absorb preemptions.
Production serving fleets at scale (chat, copilots) stay on on-demand or provisioned-throughput; spot only ever appears as a clearly separate batch lane.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QCould you serve customer traffic on spot at all, given a robust failover layer?

QHow does the batch API product from major LLM vendors relate to spot capacity?

QWhy is checkpoint frequency a knob worth tuning on spot training?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating spot as a generic cost lever and pointing it at customer-facing traffic. The discount is real, but the preemption probability eats the SLO long before the savings show up on the bill.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.