MIG (Multi-Instance GPU) statically partitions a single A100 or H100 into up to seven hardware-isolated slices, each with its own dedicated SMs, memory, L2 cache slice, and memory bandwidth quota. It is used to safely co-host smaller models, dev workloads, or per-tenant inference on one physical card without one tenant's traffic starving another's.
**MIG** stands for Multi-Instance GPU. It is a feature on NVIDIA A100 and H100 datacenter GPUs that lets you split one physical card into up to seven smaller logical GPUs. Each smaller GPU, called a MIG instance or 'slice', looks to software like its own dedicated GPU.
What makes MIG different from older sharing techniques is that the isolation is enforced in hardware. Each slice gets its own set of streaming multiprocessors (SMs), its own dedicated chunk of HBM memory, its own slice of the L2 cache, and its own portion of the memory bandwidth. If you give tenant A two-sevenths of the card and tenant B one-seventh, there is no way for tenant A's workload to spill over and slow down tenant B. The walls are physical.
You would use MIG when you want to run multiple smaller workloads on one large GPU and absolutely cannot have them interfere with each other. Common cases include hosting several small models on the same card for cost reasons, giving each developer in a team their own GPU slice for experimentation, or running per-tenant inference where each customer is guaranteed a fixed slice of capacity regardless of what other customers are doing.
**MIG** (Multi-Instance GPU) is NVIDIA's hardware-level GPU partitioning, available on A100, H100, and successor data-center cards. It splits a single physical GPU into up to seven independent instances, each with its own dedicated SMs, HBM memory partition, L2 cache slice, and memory bandwidth quota.
The partition is configured statically before workloads start. A 7-slice A100, for instance, gives each slice roughly 5 GB of HBM and one-seventh of the SMs and bandwidth. Different profiles exist (1g.5gb, 2g.10gb, 3g.20gb, 4g.20gb, 7g.40gb on A100; analogous profiles on H100), letting you mix slice sizes if the workloads have different needs. Once configured, each slice appears as its own CUDA device with a separate UUID, separate driver context, and complete fault isolation from siblings.
MIG is the right tool when you need strict performance and security isolation between workloads sharing one GPU. Multi-tenant inference platforms use it so a noisy tenant cannot starve a quiet one. ML platform teams use it to carve up expensive H100s into many smaller dev environments. CI systems use it to run many short jobs in parallel on one card.
The tradeoff is rigidity. The partition is static, so you cannot scale a slice up at runtime if it runs hot. Switching profiles requires draining the GPU. MIG also limits some features: NVLink between slices is disabled, and tensor parallelism across slices on the same card does not work. It is genuinely for parallel-tenant work, not for one big distributed job that needs all the silicon together.
Quick reference
TL;DR
MIG is NVIDIA's hardware partitioning feature that splits an A100 or H100 into up to seven isolated GPU instances, each with guaranteed SMs, memory, and bandwidth.
Imagine you rent a giant warehouse to three small businesses. If you just throw them all inside, they will fight over forklifts and parking spots. Instead, you build solid walls inside the warehouse, run a separate door and parking lot per tenant, and meter the loading dock so each business gets a guaranteed share. Now they cannot starve each other no matter what. MIG does this for a GPU. It builds walls inside the chip, gives each tenant a slice of compute units, a slice of memory, and a slice of bandwidth, so a noisy neighbor cannot eat your performance. The walls are real silicon partitions, not software polite-sharing rules.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Serving a single large model that needs the full GPU MIG isolation overhead and the loss of cross-slice NVLink hurt; the workload wants the entire card, not a slice.
Running tensor-parallel inference across multiple slices on the same card Cross-slice NVLink is disabled, so tensor parallelism cannot communicate between slices on the same physical GPU.
Workloads with bursty memory needs that exceed a single slice Slice memory is fixed at configuration time; bursting requires repartitioning, which requires draining the GPU.
Quick scan · 2–4 min
Everything important, quickly.
3 min: hardware partition vs cooperative sharing + seven-slice cap + dedicated SMs/memory/bandwidth per slice + static configuration + multi-tenant isolation use cases + the no cross-slice NVLink limitation.
Side-by-side
Sharing mode
Isolation
Setup
Best for
MIG
Hardware-enforced (SMs, memory, BW)
Static partition, requires reconfig
Multi-tenant inference, strict SLAs
CUDA MPS
Cooperative, no quotas
Runtime, on demand
Many small jobs from one team
Time-slicing
None, full context switch
Default behavior
Sequential workloads on idle GPU
Full GPU
N/A (sole tenant)
Default
Large models, training, tensor parallelism
Real-world examples
Real products, models, and research that use this idea.
1AWS, Google Cloud, and Azure all expose MIG-partitioned A100 and H100 instances as smaller, cheaper GPU SKUs (e.g., AWS g6e with a MIG-sliced H100).
2Hugging Face Inference Endpoints uses MIG to serve many small models on shared H100s with strict per-customer isolation.
3RunPod, Lambda, and other GPU clouds offer 'fractional GPU' tiers that are typically MIG slices of an A100 or H100.
4Internal ML platforms at large companies use MIG to give every data scientist a guaranteed quarter-GPU dev environment on shared H100 fleets.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is MIG limited to seven slices on an A100 or H100?
Sign in or go Pro to see the approach for this follow-up.
QIf I split an H100 into seven 1-slice instances, do I get one-seventh of the compute and bandwidth per slice?
Sign in or go Pro to see the approach for this follow-up.
QHow does MIG interact with Kubernetes GPU scheduling?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Thinking MIG is the same as CUDA MPS or time-slicing. MIG is a hardware partition with strict isolation; MPS shares resources cooperatively with no enforced quotas.