Flashcard

A MIG slice on an H100, what is it and what is it good for?

What unit measures GPU memory bandwidth and why does that number cap decode speed?

Define LLM inference and explain how it differs from training.

Why do hosted LLM APIs charge separate per-million rates for input and output tokens?

Identify the NVIDIA H100: what generation, what memory, what tensor-core formats?

Where does FP8 fit in modern LLM serving and which GPUs support it?

What is the EOS token and how does the runtime use it?

Describe what happens during the decode phase of LLM inference.

What is the context window of an LLM?

Walk through the shape of an OpenAI-compatible chat completions request and response.

Identify the B200 and GB200 NVL72 in NVIDIA's Blackwell lineup.

Describe what 'autoregressive' means for LLM generation.

What does arithmetic intensity actually measure and what does AI near 1 imply?

Define activation quantization and explain how W8A8 differs from weight-only quant.

Where does the NVIDIA A100 sit on the inference hardware ladder?

Recognize vLLM and the optimization that made it famous

Spell out TensorRT-LLM and pinpoint its production differentiator

Name TGI's maintainer and its niche among serving stacks

Identify PagedAttention and the project it ships with

Expand MQA and state its KV-cache cost tradeoff

Define GQA in transformer attention

Tulu instruction-tuning family: what is it and who publishes it?

target_modules in a PEFT config: what does this list control?

Step versus epoch in a training run: define the relationship.

Define supervised fine-tuning (SFT) in one breath.