Fresher / New Grad

Weight quantization in plain terms: what changes and what stays the same?

TTFT: name the metric it captures and what dominates it

Walk through how top_p (nucleus) sampling truncates the distribution

Compare top_k and top_p as truncation strategies for sampling

In benchmarks, tokens per second measures what exactly?

Bytes, characters, tokens: which one gets billed and how do they relate?

Throughput, for an LLM serving system: give the working definition

What is a tensor core and why does it dominate matmul throughput on GPUs?

How does the temperature parameter reshape the sampling distribution?

Define a stop sequence in an LLM API call

Server-Sent Events streaming: define it and explain why chat APIs default to it

Sequence length vs context window, what is the practical distinction and which one drives KV cache size?

The roofline model in plain language, without the chart, what is it saying?

TPM and RPM each throttle a different axis, which one caps which?

Prompt caching shows up on Anthropic and OpenAI pricing pages, what does it actually do?

Walk through what actually happens during the prefill phase of an LLM forward pass.

NVLink vs PCIe: match each interconnect to its bandwidth tier in a modern multi-GPU server.

A MIG slice on an H100, what is it and what is it good for?

What unit measures GPU memory bandwidth and why does that number cap decode speed?

max_tokens vs stop: pair each parameter with the cap it enforces on the completion.

Define LLM inference and explain how it differs from training.

End-to-end latency: fill in the queue, prefill and decode components.

Why do hosted LLM APIs charge separate per-million rates for input and output tokens?

Identify the NVIDIA H100: what generation, what memory, what tensor-core formats?

Pair each decoding strategy with its determinism and diversity profile.