LLMOps Engineer

Given draft acceptance rate α and target/draft cost ratio c, when does speculative decoding actually win?

Predict whether speculative decoding wins for given α, K, c

What is RadixAttention in SGLang and how does it differ from vLLM's prefix sharing?

Which workload most benefits from RadixAttention specifically vs simpler per-request prefix caching?

When would you choose vLLM, TensorRT-LLM, SGLang, or TGI for a production serving deployment?

Match each production serving framework to its defining strength

Apply the roofline model to LLM inference and identify where decode and prefill sit on it.

On the roofline, which intervention moves an LLM decode workload most directly toward higher peak throughput?

Given a TPM rate limit, how do you compute supportable concurrent users for a feature?

Predict supportable concurrent-user count under a 200k TPM rate limit

Match each post-training quantization method to what it protects against

When would you choose QAT over PTQ for inference quantization and what's the cost?

Which scenario most justifies the extra cost of QAT over PTQ for inference quantization?

How do Anthropic and OpenAI prompt-cache pricing models work and what counts as a cache hit?

Which prompt-construction pattern is most likely to actually hit the provider's prompt cache?

Why does prefill saturate compute while decode is bottlenecked on memory bandwidth?

Which statement best explains why prefill and decode hit different bottlenecks on the same GPU?

Why does prefill's O(seq^2) attention cost saturate compute rather than bandwidth?

Decompose the cost of a single API call into its components and explain which dominates.

Predict the total cost of one Claude Sonnet API call with realistic token counts

Compare tensor, pipeline and expert parallelism for inference serving.

Match each multi-GPU parallelism strategy to its defining property

What problem does PagedAttention solve and why does block-based allocation enable larger batches?

What is the primary mechanism by which PagedAttention increases serving throughput?

How do online (interactive) and offline (batch) inference workloads differ in optimization target?