Short Answer

Memory creeps up for hours then OOMs at 3am: diagnose the KV-cache leak pattern

Describe why structured-output mode quietly raises the per-call token bill

Name the dominant cost driver of GPT-5.5 latency at a 200k-token input

What does the first forward pass set up that the second one doesn't pay for?

Why can a single high-res image cost more input tokens than a multi-paragraph prompt?

Walk through how tensor parallelism shards one matmul across GPUs and what it adds to decode step time

Prefix-cache hit lands on a request: does speculative decoding still help that request?

Why can a 50-token reply from a reasoning model still bill you for thousands of tokens?

When does splitting prefill and decode onto separate replica pools actually pay off in production?

Does asking for top-k logprobs add real compute cost per token, or only bytes on the wire?

Greedy decoding and temperature sampling: does one cost more per token than the other?

Walk through how CUDA graphs reshape per-step decode latency and when the win disappears

Why does a reasoning model like DeepSeek V4 quietly inflate the output-token bill?

Trace how one slow request stalls every co-batched generation step.

Does Anthropic's message_start event count toward TTFT in production benchmarks?

Defend pinning the tokenizer revision alongside the model weights when shipping a fine-tune

Walk through what happens during backprop for a LoRA-wrapped linear layer

A small medical fine-tune leaks verbatim credit-card snippets under red-teaming: identify the failure mode and the standard mitigation

Merging a LoRA back into an NF4-quantized base hits one subtle problem: name it and explain why

Adam stores two fp32 buffers per trainable parameter: what does that imply for LoRA vs full FT on a 7B?

When eval prompts appear verbatim inside an SFT file after a synthetic-data run, what's the root cause?

A distilled model hits 92% on a public benchmark but 31% on an internal hold-out: diagnose.

Why might a LoRA that passes every eval check then garble one in five prod requests?

Training loss draws a sharp saw-tooth at every epoch boundary: diagnose.

Llama-4 fine-tune shipping to 800M MAU: which license clause must the team re-read?