LLMOps Engineer

When does inflating the system prompt actually lower cost per call?

TPOT is high, so the team plans to upgrade A100 to a faster compute GPU, critique

Diagnose the reasoning error in 'a bigger tokenizer vocabulary always cuts our token bill'

'Prompt caching cuts our bill by 90 percent across the board', what's overclaimed?

Find the wrong move in 'decode is slow, so let's switch to a smaller-FLOP model'

A teammate randomized system prompts per user and expects the prompt cache to still help, what's wrong?

Spot the error: 'we autoscale our LLM serving pods on CPU utilization.'

Pick the H100 workload where spot pricing is genuinely safe to deploy on

Sketch a Llama 3.1 70B self-host versus hosted-API break-even on the back of an envelope

Qwen 3.5: when does running your own GPU beat paying the hosted API per token?

Predict the break-even read count for Anthropic prompt caching given the 1.25x / 0.1x rates

Walk through how a prefix-cache hit changes the work that chunked prefill has to do

Which quantization combo squeezes the most decode bandwidth per percent of quality lost?

P99 TTFT spikes every Tuesday at 9am, order the triage steps

Rank these chat-API cost levers from highest to lowest typical ROI

Match each part of an OpenAI tool-call round-trip to the usage field that bills it

Pick the cheaper to serve option at equal quality: Llama 4 Maverick MoE or Llama 3.1 70B dense

Memory creeps up for hours then OOMs at 3am: diagnose the KV-cache leak pattern

Match Hopper FP8 and Blackwell FP4 tensor cores to their throughput and bandwidth effects on decode

Name the dominant cost driver of GPT-5.5 latency at a 200k-token input

Each decode step loads ___ from HBM, regardless of how many tokens have already been generated

What does the first forward pass set up that the second one doesn't pay for?

Walk through how tensor parallelism shards one matmul across GPUs and what it adds to decode step time

Prefix-cache hit lands on a request: does speculative decoding still help that request?

When does splitting prefill and decode onto separate replica pools actually pay off in production?