AI Product Engineer

Define TTFT, TPOT and throughput. Which should a chatbot vs a batch summarizer optimize?

A chatbot team reports TTFT P99 jumped from 300ms to 1.4s overnight. Which root cause is most likely?

Why does an agent loop with 5 tool calls have ~6× the latency of a single completion and how do you mitigate?

Which mitigation has the largest impact on agent-loop latency when tool calls are independent?

Why does streaming (SSE) improve perceived latency without changing total decode time?

Spot the errors in this stop-sequence configuration recommendation

How does the serving stack actually decide when to stop generation and what bugs come from mis-set stop sequences?

Given a TPM rate limit, how do you compute supportable concurrent users for a feature?

How do Anthropic and OpenAI prompt-cache pricing models work and what counts as a cache hit?

Why does prefill saturate compute while decode is bottlenecked on memory bandwidth?

Which statement best explains why prefill and decode hit different bottlenecks on the same GPU?

Why is the vision-encoder phase a different bottleneck class from LLM decode in a multi-modal model?

Compare greedy, sampling and beam search at inference: cost and best use case.

Why do production chat APIs almost never offer beam search as a decoding option?

For a 200-token prompt and 1000-token completion, why does decode dominate end-to-end latency?

In a production RAG system with a 2-second end to end latency budget for non-streaming responses, which step typically dominates and where should the optimization effort go?

Match each schema-validation pattern to the production scenario where it fits best.

How effective is the soft directive 'be brief' as an output-length constraint compared to the hard max_tokens parameter?

When should you NOT enable JSON mode for a production LLM call, despite needing JSON output downstream?

You're enabling response_format=json_object (OpenAI) or strict structured output (Anthropic). What's the tradeoff vs prompting for JSON in prose?

In a multi-turn conversation, when should you reset history, carry full state, or summarize?

What UX patterns help users make informed MCP approval decisions without approval fatigue?

Which approval UX pattern best reduces approval fatigue while preserving meaningful consent?

Spot the error in this Claude Desktop MCP server configuration

Complete the Claude Desktop MCP server configuration fields