AI Product Engineer

Why production chat UIs stream tokens token by token instead of returning the full reply

When does inflating the system prompt actually lower cost per call?

Predict the break-even read count for Anthropic prompt caching given the 1.25x / 0.1x rates

Describe why structured-output mode quietly raises the per-call token bill

Name the dominant cost driver of GPT-5.5 latency at a 200k-token input

Estimate the token tax a single tool round-trip adds compared with an inline answer

Why can a single high-res image cost more input tokens than a multi-paragraph prompt?

Why can a 50-token reply from a reasoning model still bill you for thousands of tokens?

Does asking for top-k logprobs add real compute cost per token, or only bytes on the wire?

Greedy decoding and temperature sampling: does one cost more per token than the other?

A response comes back with finish_reason = 'length' which interpretation is correct?

Server-Sent Events streaming: define it and explain why chat APIs default to it

Does Anthropic's message_start event count toward TTFT in production benchmarks?

How do you evaluate a customer service chatbot that handles multi-turn conversations? What dimensions matter?

A production model has no ground-truth labels on live traffic. Design a monitoring plan that catches degradation before users complain.

The PM asks how to tell if the chatbot got better after the last deploy. Walk through setting up an A/B test.

A single-turn eval shows your chatbot scores 4.2/5. But users complain it 'forgets what they said.' Which eval dimension is your single-turn setup missing?

Where do you configure MCP servers in Zed, and where do their tools appear?

How is an MCP host different from an MCP client?

How does Cursor discover and load MCP servers?

You need to add an MCP server to Claude Desktop. Which file do you edit and what goes in it?

You change one word in the system prompt and prompt caching breaks. Why?

When should a production agent pause for human approval?

How should an agent degrade gracefully when a third-party API tool with 99% uptime is unavailable?

Select all techniques that meaningfully improve the reliability of source citations in a production RAG answer (chunks resolving back to real source documents).