LLMOps Engineer

Why is causal attention cheaper than full bidirectional at the same length?

Name two production KV-cache eviction policies and what each prioritizes

Decode-phase attention is memory bandwidth bound. Explain what flips the regime.

Describe how continuous batching interacts with the KV cache

What problem does chunked prefill solve in production LLM serving?

How does prefix KV-cache sharing reduce serving cost across requests?

LLM-as-judge scores drift up across 50k weekly evals while user complaints hold steady. Diagnose likely causes and design a calibration system preventing silent score inflation.

Design a multi-dimensional eval dashboard for a production LLM product. What dimensions, views, and alerts does it need, and what is the one chart that the VP of Engineering should check every Monday?

Running LLM-as-judge on your full eval set costs $200 per run. The team runs evals 10 times a day during development. How do you cut cost without losing signal?

A production model has no ground-truth labels on live traffic. Design a monitoring plan that catches degradation before users complain.

The model provider just updated the model behind the API endpoint. Describe the regression testing process before allowing traffic.

You need to find an MCP server for Slack. Where do you look in 2026?

A local stdio MCP server has no auth token. Is that a security hole?

Design a multi-tenant remote MCP server that exposes each customer's Postgres database. Walk through transport, auth, isolation, and observability.

You want to add 5,000 Hindi medical terms to Llama 3 8B. Walk through the procedure and the risks.

You are fine-tuning Llama 3 8B on variable-length instruction data. Which padding strategy and side do you pick?

Your indexing pipeline tokenizes 10M docs in 18 hours. What single change gives the biggest speedup?

What does one agent trace capture step by step?

Why run agent-generated code inside an isolated sandbox?

What does observability mean for production LLM agents, and how is it different from app monitoring?

Why cap each agent run with a cost-per-task budget?

What is the right permission model for an agent with read-docs, run-SQL, and Slack-send tools?

How do you enforce a $0.50 per-task LLM budget at runtime beyond setting max_tokens?

Design an observability and debugging system for a production multi-agent application with 5+ collaborating agents.

You're shipping a prompt change to production. What is the strongest reason to treat prompt-template version IDs (e.g. v1.7.2) as first-class artifacts, separate from your code release version?