Senior / Staff

Advanced transformer internals: MoE, KV cache, speculative decoding, scaling laws.

• Start hereExplain scaled dot product attention.
• What is the KV cache in transformer inference?
• In LLM serving, what is the primary driver of end to end latency for a generation request?
• Explain why transformers replaced RNNs for language modeling.
• Which components make up a standard transformer block?
• Spot the error in this explanation of temperature.

End-to-end LLM platform design: multi-tenant serving, cost optimization, reliability, monitoring.

• Start hereIn LLM serving, what is the primary driver of end to end latency for a generation request?
• Which metric best measures whether a RAG answer is grounded in the retrieved context?
• Design a RAG evaluation harness, what do you measure and how?
• When should you fine-tune instead of using RAG?
• What does RAG primarily help with in LLM based applications?
• Why combine BM25 with dense embeddings for retrieval?

Alignment, safety, evaluation at scale, team structure for AI orgs, build-vs-buy decisions.

• Start hereWhat is RLHF, and why is it used after pretraining?
• What does LoRA do, and why is it popular for fine-tuning?
• Which metric best measures whether a RAG answer is grounded in the retrieved context?
• Design a RAG evaluation harness, what do you measure and how?
• When should you fine-tune instead of using RAG?
• PPO vs DPO, what's the practical difference?