LLM System Design

4 questions

End-to-end production architectures. Multi-tenant serving, rate limiting, prompt caching, model routing, cost models, latency budgets, fallback strategies.

Sort

Questions

Estimate the per call cost of a typical RAG chatbot using GPT-4o-mini.
FlashcardEasy·Qual 4.4OpenAI
In LLM serving, what is the primary driver of end to end latency for a generation request?
Multiple ChoiceMedium·Qual 4.7MetaNVIDIA
Design a RAG evaluation harness, what do you measure and how?
Short AnswerHard·Qual 4.7
Why combine BM25 with dense embeddings for retrieval?
Multiple ChoiceEasy·Qual 4.6