Groq

What is the KV cache in transformer inference?

In LLM serving, what is the primary driver of end to end latency for a generation request?

Why combine BM25 with dense embeddings for retrieval?

Why do reward models often reward longer responses, and how do you mitigate it?

An uploaded 300 page manual: the assistant answers well about early pages but ignores later ones

Decode the acronym MHA in the transformer attention context.

Design the parallelism layout for training a 16-expert MoE across 64 GPUs.

What slice level eval reveals about an 85% faithful RAG system suspected to fail on multi-hop queries

How does a base model differ from an instruction tuned model?

Sliding window attention can't see beyond the window. Describe the trick that recovers older context.