Short Answer

Sharding a vector index that does not fit on one node, how is it typically done, and what's the query-time cost?

Define recall@10 for a vector index, and spell out exactly what 'ground truth' means in that definition.

Walk through how Product Quantization compresses a 1024-dim float32 vector down to 32 bytes, step by step.

Compare namespace-per-tenant against single-shared-index-with-tenant-filter for a B2B RAG product serving 5000 customers.

A team indexed embeddings under cosine but queried with Euclidean, and recall collapsed. Diagnose the cause.

Estimate the RAM cost of an HNSW index over 100M vectors at 1024 dimensions with M=32, including graph overhead.

Name two situations where Flat brute force search is still the correct production choice in a 2026 vector pipeline.

Aggressive metadata filtering can silently destroy recall on an HNSW index even when matches exist. Why?

Defend untying the output head: what does an extra ~500M parameters at the tail of the stack buy?

Same block, two regimes: training parallelism versus inference autoregression

Chinchilla's lesson: how did it reshape architecture and training choices after 2022?

Why does a deep transformer stack stop training if you remove the residual connections?

Walk through why pre-norm trains stably past 100 layers while post-norm needs warmup tricks

Defend the 4x FFN expansion ratio, then explain why SwiGLU breaks the simple rule

Name a 2026 task where an encoder-decoder transformer still beats a decoder-only LLM

Spend a fixed parameter budget: more layers or wider d_model, and what does the evidence say?

Why did decoder-only architectures take over tasks T5-style encoder-decoders used to win?

Two providers both advertise $0.50 per million output tokens, why is that comparison still dishonest?

Name the architectural shifts vLLM v1 made relative to v0

Per-token decode time grows linearly as a chat session lengthens, debug it

When does inflating the system prompt actually lower cost per call?

Sketch a Llama 3.1 70B self-host versus hosted-API break-even on the back of an envelope

Qwen 3.5: when does running your own GPU beat paying the hosted API per token?

Walk through how a prefix-cache hit changes the work that chunked prefill has to do

Pick the cheaper to serve option at equal quality: Llama 4 Maverick MoE or Llama 3.1 70B dense