Flashcard

How does an HNSW index handle deletes and updates under a heavy write workload?

pgvector ships two index types. Which should be your default reach in 2026, and why was it added later than the other?

Explain exactly what the ef_search knob on an HNSW index controls at query time.

Where does Hamming distance become the natural metric inside a vector database, and how is it implemented efficiently?

Explain what NVIDIA CAGRA buys you over CPU HNSW and the operational catch that decides whether you should reach for it.

Define tied embeddings and explain why Llama-3 chose to untie them

Describe the residual stream as a read/write surface that sublayers act on

What does the router inside a Mixture-of-Experts block actually do, per token?

Define d_model and name three architecture knobs that scale with it

Why production chat UIs stream tokens token by token instead of returning the full reply

Weight quantization in plain terms: what changes and what stays the same?

TTFT: name the metric it captures and what dominates it

Walk through how top_p (nucleus) sampling truncates the distribution

Compare top_k and top_p as truncation strategies for sampling

In benchmarks, tokens per second measures what exactly?

Bytes, characters, tokens: which one gets billed and how do they relate?

Throughput, for an LLM serving system: give the working definition

What is a tensor core and why does it dominate matmul throughput on GPUs?

How does the temperature parameter reshape the sampling distribution?

Define a stop sequence in an LLM API call

Server-Sent Events streaming: define it and explain why chat APIs default to it

Sequence length vs context window, what is the practical distinction and which one drives KV cache size?

The roofline model in plain language, without the chart, what is it saying?

Prompt caching shows up on Anthropic and OpenAI pricing pages, what does it actually do?

Walk through what actually happens during the prefill phase of an LLM forward pass.