LLMOps Engineer

Identify the workload pattern where a traditional ANN-indexed vector database is the wrong tool, and a log-structured or batch-rebuild approach fits better.

Which of these 2026 vector databases are credible at billion-vector scale? Select all that apply.

How does an HNSW index handle deletes and updates under a heavy write workload?

Sharding a vector index that does not fit on one node, how is it typically done, and what's the query-time cost?

Compare namespace-per-tenant against single-shared-index-with-tenant-filter for a B2B RAG product serving 5000 customers.

What does Matryoshka representation learning buy a vector-database operator beyond regular embeddings?

Estimate the RAM cost of an HNSW index over 100M vectors at 1024 dimensions with M=32, including graph overhead.

Explain what NVIDIA CAGRA buys you over CPU HNSW and the operational catch that decides whether you should reach for it.

What problem does DiskANN solve that HNSW does not, and when do you reach for it?

Doubling the embedding dimension from 768 to 1536 — what changes in storage, query latency, and PQ trainability?

A team's Pinecone bill 5x'd month over month with no traffic change. Order the most likely causes to investigate, from most likely to least likely.

When does binary quantization beat Product Quantization as the production compression choice for a vector index?

Same block, two regimes: training parallelism versus inference autoregression

Select the correct comparisons of an MoE transformer versus a dense transformer at equal active parameters

Pick the statements that accurately describe how an MoE sublayer replaces a dense FFN

Speculative decoding is already on and batch size just climbed to 64, keep it or kill it?

Match each GPU generation to the quantization format that lights up its native tensor cores.

For 1M-token context decode, which parallelism axis actually relieves KV pressure?

Two providers both advertise $0.50 per million output tokens, why is that comparison still dishonest?

Heavy multi-turn chat shares system prompts across requests, pick the serving framework that exploits that best.

At batch 16 and 4k context with FP8 weights, which model still fits on one 80 GB H100?

Which deployment shape best meets a strict P99 TPOT SLA on bursty traffic?

Name the architectural shifts vLLM v1 made relative to v0

Pick the serving changes TensorRT-LLM 0.18 (early 2026) actually shipped

Per-token decode time grows linearly as a chat session lengthens, debug it