Easy to grasp

Imagine handing a note to a librarian who immediately puts it in a tray on the counter and says, got it. The note is safe and is going into the catalogue, but the catalogue card has not been typed and filed yet. If you ask the catalogue ten seconds later whether the note exists, the catalogue says no, because the typing happens in batches every few minutes. The note is not lost. The catalogue simply has a refresh schedule, and your query landed between two refreshes. Most vector databases work the same way: a write is accepted and durable long before it is searchable.

Think of a cemetery. The person is gone but their stone is still standing on the same plot. Anyone walking through the cemetery sees the marker but knows nobody is home. Vector databases use the same trick: deletions get a marker that says nothing live is here, but the structure stays in place because pulling it out would mean redoing the paths every other stone takes. A groundskeeper comes by occasionally and reorganizes whole sections, which is the compaction step that finally reclaims the space.

Imagine a city where buildings cannot be demolished, only marked condemned. The streets still wind through them, the map still lists them, the postal carrier still walks past them. The city has the same number of living residents but takes longer to navigate, takes the same amount of land, and gets confusing in neighborhoods that have been condemned often. New buildings still go up at the same speed; the condemned signs do not help construction. And the GPS coordinates of any address are unchanged by condemnations. That is exactly what tombstones do to a vector index.

Think of a vector database as a library with several rooms. If a brand-new book is missing, somebody is still wheeling it from the truck. If old books mysteriously slow things down even after a purge, the librarians left empty cards on the shelves. If two friends ask the same question and get different answers, they walked into different rooms. If two libraries rate the same book differently, they grade on different scales. If a request for ten Spanish books gives you three, the library only checked a small shelf before filtering by language. Each symptom points at a specific layer of how the search is built, and matching the symptom to the layer is half the debugging job.

Imagine renting a stadium because once a day, for an hour, a thousand fans show up. The rest of the day the seats are empty but you still pay full rent. That is what dedicated pods feel like for a workload that bursts once a day. Now imagine a venue that opens only when fans arrive and charges per ticket sold, with a tiny daily fee for keeping your team's posters on the wall. That is serverless vector pricing. The bursty workload pays for the busy hour and almost nothing for the quiet hours. The match between when you actually use capacity and when you pay for it is what makes the bill small.

Imagine your library is mostly in a warehouse three towns over, and a single trip to the warehouse takes thirty minutes regardless of how many books you grab. If a book sends you to fetch the next one in the warehouse, then the next, then the next, you spend the whole afternoon driving. That is what DiskANN does on object storage. SPANN keeps a single map of which shelf to visit and, on each query, drives to a couple of shelves and brings back whole boxes at once. Fewer trips, more parallel, far less wall-clock time. The medium being slow per request is what makes the difference, not the total weight of books.

Imagine you order food from sixteen kitchens at once and your dinner is only ready when the last kitchen finishes. Each kitchen is late only 1 time in 100, which sounds great. But you are betting that all sixteen kitchens are on time at the same moment. The chance of that is 0.99 multiplied by itself sixteen times, which is about 85%. So 15% of your dinners are late, even though each kitchen is almost always quick. The more kitchens you involve, the more often somebody is having a bad minute and the more often your dinner waits. That is exactly what happens when a search query fans out to many shards: the slowest one sets the clock.

Imagine a kitchen where the staff usually plates orders quickly because each cook has a clear counter. Now the manager starts a deep clean of one section in the background, taking over an oven and one of the prep stations. Most orders flow through the unused stations just fine and arrive on time. But a few orders need the section being cleaned. Those orders wait for the deep clean to pause, or get bumped from station to station while space frees up, so they take much longer than usual. The median order looks healthy. The slowest few are terrible. That is what tail latency measures, and that is what an index rebuild does to queries.