Search has traditionally worked by matching words. Type in “quick healthy breakfast ideas” and a keyword based system looks for pages containing those exact words, ranking results by how often those words appear. Vector search takes a different approach. It looks for content that means the same thing, even when the words used are completely different.
This is not a small technical detail, it changes what search can actually do. A vector search system can match “quick healthy breakfast ideas” against an article titled “nutritious morning meals” because both phrases sit close to each other once converted into numbers. Understanding how that conversion works, and what it costs you in return, matters for anyone building search or retrieval features today.
Understanding vector embeddings
At the core of vector search sits the embedding. An embedding is a list of numbers that represents a piece of text, an image, or any other content. You get from raw text to that list of numbers by passing the content through an embedding model, usually a neural network trained specifically for this job.
The model does not produce numbers randomly. It learns to place similar concepts close together in this number space and dissimilar concepts far apart. Feed it the words “lion” and “bobcat” and the resulting vectors land near each other because both describe wild cats of similar size and temperament. Feed it “cat” and you get a vector that sits close to both, since a house cat shares real similarity with its bigger cousins, just not as much.
A typical embedding model produces vectors with hundreds or even thousands of dimensions. Each dimension does not correspond to something a human can name directly, unlike height and weight in a straightforward spreadsheet. Instead, the dimensions together encode abstract properties of meaning that the model learned during training, and no single number in the list tells you much on its own.
How vector search actually works
Once you have embeddings for your content, running a search follows a predictable sequence. The search query itself gets converted into a vector using the same embedding model that indexed your content. The system then compares that query vector against every vector stored in the database and returns the ones sitting closest to it.
Distance here usually means cosine similarity, which measures the angle between two vectors rather than their absolute position. Two vectors pointing in roughly the same direction score high on cosine similarity even if one is much longer than the other, and that matters because embedding magnitude often reflects things like text length rather than meaning. Euclidean distance and dot product are the other two common choices, and most vector databases let you pick which one to use per index.
Where vector databases fit in
Storing a few thousand embeddings in memory and comparing a query against each one is trivial. Storing hundreds of millions of embeddings and getting a response back in under fifty milliseconds is a different problem entirely, and that is the problem vector databases exist to solve.
A brute force comparison against every stored vector, known as exact nearest neighbour search, becomes too slow once your collection grows past a few hundred thousand items. Vector databases instead use Approximate Nearest Neighbour algorithms, commonly abbreviated ANN, which build index structures such as HNSW graphs or IVF clusters. These structures let the system skip most of the collection and still return results that are correct, or very close to correct, almost every time.
That word approximate is worth pausing on. ANN indexes trade a small amount of recall for a large amount of speed, and you can usually tune how much of each you want. Push an HNSW index toward higher recall and query latency goes up along with memory usage, push it toward speed and you occasionally miss a result that a brute force search would have found.
For most product search or retrieval augmented generation use cases, ninety five percent recall at ten milliseconds beats one hundred percent recall at eight hundred milliseconds. That trade-off is worth making deliberately rather than by default.
Purpose built vector databases such as Pinecone, Weaviate, Qdrant, and Milvus were designed from day one around this problem. They generally give you the most tuning control, along with features like metadata filtering and hybrid search, out of the box. General purpose databases have caught up quickly as well. PostgreSQL, through the pgvector extension, now handles vector search well enough for a large share of production workloads, and it saves you from running a separate database just for embeddings if your application data already lives in Postgres.
In practice, I would default to pgvector for anything under a few million vectors, particularly when the rest of the application data already lives in a relational database. Once you cross that scale, or once you need features like high volume real time upserts or strict multi tenant isolation, a dedicated vector database starts paying for its added operational complexity.
Vector search versus keyword search
Keyword search is not obsolete, and it is worth being clear about that before writing it off. It matches exact terms fast, it is inexpensive to run, and it handles structured or precise queries, such as a product SKU or an exact error code, better than vector search does. Vector search has no real advantage when a user already knows the exact term they are looking for.
Where keyword search falls short is anything involving synonyms, related concepts, or natural language questions. Searching for “red shoes” against a keyword index misses listings tagged “scarlet footwear,” and a plain keyword system has no real way to understand a question like “what should I wear for a rainy day hike.” Vector search handles both of these naturally because it compares meaning rather than tokens.
Most production search systems I have worked on end up combining both approaches rather than picking one. A hybrid search runs a keyword match and a vector match in parallel, then merges and re-ranks the combined results. This tends to outperform either approach alone, since keyword search still catches exact matches that a vector search might rank lower than expected, especially for short queries with little semantic content.
Practical considerations before you adopt it
Embedding generation costs money and adds latency, whether you call an API for it or run a model locally. Every document you index needs an embedding call, and every search query needs one too, so factor that into your cost model before committing to vector search at scale, particularly for content that changes frequently and needs re-indexing.
Embedding quality also depends heavily on which model you choose, and switching models later is not free. Vectors produced by two different embedding models are not comparable to each other, so changing your embedding model means re-embedding your entire dataset from scratch. Pick a model with your expected data and query patterns in mind, and expect to revisit that choice as better models become available, since this space keeps moving.
Summary
Vector search moves retrieval from matching exact words to matching meaning, which is what makes it useful for the chatbots, recommendation engines, and retrieval augmented generation applications now common across the industry. Embeddings do the work of representing meaning as numbers, and vector databases do the work of searching through those numbers fast enough to be useful in a real application. Neither piece replaces keyword search entirely, but together they solve problems that keyword search alone could never handle well.
Leave a Reply