Embeddings explained

If you have ever wondered how a search can find the right answer even when you use none of the same words as the document, the answer is usually embeddings. They are one of the core ideas behind Retrieval-Augmented Generation (RAG), the technique of giving a language model relevant documents to read before it answers. In this guide we will explain what embeddings are, how searching by meaning works, and how to choose a model, all without assuming you have seen any of this before.

What an embedding is

An embedding is a list of numbers that represents the meaning of a piece of text. You take a sentence, a paragraph, or a short document, feed it to a model, and get back a fixed-length list of numbers, for example 384 or 1,536 of them. That list is called a vector, which is simply a mathematician’s word for an ordered list of numbers.

The clever part is what those numbers capture. Texts that mean similar things get similar numbers. Texts that mean different things get different numbers. So “the cat sat on the mat” and “a feline rested on the rug” end up with vectors that are close together, even though they share almost no words. “The stock market fell sharply” lands somewhere else entirely.

You can picture each vector as a point in space. In everyday life we use three dimensions: left-right, up-down, and forward-back. An embedding just has many more dimensions, hundreds or thousands, which is impossible to draw but works exactly the same way. Points that sit near each other mean similar things. Points far apart mean different things.

How similarity works

Once your text is a set of points in space, “find me the most relevant document” becomes “find me the nearest points”. This is why embeddings are so useful for search.

Here is the practical flow. You take a collection of documents, split each one into smaller passages (this splitting is called chunking, because a whole book is too big to embed as one lump), and create an embedding for every chunk. You store all those vectors in a vector database, a store built specifically for finding the nearest vectors quickly. When a question comes in, you embed the question too, then ask the database for the chunks whose vectors are closest to the question’s vector. Those nearby chunks are your most relevant passages.

Closeness is measured with a bit of maths, most often cosine similarity, which compares the direction two vectors point in rather than how long they are. You do not need to calculate this yourself. Every vector database does it for you. The idea to hold on to is the one that matters: close together means similar in meaning.

This is also why embedding search beats old-fashioned keyword search for many tasks. Keyword search looks for exact words. Embedding search looks for meaning, so a question about “how to reset my password” can find a document titled “recovering account access” even with no shared keywords.

Embedding models

The thing that turns text into a vector is an embedding model. It is a neural network trained on enormous amounts of text so that it learns which words and phrases tend to mean similar things. You do not train it yourself. You use one that already exists, either by calling it through an API or by running an open one on your own machine.

There are two broad options:

Hosted models, which you reach over an API. You send text and receive vectors. They are quick to start with and need no hardware, but you pay per use and your text leaves your machine.
Open models that you download and run yourself. They are free to run, keep your data local, and give you full control, but you manage the hardware and setup.

One rule matters above all: you must use the same model for your documents and your questions. Two different models produce vectors that are not comparable, like measuring one thing in centimetres and the other in inches. If you re-embed your documents with a new model, re-embed your questions too.

Choosing dimensions and a model

Dimensions are how many numbers each vector contains. More dimensions can capture finer shades of meaning, but they cost more storage and make search a little slower. A model producing 384 dimensions is lightweight and fast. One producing 1,536 or more is richer but heavier. For most first projects, a smaller model is a fine starting point, and you can move up if results are not sharp enough.

When picking a model, weigh a few practical things:

Quality for your kind of text. A model tuned on general web text may struggle with legal or medical language.
Language coverage. If your documents are not in English, choose a multilingual model.
Cost and privacy. Hosted is convenient, open keeps data in-house.
Speed and size, which matter more as your collection grows into millions of chunks.

A sensible way to decide is to try two or three candidates on a handful of real questions from your own use case and see which returns the most relevant passages. Judge by results, not by benchmark tables alone.

Next steps

You now know the shape of it: embeddings turn text into vectors, similar meanings sit close together, and search becomes a matter of finding the nearest points. To go further, look at the kinds of data you might feed into a RAG system by exploring the RAG Repo directory, where some sources come already chunked or embedded, and check any unfamiliar words in the glossary. When you are ready, pick one small collection of documents, embed it with a single model, and try a few questions. Seeing it work on your own data teaches more than any diagram.