Embeddings Explained: How AI Search Finds Meaning

Every semantic search engine, recommendation feed, and retrieval-augmented generation pipeline sits on the same hidden layer: embeddings. This guide explains what embeddings are, how models turn text into vectors, and how to pick the right embedding model for your product.

The verdict

An embedding is a list of numbers that captures what a piece of text means, not just the words it contains. Two sentences with similar meaning get similar number lists, so a machine can measure meaning the way it measures distance.

For most teams, the embedding model matters less than how you use it. Chunk your text well, use cosine similarity for comparisons, and pick dimensions your database can afford. OpenAI’s text-embedding-3-small is the safe default; upgrade only when quality measurements tell you to.

What embeddings actually are

Computers cannot read. They count. An embedding model bridges that gap by converting any piece of text into a fixed-length list of floating-point numbers, called a vector. OpenAI’s text-embedding-3-small turns a sentence into 1,536 numbers. Google’s text-embedding-004 produces 768. Cohere’s embed-english-v3.0 produces 1,024 (see Cohere’s embed documentation for the full model table).

Also Read:

LoRA Fine-Tuning Explained: Adapt LLMs for Less

Those numbers are not random. Training arranges them so that texts with related meaning land near each other in a high-dimensional space. A help article about resetting passwords sits close to one about changing credentials, even if the two share almost no words. An article about pasta recipes sits far from both.

This property is why embeddings beat keyword search. Keyword search looks for matching letters. Embeddings look for matching intent. A user who types “how do I get my money back” finds the refund policy even when the word “refund” never appears in their query.

How a model turns text into numbers

Before embedding, text goes through tokenization: the model chops the input into tokens, short pieces of words and punctuation. The word “embeddings” might split into two or three tokens. Every model publishes a token limit; OpenAI’s v3 embedding models accept up to 8,191 tokens per call.

Also Read:

LLM Quantization Explained: Fit Bigger Models in Less VRAM

Each token starts as a lookup in a large table that assigns it an initial vector. Then several transformer layers adjust those vectors by reading the surrounding tokens. The word “bank” receives a different vector in “river bank” than in “bank account” because the layers factor in context. If you want to see the mechanics, the LlamaIndex OpenAI embedding integration docs walk through a working example.

Finally, the model pools the token vectors into one vector for the whole input. That single vector, typically normalized to unit length, is the embedding. The training objective rewards the model when related texts end up close together and unrelated texts end up far apart. Models learn this arrangement from massive text pairs, not from any human labeling each vector by hand.

Also Read:

AI Agents vs Workflows: When to Use Which

Cosine similarity: how distance becomes meaning

Once texts are vectors, comparing them is arithmetic. The industry standard is cosine similarity: it measures the angle between two vectors and returns a score from -1 to 1. A score near 1 means the vectors point almost the same way, which the model reads as similar meaning. A score near 0 means unrelated. Negative scores mean opposing usage patterns, which is rare in practice.

Cosine similarity ignores vector length on purpose. A long paragraph and a short sentence can express the same idea, and their vectors will point the same way even though their magnitudes differ. That is why most embedding pipelines normalize vectors before storing them: normalization makes cosine similarity reduce to a simple dot product, which is fast.

Two sibling metrics appear in tooling. Euclidean distance measures the straight-line gap between vector endpoints. Dot product multiplies the vectors element by element. When both vectors are normalized, all three metrics rank results identically, so cosine similarity remains the default choice across vector databases.

Also Read:

Prompt Injection Explained: How It Works and How to Stop It

Popular embedding models compared

The market has consolidated around a few production-ready models. Dimensions, context length, and cost are the practical differences.

One detail deserves attention: OpenAI’s v3 models let you truncate dimensions with a dimensions API parameter. You can request a 512-dimension version of text-embedding-3-large and lose only a small fraction of quality while cutting storage and query cost sharply. OpenAI’s embedding guide documents this parameter alongside the available models. Cohere’s newer v4/v5 models offer similar choices from 256 up to 2,048 dimensions. Start small, measure retrieval quality on your own data, and only scale up when the numbers justify it. An independent model comparison tracks dimensions, pricing, and benchmark scores across providers.

Model choice also locks your database schema. If you store 1,536-dimension vectors and later switch to a 3,072-dimension model, you must re-embed everything. Pick a model you plan to keep, and version your embeddings so a future migration does not corrupt existing indexes.

Where embeddings show up in real products

Semantic search is the obvious application, but embeddings power a wider set of features.

Retrieval-augmented generation (RAG). A RAG pipeline embeds your documents, embeds the user’s question, and retrieves the closest chunks as context for the language model. The embedding is the entire retrieval mechanism. Bad chunking or a mismatched model degrades the whole system. We compared when RAG beats fine-tuning in our RAG vs fine-tuning guide.

Recommendations. Embed user behavior and item descriptions in the same space, then recommend items whose vectors sit near the user’s vector. News feeds, video platforms, and marketplaces all use variants of this pattern.

Clustering and deduplication. Support teams cluster thousands of tickets to find the recurring issues. Content platforms deduplicate near-identical uploads. Both tasks reduce to grouping vectors by distance.

Classification without labels. Embed a set of category descriptions, embed an incoming text, and assign it to the closest category. This zero-shot approach replaces many hand-labeled classifiers for routing and triage workloads.

Vector databases such as pgvector, Pinecone, and Qdrant provide the storage and nearest-neighbor indexes these workloads need. Our comparison of pgvector, Pinecone, and Qdrant breaks down the tradeoffs.

Common mistakes when working with embeddings

Chunking badly. The number one retrieval failure is not the model; it is the chunks. Splitting documents mid-sentence destroys meaning. Use sentence-aware chunking with a small overlap, typically 10 to 20 percent, and attach metadata like the source page to each chunk.

Mixing models. Embeddings from different models live in different vector spaces. You cannot compare an OpenAI vector with a Cohere vector, and you cannot swap models without re-embedding. Freeze one model per index.

Skipping normalization. Some models return unnormalized vectors. Normalizing before storage makes cosine similarity correct and consistent across providers. It costs one line of code and prevents silent ranking bugs.

Ignoring the cost of dimensions. A 3,072-dimension index costs roughly double the storage and compute of a 1,536-dimension index. For most search features, the quality gain does not justify the bill. Measure with your own relevance judgments before paying for dimensions you cannot see.

Testing on toy data. Embeddings look great on five sample queries. Production queries are messy, misspelled, and vague. Build an eval set of at least 50 real queries with known-good answers, and score recall before shipping. Our guide to building LLM evals that catch real failures applies directly to retrieval testing.

Embeddings vs the rest of the AI stack

Embeddings often get confused with neighboring concepts. The distinctions matter when you architect a system.

Embeddings are not LLM prompts. A prompt asks the model to generate text; an embedding asks it to measure meaning. They use different endpoints, different pricing, and different failure modes.

Embeddings are not function calling. Function calling lets a model request an action from your code; embeddings let you search your data before the model responds. Many agents use both: embeddings retrieve context, function calling performs the task. Our function calling guide covers the action side.

Embeddings are not a substitute for good data hygiene. A model can only retrieve what you indexed. If your knowledge base is stale, duplicated, or contradictory, the embedding layer faithfully returns stale, duplicated, contradictory results. Fix the source documents first.

Practical next steps

If you are adding semantic search to an existing product, follow this order:

1. Define the retrieval task. Write 50 real user queries and the documents each one should return. This eval set is your compass for every later decision.
2. Pick one model and freeze it. text-embedding-3-small is the safe default for English; text-embedding-004 for Google Cloud shops. Record the model version alongside your vectors.
3. Chunk with care. Sentence-aware chunks, modest overlap, metadata attached. Revisit chunking before blaming the model.
4. Normalize and index. Store normalized vectors in pgvector or a managed vector database with an HNSW index for fast approximate search.
5. Measure, then tune. Score recall@k on your eval set. Only then consider larger dimensions, rerankers, or hybrid search with keyword matching.
6. Watch the cost. Embedding pricing is per token. Cache embeddings for documents that rarely change, and consider caching strategies for the generation layer too to keep the total LLM bill predictable.

Conclusion

Embeddings turn the fuzzy notion of meaning into numbers a database can rank. The concept has been stable for years; the models keep getting cheaper and more accurate. What separates working systems from demos is unglamorous: careful chunking, one frozen model, normalized vectors, and an eval set of real queries.

Start with the simplest configuration that passes your evals. The embedding model is rarely the bottleneck. The data pipeline around it is.