What is a text embedding?

A vector — a list of numbers — that represents the meaning of a piece of text, positioned so that texts with similar meaning have nearby vectors. It's how machines turn meaning into geometry.

How is similarity between embeddings measured?

Most commonly with cosine similarity — the angle between two vectors. Vectors pointing the same way mean similar meaning. Vector databases compute this across millions of vectors in milliseconds.

Can I compare embeddings from two different models?

No. Each model produces vectors in its own coordinate space, so comparing across models is meaningless. If you switch embedding models, you must re-embed your entire corpus.

Embeddings: Turning Words Into Coordinates

Here's a magic trick that stops being magic the second you see how it works, and then becomes magic again because the underlying idea is so clean.

A computer has no idea what "dog" means. It can't pet one. But it can place the word "dog" at a specific point in space, and place "puppy" right next to it, and "cat" nearby, and "tax audit" very, very far away. Once meaning becomes *location*, "find me similar things" becomes "find me nearby things" — and that's just geometry.

From words to coordinates

An embedding model takes text and spits out a vector — a list of numbers. Not 2 or 3 numbers like a map; usually hundreds or thousands. Each dimension captures some squishy, learned aspect of meaning that no human labeled and no human can fully name. One dimension might loosely track "animalness," another "formality," another something with no human word for it at all.

embed.ts

const dog   = await embed('dog');    // [0.21, -0.04, 0.88, ... 1536 dims]
const puppy = await embed('puppy');  // [0.19, -0.02, 0.85, ...]
const audit = await embed('tax audit'); // [-0.71, 0.40, 0.03, ...]

similarity(dog, puppy); // 0.91  -> basically neighbours
similarity(dog, audit); // 0.06  -> different universes

The remarkable part: nobody programmed these positions. The model learned them by reading enormous amounts of text and noticing which words show up in similar contexts. "Dog" and "puppy" appear in the same kinds of sentences, so they drift together. Meaning, it turns out, is mostly about company you keep.

"Close" is just an angle

The standard way to measure closeness is cosine similarity — the angle between two vectors. Pointing the same way means similar meaning; perpendicular means unrelated. The whole formula fits in a few lines, and you'll never need to write it because every vector DB does it for you, but here it is so it stops being scary:

cosine.ts

function cosine(a: number[], b: number[]) {
  let dot = 0, ma = 0, mb = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    ma += a[i] * a[i];
    mb += b[i] * b[i];
  }
  return dot / (Math.sqrt(ma) * Math.sqrt(mb)); // -1 .. 1, higher = closer
}

Note

You almost never compute this by hand. Pinecone, pgvector, Qdrant, and friends do approximate nearest-neighbour search across millions of vectors in milliseconds. You just bring the vectors.

Why not just use keywords?

Keyword search matches letters. Embeddings match meaning. Search "how do I cancel my plan" with keywords and a page titled "Ending your subscription" never shows up — zero shared words. With embeddings, the two sit close in space because they *mean* the same thing. That's the entire leap from "search" to "semantic search."

The famous party trick

Because meaning lives in space, you can do *arithmetic* on it. The classic example, which genuinely worked and genuinely melted my brain the first time:

king − man + woman ≈ queen

Subtract the "man" direction, add the "woman" direction, and you land near "queen." The model never learned royalty or gender as concepts. It just learned that words live in relationships, and those relationships turned out to be directions you can walk along. Paris − France + Italy lands near Rome. It's eerie and wonderful.

What you actually build with this

Semantic search — match by meaning, so "how do I cancel" finds the page titled "Ending your subscription."
RAG — retrieve the right context chunks to ground an LLM's answer.
Recommendations — "users who liked this" becomes "items near this in space."
Clustering & dedup — group similar tickets, find near-duplicate docs, detect outliers.
Classification — embed a text, see which labelled cluster it lands nearest.

A tiny semantic search, end to end

Strip away the infrastructure and the whole idea is about ten lines:

search.ts

const docs = ['Reset your password', 'Cancel your subscription', 'Upgrade your plan'];
const index = await Promise.all(docs.map(async d => ({ d, v: await embed(d) })));

async function search(query: string) {
  const q = await embed(query);
  return index
    .map(({ d, v }) => ({ d, score: cosine(q, v) }))
    .sort((a, b) => b.score - a.score)[0].d;
}

await search('how do I stop being billed'); // -> 'Cancel your subscription'

Three things that'll bite you

01Don't mix models. Vectors from model A and model B live in different spaces. Comparing them is comparing latitude to Fahrenheit.
02Chunk size matters. Embed a whole 50-page PDF as one vector and you get the average of everything, which means the meaning of nothing.
03Re-embed when you upgrade. New embedding model = new coordinate system. Your old vectors are now in the wrong city.

Heads up

If your semantic search suddenly returns nonsense after a model upgrade, you almost certainly forgot to re-embed your corpus. Ask me how I know.

That's the whole idea. Meaning becomes geometry, similarity becomes distance, and a machine that understands nothing can suddenly find exactly what you meant. Not bad for a list of numbers.

Key takeaways

01An embedding turns text into coordinates where similar meanings sit close together.
02'Similar' becomes 'nearby' — so semantic search is just nearest-neighbour lookup.
03Never mix embedding models, and re-embed your corpus when you upgrade one.

FAQ

AIEmbeddingsVector SearchRAG