Turn text into vectors

Here's the one idea the whole engine rests on: the model turns a piece of text into a list of numbers — an embedding — that captures its meaning. Two sentences that mean similar things get similar lists, even if they use completely different words. That's what lets us search by meaning instead of matching keywords.

(If you want the theory behind why this works, the How Machines Understand Meaning course is the visual companion to this one. Here, we just use it.)

Follow along in a Python shell (python) or a scratch file:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

vector = model.encode("A dog runs across the park")
print(type(vector))    # <class 'numpy.ndarray'>
print(vector.shape)    # (384,)
print(vector[:5])      # the first 5 of 384 numbers

That's it — model.encode() takes text and hands back a 384-number vector. Those numbers aren't meaningful one by one; it's the whole pattern that encodes the sentence's meaning as a single point in a 384-dimensional space.

Similar meaning, similar vector

The magic is that meaning maps to location. Encode two sentences that mean nearly the same thing and their vectors come out close together; encode an unrelated one and it lands far away. We'll measure "close" properly in Chapter 4, but you can already feel it:

a = model.encode("How do I reset my password?")
b = model.encode("I forgot my login details")
c = model.encode("What time does the store close?")

# A quick-and-dirty closeness score (higher = more similar).
import numpy as np
def similarity(x, y):
  return np.dot(x, y) / (np.linalg.norm(x) * np.linalg.norm(y))

print("a vs b:", round(similarity(a, b), 3))   # high — same meaning
print("a vs c:", round(similarity(a, c), 3))   # low  — unrelated

a vs b: 0.612
a vs c: 0.061

Look at what just happened: sentence a and sentence b share not a single important word — "reset / password" vs "forgot / login" — yet the model scores them as strongly related, and correctly pegs the store-hours question as unrelated. A keyword search would have missed that completely.

That closeness score is the heart of the engine. In the next chapters we'll compute it against a whole collection of documents at once, and turn "which vector is closest?" into working search. First, let's build the collection.