Embed your corpus

From here we're building one real program. By the end of Chapter 5 the code from these chapters stacks into a single search.py you run. This chapter builds the thing we'll search: the corpus.

A corpus is just the collection of documents you want to search over — help articles, product descriptions, notes, paragraphs of a PDF, anything. We'll use a small set of support-style questions so the results are easy to eyeball, but the code is identical for thousands of documents.

Start your search.py

Create a file called search.py. Load the model and define the corpus as a plain Python list:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

corpus = [
  "How do I reset my password?",
  "Where can I see my past orders?",
  "What is your refund and return policy?",
  "How do I update my shipping address?",
  "Do you ship internationally?",
  "How can I contact customer support?",
  "How do I cancel my subscription?",
  "What payment methods do you accept?",
]

Embed all of it at once

In Chapter 2 we encoded one sentence at a time. encode() also takes a list and returns one vector per document — a matrix with one row each. Add this to search.py:

corpus_embeddings = model.encode(corpus, normalize_embeddings=True)

print(corpus_embeddings.shape)   # (8, 384) — 8 documents, 384 numbers each

Two things worth pausing on:

• One matrix, one call. corpus_embeddings is an 8 × 384 grid: row i is the embedding of corpus[i]. Encoding the whole corpus in a single call is far faster than looping, and it lines the data up for the fast search we're about to write.
• normalize_embeddings=True. This scales every vector to length 1. That one flag is a small setup step now that makes the similarity math in the next chapter both simpler and faster — a dot product will give us cosine similarity directly. More on exactly why in Chapter 4.

Run it to confirm the shape:

python search.py

(8, 384)

That matrix is your search index — eight meanings, laid out as eight points in space. All that's left is: given a query, find the nearest point. That's the next chapter.