Evaluate & what's next

Your recommender makes confident predictions — but are they any good? The recommendations looked right, but "looks right" isn't a metric. Here's the honest test, and then where real systems go from here.

Hide a rating you already know

The trick to grading a recommender: take a rating you do know, pretend it's missing, predict it, and compare. If the prediction lands near the true value, the model works. This is called leave-one-out:

def predict_holding_out(user, movie):
  original = R[user, movie]
  R[user, movie] = 0                 # hide it
  recompute()                        # rebuild means / centered / similarity
  guess = predict(user, movie)
  R[user, movie] = original          # put it back
  recompute()
  return guess

for user, movie in [(1, 1), (2, 0), (3, 3)]:
  true = R[user, movie]
  guess = predict_holding_out(user, movie)
  print(f"{users[user]:5s} / {movies[movie]:12s}  true {true:.0f}  predicted {guess:.2f}")

Bob   / Inception     true 5  predicted 4.68
Carol / The Matrix    true 4  predicted 4.36
Dave  / Notting Hill  true 5  predicted 3.74

Each prediction lands within about a star of the truth — the model would have guessed these right. (Wrap recompute() around the mean/centered/similarity lines from earlier so they can be rebuilt after hiding a rating.) Averaging the error across many held-out ratings gives a single quality score you can watch as you improve the recommender.

The cold-start problem

Collaborative filtering has one famous weakness. A brand-new user has rated nothing, so they have no neighbors and we can't find anyone "like them"; a brand-new movie has no raters, so it never gets recommended. This is the cold-start problem — the recommendations course covers how real systems paper over it (ask new users for a few favorites, fall back to popularity, use movie metadata).

Where recommenders go next

What you built — memory-based collaborative filtering — is the foundation. The step up:

• Matrix factorization. Instead of comparing raw ratings, learn a short "taste vector" for every user and movie so their dot product predicts the rating. It's trained with gradient descent — the same loop from A Neural Net From Scratch — and it's what won the Netflix Prize. The taste-vectors chapter shows the intuition.
• Implicit feedback. Real systems rarely have star ratings; they have clicks, watches and skips. Same math, different signal.
• Hybrids. Blend collaborative filtering with content features and popularity to survive cold start.

You didn't just read about how your feed knows you — you built the engine, from a ratings matrix to a ranked recommendation. Every "you might also like" you see next will look a little less like magic.