Augment & generate

The A and G of RAG, and the payoff of the whole course. We take the chunks the retriever found, fold them into a prompt as context, and ask the local model to answer using only that context. The model stops guessing from memory and starts reading from your documents.

The prompt is everything

A RAG prompt has three jobs: hand over the context, ask the question, and instruct the model to stay grounded. That last instruction — answer only from the context — is what turns a bluffing model into an honest one. Add this to rag.py:

import ollama

def answer(query):
  context = "\n".join(f"- {chunk}" for chunk in retrieve(query))

  prompt = f"""Answer the question using ONLY the context below.
If the answer is not in the context, say you don't know.

Context:
{context}

Question: {query}"""

  reply = ollama.chat(
      model="llama3.2",
      messages=[{"role": "user", "content": prompt}],
  )
  return reply["message"]["content"]

Read what this does: retrieve(query) pulls the relevant chunks (Chapter 4), we format them into a bulleted context block, and the f-string builds a prompt that carries the context, the question, and the grounding instruction. Then one ollama.chat call generates the answer locally.

Ask it the question that failed

Remember Chapter 2, where the model invented "$9.99/month"? Ask again — now with retrieval in front of it:

print(answer("How much does the Pro plan cost?"))

python rag.py

The Nimbus Notes Pro plan costs $6 per month. It also adds unlimited
notebooks, version history, and offline sync.

$6 per month — the real number, straight from your document, not the model's imagination. Same model, same question as Chapter 2; the only thing that changed is that we put the right context in front of it. That's RAG working.

Try a few more — answer("Is there an Android app?"), answer("What's the refund policy?") — and watch it answer each from the docs. One rag.py now retrieves and generates. The last step turns it into an actual chatbot.