a hands-on course

Build a
Voice Agent

(Voice AI)

Give Claude ears and a voice: talk to it, and it talks back. In Python, in about half an hour.

A free, hands-on tutorial: build a voice agent you can actually talk to. Transcribe your speech locally with Whisper, send it to Claude with a prompt written for speaking, say the reply with your computer's built-in voice, then stream it so the agent starts talking at the first sentence. (Needs an Anthropic API key and spends a few cents. Speech-to-text and text-to-speech are free and run on your machine.)

in the real world

Voice assistants, phone support bots, in-car helpers and hands-free coding tools are all this same loop of ears, brain and mouth. Build it once here and every voice AI demo stops being magic. (Heads up: this one needs an Anthropic API key and costs a few cents to run.)

Start with the voice loop →

Part 1 — Ears

1

Ears, brain, mouth

A voice agent is three programs in a loop: speech-to-text, a language model, text-to-speech. The map, and your setup.

2

Ears: speech to text

Record from the mic with push-to-talk and transcribe it locally with Whisper. Free, offline, and about a third of a second.

Part 2 — Brain and mouth

3

Brain: Claude, tuned for talking

Send the transcript to Claude with a system prompt written for the ear: short, plain sentences with no markdown.

4

Mouth: text to speech

Speak the reply with your computer's built-in voice, then close the loop. You now have a working voice agent.

Part 3 — Make it feel live

5

Speak while it thinks

Stream Claude's reply and start talking at the first full sentence, so the silence no longer grows with the length of the answer.

6

Latency, cost & where next

Where each fraction of a second goes, what a conversation costs, and the upgrades that turn a demo into a product.