Speak while it thinks

The agent works, but the silence after each question drags. The fix is the most useful trick in voice AI: don't wait for the whole reply. Claude writes its answer a few words at a time, so as soon as the first sentence is finished, start saying it. By the time the voice gets to the end of that sentence, Claude has written the next one.

Three pieces make this work: stream the reply, cut it into sentences as it arrives, and speak those sentences on a separate thread so the talking doesn't block the streaming.

1. Stream the reply

Instead of messages.create, which returns once the reply is complete, use messages.stream. Its text_stream gives you the text piece by piece as Claude writes it:

with client.messages.stream(
  model="claude-opus-5",
  max_tokens=16000,
  system=SYSTEM,
  output_config={"effort": "low"},
  messages=messages,
) as stream:
  for chunk in stream.text_stream:
      print(chunk, end="", flush=True)   # "Mount" " Fuji is" " the tallest" ...
  reply = stream.get_final_message()     # the complete message, for the history

text_stream yields only the text Claude is writing, so any thinking blocks are skipped for us. get_final_message() gives back the full reply once the stream ends, in the same shape as before, so the history code doesn't change.

2. Cut the stream into sentences

The chunks don't line up with sentences. One might be " meters. It's", the end of one sentence and the start of the next. So we add each chunk to a buffer and split the buffer wherever a sentence ends:

import re

SENTENCE_END = re.compile(r"(?<=[.!?])\s+")

That pattern matches whitespace that follows a ., ! or ?, which is the gap between two sentences. Splitting on it gives a list where every item except the last is a finished sentence. The last item is the sentence still being written, so it goes back into the buffer:

buffer = "about 3,776 meters. It's also"
*finished, buffer = SENTENCE_END.split(buffer)
# finished == ["about 3,776 meters."]      -> speak it now
# buffer   == "It's also"                 -> keep waiting for the rest

Decimals like 3.5 don't get split, because the . isn't followed by a space.

3. Speak on a separate thread

If the loop called speak() directly, it would stop reading the stream until each sentence had been spoken, and we'd be back to waiting. So the sentences go into a queue, and a second thread, the speaker, takes them out and says them one at a time:

import queue
import threading

def speaker(sentences):
  while (sentence := sentences.get()) is not None:
      speak(sentence)

The queue is the hand-off point between the two threads. The streaming loop puts sentences in as fast as Claude writes them, and the speaker takes them out as fast as the voice can say them. None is our signal that the reply is over.

Put it together

Replace think() with think_and_speak():

def think_and_speak(text):
  messages.append({"role": "user", "content": text})

  sentences = queue.Queue()
  voice = threading.Thread(target=speaker, args=(sentences,))
  voice.start()

  buffer = ""
  with client.messages.stream(
      model="claude-opus-5",
      max_tokens=16000,
      system=SYSTEM,
      output_config={"effort": "low"},
      messages=messages,
  ) as stream:
      for chunk in stream.text_stream:
          print(chunk, end="", flush=True)
          buffer += chunk
          *finished, buffer = SENTENCE_END.split(buffer)
          for sentence in finished:
              sentences.put(sentence)      # speak it while Claude keeps writing
      reply = stream.get_final_message()

  if buffer.strip():
      sentences.put(buffer)                # whatever's left after the last "."
  sentences.put(None)                      # tell the speaker we're done
  voice.join()                             # wait until it finishes talking
  print()

  messages.append({"role": "assistant", "content": reply.content})

And in the main loop, the last three lines become two:

print("Claude: ", end="")
      think_and_speak(heard)

voice.join() matters. It makes think_and_speak wait until the last sentence has been spoken, so the agent never starts listening while it's still talking (and ends up transcribing its own voice).

What changed

We tested this loop with a simulated three-sentence reply arriving at a realistic token pace. The first sentence started speaking 0.25 seconds into the reply, while the other two sentences were still being written. The waiting version couldn't start until the whole reply was done.

speak @ 0.25s: 'Mount Fuji is the tallest, at about 3,776 meters.'
speak @ 2.07s: "It's also an active volcano!"
speak @ 3.05s: 'Want to know when it last erupted?'

The longer the answer, the bigger the win: with waiting, the silence grows with the length of the reply, but with streaming it's always just the time to the first sentence.

Next: where the rest of the time goes, and where to take the agent from here.