Choppy audio when streaming Google Cloud Text-to-Speech to Asterisk for real-time bidirectional translation

Google Cloud Build with AI Conversational Agents (Dialogflow CX)
1

I’m building a real-time bidirectional phone translation system using Google Cloud.

I use Google Speech-to-Text (streaming) to transcribe one caller, Translation API to translate, and Text-to-Speech (LINEAR16, 8000Hz) to synthesize audio for the other caller. The audio is streamed to Asterisk via AudioSocket in 320-byte frames every 20ms.

The translation logic works correctly in both directions. The problem is audio quality: the synthesized Text-to-Speech audio plays back choppy and slow — words are cut up and stretched.

A one-directional voice bot using nearly identical code sounds perfectly smooth. The choppiness only appears when handling two simultaneous streams in the same Node.js process.

I have already verified: sample rate is 8000Hz, codec is slin on both legs, CPU is near 0%, and Text-to-Speech latency is about 1.5 seconds. I also tried different frame sizes, jitter buffering, and clock-based pacing — with no improvement.

My questions:

  1. Is there a recommended way to stream Google Text-to-Speech audio for two simultaneous real-time directions without choppiness?
  2. Could the issue be related to how Google Text-to-Speech streaming output is chunked or paced?
  3. Is there a better Google Cloud approach (for example, a different API or streaming method) for low-latency bidirectional voice translation?

Thank you.

2

Important update — I found a key clue:

The choppiness ONLY happens while BOTH legs (agent + customer) are connected simultaneously. The moment EITHER leg hangs up, the remaining leg instantly hears the translated audio perfectly clear and smooth — no choppiness at all.

Both legs are handled in the same Node.js process, each with its own AudioSocket TCP connection (port 9090), and each sends 320-byte frames every 20ms via setInterval. CPU stays near 0% even during the overlap.

This strongly suggests the issue is contention between the two simultaneous AudioSocket streams, not TTS quality or CPU.

Is there a known issue with handling two concurrent AudioSocket connections in one process? Should each leg run in a separate worker thread/process, or is AudioSocket unsuitable for bidirectional real-time audio — should I switch to ARI external media instead?