Gemini Live API: Sparse transcription updates and freezing

Google Cloud Build with AI AI APIs
1

Greetings,

I am working on an app that uses gemini-2.5-flash-native-audio-preview-09-2025 for real-time captions. I believe audio is streaming properly to the model, but the model occasionally sends partial captions, or it stops sending transcription updates entirely for 5-10 seconds, then resumes, sometimes with garbled/non-English text. The issues usually begin ~30-60 seconds after the start of a session. I am using 50msec audio segments.

  1. Is this a known issue or limitation with gemini-2.5-flash-native-audio-preview-09-2025?
  2. Are there any known workarounds?
  3. Are there recommended configuration settings that may eliminate these issues or perform better?
  4. What are the recommended chunk size and buffering strategies?
  5. Is there a better model or API approach?

Thanks in advance.

-John

2

Hello @John_Lange,

From Live API Best Practices:

  • Chunk Size and Latency: Send audio in chunks of 20ms to 40ms.

Also,

You have to enforce its Language Specification and System instruction design (example).

If you set "interrupted": true, any audio from the client will cut the Live Streaming current output.

Last, be sure that you’re using a region that is as close as possible from your app.

3

We’re seeing the same kind of delay(10-20 seconds) with Gemini Live on gemini-live-2.5-flash-native-audio, even with thinking disabled, so adding our setup for context.

Our setup

  • Model: gemini-live-2.5-flash-native-audio

  • SDK: @google/genai ^1.39.0 (Node.js)

const config = {
  systemInstruction: "<string>",  // one system prompt; we use a single text systemInstruction
  tools: [geminiTools],           // function declarations for tool/function calling
  generationConfig: {
    maxOutputTokens: 4096,
    thinkingConfig: {
      thinkingBudget: 0           // thinking disabled
    },
    temperature: 0.5
  },
  responseModalities: [Modality.AUDIO],
  outputAudioTranscription: {},
  realtimeInputConfig: {
    activityHandling: ActivityHandling.NO_INTERRUPTION
  },
  contextWindowCompression: { slidingWindow: {} },
  sessionResumption: resumptionHandle ? { handle: resumptionHandle } : {}
};

ai.live.connect({
  model: 'gemini-live-2.5-flash-native-audio',
  config,
  callbacks: { ... }
});

am i missing anything, or this is expected with this model, we are sending audio chunk of 100ms.