When building voice-native conversational agents, the leap from a cool prototype to a production-ready enterprise system requires more than just low latency and natural-sounding voices. Real-world applications demand rigor: you need to know exactly what was said (both by the user and the model) for auditing purposes.
In this post, we’ll explore how to build a production-grade information-gathering agent using the Gemini Live API and the Agent Development Kit (ADK). We will focus on persisting raw audio streams for session audits
The Challenge: Accuracy in the Real World
Imagine an automated assistant tasked with collecting user information. In our project, we built an agent named “Alex” using the gemini-live-2.5-flash-native-audio model. Alex’s primary goal is to politely gather a user’s First Name, Last Name, and Email Address, explicitly asking the user to spell them out to ensure accuracy.
When your business logic relies on exact character matches—like spelled-out names or alphanumeric email addresses—capturing the exact interaction is no longer just a UX concern; it is the foundation of the application. If a customer disputes a transaction or an interaction fails, developers need access to the original audio to diagnose the failure and understand exactly what was said.
High-Level System Architecture
To meet these robust audit requirements, we designed an architecture that seamlessly transitions from real-time streaming to secure, long-term storage.
Throughout the active call, the conversational AI is powered by the Gemini Live API, while the ADK runs quietly in the background to capture every fragment of the audio stream. The most critical part of this workflow triggers at the end of the call: when the user hangs up or the WebSocket connection disconnects, a background task automatically triggers within the server to initiate our post-processing pipeline.
This pipeline takes the raw, fragmented audio captured during the session, sorts it by timestamp, resamples it, and stitches it together into a single cohesive .wav file. Finally, it automatically moves the finalized audio file into a Google Cloud Storage (GCS) bucket (or local disk). This ensures the interaction is securely archived and readily available for compliance or support reviews.
To understand how this continuous capture and final upload to GCS is made possible, let’s go deeper under the hood of the ADK to see how it handles the raw data stream.
Session Auditing and Persisting Live Blobs
In a standard text-based LLM application, logging the transcript is usually sufficient. In voice applications, however, the raw audio contains critical context: tone, background noise, and pronunciation nuances.
The ADK makes archiving these raw data streams straightforward. Enabling the save_live_blob=True parameter in your RunConfig triggers an internal background task designed to intercept and archive the raw data streams during an active session.
Here is an example of how simply this is configured during session initialization:
LIVE_AGENT_RUN_CONFIG = RunConfig(
# Configure real-time audio input behavior
realtime_input_config=types.RealtimeInputConfig(
automatic_activity_detection=types.AutomaticActivityDetection(
disabled=False,
# Adjust sensitivity for detecting when the user starts or stops speaking
start_of_speech_sensitivity=types.StartSensitivity.START_SENSITIVITY_LOW,
end_of_speech_sensitivity=types.EndSensitivity.END_SENSITIVITY_LOW,
prefix_padding_ms=20,
silence_duration_ms=150,
)
),
# Set the language and the specific voice persona for the agent
speech_config=types.SpeechConfig(
voice_config=types.VoiceConfig(
prebuilt_voice_config=types.PrebuiltVoiceConfig(
voice_name="Achird",
)
),
language_code="en-US",
),
session_resumption=types.SessionResumptionConfig(transparent=True),
streaming_mode=StreamingMode.BIDI,
response_modalities=["AUDIO"],
input_audio_transcription=types.AudioTranscriptionConfig(),
output_audio_transcription=types.AudioTranscriptionConfig(),
# Enable raw audio capturing for session auditing
save_live_blob=True
)
Under the hood, the ADK handles this through a well-defined architecture:
-
The “Recorder” Logic: This module serves as the session engine, intercepting audio fragments and redirecting them. It monitors the save_live_blob boolean to cache outgoing responses.
-
The “Buffer” Management: This handles volatile memory storage, sequentially appending raw chunks before concatenating them and sending them to the artifact service.
-
The “Filing Cabinet”: This executes the physical storage operations, synchronously generating directories and writing raw .l16 or .pcm files alongside their metadata.
Stitching the Pieces Together: From Raw Blobs to Auditable Audio
While the “Filing Cabinet” handles the heavy lifting of saving the data, the reality of live streaming is that audio arrives in asynchronous, fragmented chunks. The model’s responses are stored as 24kHz .pcm files, while the user’s input is stored as 16kHz .l16 chunks.
For an auditor or support agent investigating a disputed interaction, sifting through hundreds of isolated milliseconds of audio isn’t practical. We need a cohesive, chronological playback file.
To solve this, we implemented a dedicated python function, which performs three crucial operations:
-
Timestamp Sorting: It extracts the millisecond timestamps generated by the ADK from the filenames, ensuring the conversational order is perfectly preserved.
-
Resampling: It dynamically resamples the model’s 24kHz output down to match the user’s 16kHz input using linear interpolation.
-
Merging: It concatenates the sorted, resampled audio chunks into a single, unified .wav file.
To make this completely hands-off in a production environment, we bound this conversion logic directly to the finally block of the WebSocket endpoint in our main.py server. When a client disconnects or the call hangs up, the system automatically sweeps the artifact directories for that specific session and produces the final audit-ready audio file on the fly. Once the file is generated, the background task offloads it directly to a Google Cloud Storage (GCS) bucket, tying up the pipeline and ensuring secure, long-term archiving.
The Takeaway
Moving a voice agent from a weekend prototype to an enterprise-grade application means planning for the worst-case scenario. When an interaction fails, a transaction is disputed, or a user’s alphanumeric information is recorded incorrectly, you cannot afford to have a “black box” system.
By leveraging the ADK’s native real-time blob saving and implementing automated post-processing to stitch those raw audio chunks into a playable file, you can build conversational AI that isn’t just fast and fluid—but fully transparent and auditable.
While capturing the audio solves the auditing challenge, you also need to guarantee the model correctly interprets and generates the speech itself. For a detailed breakdown of how to rigorously evaluate Speech-to-Text (STT) and Text-to-Speech (TTS) pipelines in these production environments, check out my full write-up: A Production-grade dive into Gemini Live with ADK: Verifying Speech-to-Text and Text-to-Speech.
Conclusion
Taking a voice agent from development to production requires treating audio as a first-class data citizen. By leveraging the ADK’s save_live_blob feature, developers can maintain a perfect audit trail of all customer interactions.
You can explore the source code for this full use-case implementation on GitHub.
Ready to build your own verifiable voice experiences? Check out the Gemini Live API documentation and start instrumenting your conversational agents today!