Play audio on Google Phone Gateway

Google Cloud Build with AI Conversational Agents (Dialogflow CX)
1

I have implemented a page fulfillment and a webhook on that fulfillment that does text to speech generation. Audio file generated is stored in Storage bucket. The wav file plays well on Google Phone Gateway if I configure “Pre recorded audio” directly using “Agent responses” and specify google cloud (gc) based path of the wav file.

However, no audio gets played if I REMOVE pre-recorded audio and instead only configure the webhook on fulfillment. Webhook returns the following JSON with audio_url having both HTTPS and GC based URLs tried.

ssml = f'<speak><audio src="{audio_url}"></audio></speak>'
       
        # Build the fulfillment response
        fulfillment_response = {
            "fulfillmentResponse": {
                "messages": [
                    {
                        "text": {
                            "text": [ssml]
                        }
                    }
                ]
            }
        }
2

@ngargi009 for pre-recorded audio, the format is Ulaw 8 hrz, I got this same problem trying to get pre-recorded audio and I created a converter that you can deploy on your local as react, docker, simple html, or access it on url. https://mizar-contasti.github.io/wav-to-ulaw-simple/

3

Hi @ngargi009 ,

Welcome to the Google Cloud Community!

The challenge you’re facing likely stems from how your SSML response is processed by the webhook versus the “Pre-recorded audio” configuration. Here are some suggestions and approaches you can try:

  • Ensure the audio_url variable in the code above contains the correct path to your audio file. Double-check for typos.
  • Ensure the Dialogflow service account has the necessary roles and permissions (e.g Storage Object Viewer).
  • If you’re using HTTPS URLs, make sure the audio file is publicly accessible. Check that the server hosting the file returns the correct Content-Type header (e.g., audio/wav). Test the URL by opening it in a browser to ensure the audio plays correctly.
  • Test your URL (both gs:// and https://) independently:
    • gs:// URLs: Use the gsutil ls gs://your-bucket-name/your-audio-file.wav command in the Google Cloud SDK to ensure the file exists and you have access to it.
    • https:// URLs: Open the URL in a web browser. The audio should either start playing or download immediately. Check your browser’s developer tools (Network tab) to verify the correct Content-Type header is returned.
  • Test your Dialogflow agent using the Dialogflow simulator in the console. The simulator provides valuable error messages that can help pinpoint the problem.

You may also check these documentation for additional information about Dialogflow webhook audio responses:

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

4

Is the JSON structure I have added in question correct?

5

If I directly add wav file as pre-recorded audio under agent responses, it works. So its not a problem with file format.

6

Ohh I thought was this