Server Sent Events(SSE) support on ApigeeX

Google Cloud Apigee
1

Hello,

We are proxying the Azure Open AI endpoints via ApigeeX. When streaming is enabled on the Open AI request, Azure sends the response as SSE with content-type being text/event-stream. Even though the message structure and content-type is getting passed on correctly from ApigeeX to client, the Wait Time(TTFB) is taking too long indicating Apigee is not sending the first byte as soon as it receives from Azure endpoint, but rather buffering it and responding with all chunks at once in the end. I have also enabled both response and request streaming on proxy and target endpoints, but it did not fix the issue.

Has anyone proxied a SSE target with Apigee?

I have attached the screenshots showing the difference in behaviour.

Azure Wait time

rajeevyes1_1-1732622024559.jpeg
rajeevyes1_1-1732622024559.jpeg1544×228 30 KB

Apigee Wait Time

rajeevyes1_2-1732622058448.jpeg
rajeevyes1_2-1732622058448.jpeg1530×242 29.7 KB

2

Hi @rajeevyes1 , thank you for your question - If you can share more details about your use case or challenges, it will help others provide better insights.

We appreciate you engaging with the community and will keep an eye on this discussion to ensure it gets some attention. :wink:

1 Like
3

Hi @rajeevyes1 – SSE is not currently supported.

Please DM me here, stating what company you’re at and including a link to this post. Thanks.

1 Like
4

Hi @omidt , I have DMéd you.

1 Like