How does HTTP streaming on Edge really work?

Google Cloud Apigee
1

Hi, I have a question about how HTTP Streaming works on Edge. The docs contradict what some other community posts suggest so I’d like to clear it up if possible.

In the target response flow shown below, when will XMLtoJSON-Policy be executed?

a. Only if the response is > 3MB (is this the MP buffer size?)

b. Never (because streaming is explicitly enabled)

The docs currently suggest “b”. However, when I call this yahoo weather API it has a small response size, and the policy does get executed. This suggests “b” is actually incorrect.

So then, how and when does Edge determine if the payload size is > 3MB? Does it look at the Content-Length Header on the response or something else? If I try to trick Edge by setting a very high Content-Length header right before the XMLtoJSON policy, it still gets executed. So, maybe something else is going on?

<TargetEndpoint name="default">
    <Description/>
    <HTTPTargetConnection>
        <URL>http://weather.yahooapis.com</URL>
        <Properties>
            <Property name="response.streaming.enabled">true</Property>
        </Properties>
    </HTTPTargetConnection>
    <PreFlow name="PreFlow">
        <Request/>
        <Response>
            <Step>
                <Name>XMLtoJSON-Policy</Name>
            </Step>
        </Response>
    </PreFlow>
    <PostFlow name="PostFlow">
        <Request/>
        <Response/>
    </PostFlow>
</TargetEndpoint>
1 Like
2

@wwitman In terms of expected behaviour, option b is correct. When streaming is enabled for an endpoint, all policies which require payload processing should be skipped. However there’s a bug in the current system due to which this behaviour is not working as expected and you see the policy getting executed. When you bump up the response size to exceed the buffer limit (which is probably set to 10MB in cloud), you might see a “toobigbody” error.

You can find more info here:

[https://apigeesc.atlassian.net/browse/APIRT-1525](https://apigeesc.atlassian.net/browse/APIRT-1525)
1 Like
3

Thanks, @argya!