Does Apigee support compression/de-compression with GZIP/deflate compression?

Google Cloud Apigee
1

We have a requirement to support compression/de-compression on an Apigee bundle.

I found a link on Apigee specifically below help page:

http://apigee.com/docs/api-services/reference/endpoint-properties-reference

I had a couple of queries around this:

  1. Does it actually compress/decompress when I send traffic to Target endpoint? Or does it support the headers related to compression/decompression only?

  2. In the scenario, where I get gzip/deflate compressed data, is it possible for my proxy to uncompress this data before sending it to backend target?

  3. In the same scenario as above, Can I send the compressed data as is to target endpoint

Any more details like how to validate and test this would be great.

3 Likes
2

Hi,

Apigee Edge takes care of all compression automatically. As the document you linked to says, by default it will use the compression requested by the client, but you can change that using the target endpoint configuration.

The message is uncompressed when executing the request and response flows – you never will see compressed data, and this all happens automatically.

You can test this using a test target like httpbin.org. The endpoint http://httpbin.org/gzip returns gzipped data, and you can see what kind of compression was sent to it.

Mike

3 Likes
3

Thanks @Mike Dunker. The HTTPBin link you posted seems to be “getting” data. Is there a way to do this when I am “posting” data? ie. Verify that the message I am posting to Apigee goes to the target in a compressed format.

Thanks,

Girish

4

Hi Girish,

You can test this by using another proxy as a test target. Send a request via curl to a proxy and manually set the Content-Encoding header:

curl -X POST -H "Content-Encoding: gzip" "http://{myorg}-test.apigee.net/gziptest" -d '{ "test": "a" }'

Edge will expect gzip’d data. Since curl didn’t gzip the data, you’ll see the following error from Edge:

{"fault":{"faultstring":"Failed to Decompress","detail":{"errorcode":"messaging.adaptors.http.DecompressionFailure"}}}

Use the Edge proxy as the target. Trace the target proxy, and if you can see the Content-Encoding header set to gzip and can see the payload, then Edge was able to uncompress the data, and the data came in correctly gzip’d.

Note that you typically don’t want to call back into an Apigee proxy from another Edge proxy in the same org. There are performance implications – see the conversation about proxy chaining here: http://community.apigee.com/questions/1260/api-chaining.html . Just for testing is OK.

Mike

1 Like
5

Thanks @Mike Dunker. I was successfully able to test the compression and decompression with the “right” headers and “route rules”.

Thanks,

Girish

6

On a side note - be aware that cached responses are stored in the format in which they were received… so if your cache keys do NOT include accept-headers you may run into a case where you are serving a gzipped payload to a client that did not specifically accept such a payload. As well, if for performance reasons (large payloads/high latency clients) you want to be certain to serve a compressed response, be aware of this behavior.

1 Like
7

Thanks for the useful note. Better now than scratch my head later :slight_smile:

8

What if a backend always responds with gzip, but the use-case is for Apigee to always respond with the decompressed response?

I have tried adding none to the Proxy Endpoint as well as Target Endpoint to no avail.

In all cases, the response come back as gzipped.

Thanks.

1 Like
9

Same here, my node app gzip it and looks like Apigee gzip again before it got returned to client. If I curl url | gunzip | gunzip I see what is expected.

10

I’m seeing the same thing @Louis.lim5 - someone in this thread: gzip problems implies it may be a known problem with a fix coming. Don’t suppose you’ve found a solution already?

11

Hello

Louis-Etienne.Dorval

I think what might work for you in this case is,

Do not enable/set compression in your nodejs code, so nodejs returns uncompressed response.

In Edge Proxy - in the response flow - Add a header ‘Content-Encoding: gzip’ to your response using “AssignMessage” policy

Let me know if this workaround works for you

Thanks,

Mukundha

12

For the records, my question was basically: I’m using the NodeJS integration with Apigee Edge and the compression doesn’t work.

Thanks @Mukundha Madhavan it’s working perfectly now !

13

if the response received from System SouthBound to APIGEE is uncompressed , can we configure APIGEE to compress the response before its sent to the client?

14

yes , you can compress response .

You have to add header in response flow , attach below Assign Message in response flow -

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage async="false" continueOnError="false" enabled="true" name="AM-Set-Compress-Response">
    <DisplayName>AM-Set Compress Response</DisplayName>
    <Add>
        <Headers>
            <Header name="Content-Encoding">gzip</Header>
        </Headers>
    </Add>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
    <AssignTo createNew="false" transport="http" type="request"/>
</AssignMessage>
2 Likes
15

@Sartaj Singh Sisodiya, Do you know a way to decompress the response using assign message policy? Our backend always responds with a gzipped response.

16

In the case where a backend target returns compressed data and you want to un-compress it before sending to client, you can do so easily by using an Assign Message in the response flow.

For example using httpbin.org/gzip as the backend for testing.

curl -i [https://ORG/proxy/gzip](https://ORG/proxy/gzip) -H "Accept:application/json"

Add the following condition in the response flow:

<Response>
    <Step>
        <Condition>(request.header.accept = "application/json") and (response.header.content-encoding = "gzip")</Condition>
        <Name>AM-ZipResponseInflate</Name>
    </Step>
</Response>

Then the Assign Message policy is:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage async="false" continueOnError="false" enabled="true" name="AM-ZipResponseInflate">
    <DisplayName>AM-ZipResponseInflate</DisplayName>
    <Set>
        <Headers>
            <Header name="Content-Type">application/json</Header>
        </Headers>
        <Payload contentType="application/json">{response.content}</Payload>
        <StatusCode>200</StatusCode>
        <ReasonPhrase>Inflated Response</ReasonPhrase>
    </Set>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
    <AssignTo createNew="true" transport="http" type="response"/>
</AssignMessage>

NOTE: the use of createNew=“true”

17

Hi @Mike Dunker ,

Apigee documentation says , for Transport Property, Compression.Algorithm supported values are gzip,deflate and none.

Does that mean Apigee dont support br i.e. compression using Brotli algorithm?

We have a requirement where we need to support all compression algorithms (Accept-Encoding header - HTTP | MDN), so checking if we need to explicitly do something to support br .

Thanks,

Anoop

18

Hi @kurtkanaskie Thanks for your information, I have some question need your help.

As we known, By default To prevent memory issues in Edge, message payload size on the Router and Message Processor is restricted to 10MB. Exceeding those sizes results in a protocol.http.TooBigBody error( https://docs.apigee.com/private-cloud/v4.18.01/set-message-size-limit-router-or-message-processor ).

also, Apigee Edge takes care of all compression automatically. by default it will use the compression requested by the client.(https://www.googlecloudcommunity.com/gc/Apigee/Does-Apigee-support-compression-de-compression-with-G… ).

Could you please help clarify:

• Are these limits checked before or after encoding of content? (e.g. gzip)?
• If backend provides an uncompressed body and client supports compression, does API Gateway compress the body in-flight? If so, is the 10MB limit enforced on the compressed or un-compressed body?
• If backend service provides a compressed body and appropriate Content-Encoding header, does API Gateway enforce the 10MB limit on the compressed or un-compressed body?
Any more details like relevant reference documents would be appreciate. Thank you.

19

Great questions, I don’t know off hand. I suspect anytime the “body.buffer” size exceeds 10 Meg it would trigger an error.

For example, client sends zip request of 8 Meg, proxy processes the payload using policies and the compressed request is un-compressed and exceeds 10 Meg. I would think that would trigger an error.

However, if the proxy is a simple passthrough of an 8 Meg compressed request, it may not trigger the error.

The only way to know for sure would be to determine the behavior through testing.

20

Thank you, kurtkanaskie