Unexpected HTTP Request Duplication in Application Integration

Google Cloud Integration Services
,
1

Hi,

We are observing an intermittent and unexplained duplication of HTTP requests within an Application Integration flow.

The Application Integration utilizes a reusable sub-integration to manage all HTTP requests via a configured HTTP Connector. Crucially, this sub-integration does not implement any retries.

For the POST requests, particularly the second one, we are sporadically seeing a second, identical request being generated after the first call. This duplication occurs even when the first call is successful (returns a 200 OK) or when it returns an error.

  • Observation Conflict: Connector logs confirm that two distinct requests are being sent to the target system. However, the Application Integration’s execution log only records a single sub-integration/HTTP call.

  • Analysis: There is no logic within our Application Integration that dictates or explains the generation of a second request. Furthermore, a retry mechanism should not be activated following a successful 200 OK response.

To prevent data loss, Http Connector has this internal logic to an “at-least-once” delivery guarantee and retries the request. Evidence suggests the connector is timing out internally and reattempting the POST request despite the original GET request success (200 OK). This behavior is causing idempotency issues on the backend.

A more robust architecture is required to prevent these internal timeouts and subsequent retry scenarios, and right now we are trying to find a different solution to solve this problem.

The actual architecture involves a flow starting with the Client System, which sends a request to Apigee. This request, in turn, activates Application Integration. Application Integration uses an Http Call Handler and an Http Connector to establish communication with Envoy, which is then responsible for routing the traffic to the final Target Backend System.

Your input is highly valued.

Cheers.

2

Hi @pedrofrzmatos,

Thanks for reaching out! We’re going to make sure your questions gets noticed.

While we wait for the community to chime in, feel free to check out the Application Integration Knowledge Hub and our On-Demand Video Library for additional curated content by Googlers.

Have a nice day :slight_smile:

3

@pedrofrzmatos This is expected behaviour and connector cannot guarantee only once delivery because of multiple distributed systems involved. Same has been documented in our public documentation: https://docs.cloud.google.com/integration-connectors/docs/connectors/http/configure#exp6

1 Like
4

Hi @gowthamnagarajn,

Thanks for reaching out. I fully understand that part, but taking in consideration our architecture we need an extra step to validate those duplicate request, and the solution right now that we are under investigation is having an apigee proxy for caching after the HTTP connector and before taget end system. If you have any insight on that please share it.

Side note, I understand that design choice and behaviour of the connector, but if we really want to do retries we can simply do it in application integration. Right now, for us this is causing those duplicate requests.

Cheers.