It’s often helpful to be able to failover between backend target servers in Apigee proxies, either using a Circuit Breaker strategy after a certain numer of errors has been reached, or just by having a simpler primary and secondary server, with the secondary server only being used if the primary server fails to successfully respond to a request.
Circuit Breakers
Circuit Breaking is a common strategy of trying a primary target until a certain threshold of errors is reached, and the circuit is broken and the backup or failover options are then tried. You can easily configure a circuit breaker using the Apigee Load Balancing configuration, see resources here:
Load balancing across backend servers
YouTube guide to circuit breaking with Apigee
Failover
Sometimes target backend load balancing and circuit breaking isn’t possible though, for example when backend servers need parameters or have dynamic URLs that can’t be configured with target servers.
In this case it’s possible switch to ServiceCallout policies and a no-target proxy. The ServiceCallouts can be called in a series in the PostFlow and conditions for the failover steps.
In the screenshot above, there is a SC-CallGeminiPrimary and SC-CallGeminiSecondary policy, with the SC-CallGeminiSecondary only being called if the variable geminiResponse.status.code != 200, which happens if the primary gemini call fails for whatever reason. There is no target configured, so all target handling is done by the service callout policies.
Both SC policies have the same configuration, with the difference that the SC-CallGeminiSecondary calls a backup Gemini endpoint in europe-west4.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ServiceCallout continueOnError="true" enabled="true" name="SC-CallGeminiPrimary">
<DisplayName>SC-CallGeminiPrimary</DisplayName>
<Properties/>
<Request clearPayload="false" variable="myRequest">
<IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
<Set>
<Verb>POST</Verb>
<Payload contentType="application/json">
{request.content}
</Payload>
</Set>
</Request>
<Response>geminiResponse</Response>
<HTTPTargetConnection>
<Properties/>
<URL>https://europe-west1-aiplatform.googleapis.com/v1/projects/{organization.name}/locations/europe-west1/publishers/google/models/gemini-2.5-flash:generateContent</URL>
<Authentication>
<GoogleAccessToken>
<Scopes>
<Scope>https://www.googleapis.com/auth/cloud-platform</Scope>
</Scopes>
</GoogleAccessToken>
</Authentication>
</HTTPTargetConnection>
</ServiceCallout>
After the primary (and if needed secondary) steps have been run, the AM-SetResponse policy sets the result to the final response content.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage continueOnError="false" enabled="true" name="AM-SetResponse">
<DisplayName>AM-SetResponse</DisplayName>
<Properties/>
<Set>
<Payload>
{geminiResponse.content}
</Payload>
</Set>
<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
<AssignTo createNew="false" transport="http" type="request"/>
</AssignMessage>
Conclusion
Both approaches have their usage - target load balancing if the backends are fixed hosts and fit into a target configuration, or failover service callouts if the targets have dynamic URLs (for example for different Gemini regions, as in the example above).
The sample proxy is available here: GitHub - api-integration-samples/apigee-failover-proxy: This sample proxy shows how to do failover with service callouts if target service load balancing can't be used. .
