Apigee Hybrid 1.16.7 - Environment stuck in creating state. Runtime contract not available in synchronizer

[madhu_konduru]

Hi Team,

I’m facing an issue in Apigee Hybrid 1.16.7 where the environment dev-env remains in Creating state.

Current Symptoms:

1. apigee-runtime pod: 0/1 Running
2. apigee-synchronizer pod: 0/1 Running

Runtime readiness fails with

Error: VersionLoadProbe failed due to Version not loaded by Message Processor

Synchronizer readiness fails with

Error: Runtime Contract not yet available in synchronizer

Verified

1. Organization (PII Removed by Staff) exists and is ACTIVE.
2. Environment dev-env exists and is ACTIVE.
3. Proxy deployment exists in dev-env.
4. Workload Identity and IAM permissions are configured correctly.
5. GenerateAccessToken requests succeed.
6. MART is reachable from Synchronizer.
7. runtimeConfig API is available.

Despite this, Synchronizer never receives the runtime contract, so Runtime cannot load an active version and the environment remains stuck in Creating.

Has anyone encountered a situation where the runtime contract is not generated/published to Synchronizer even though the environment and deployments exist?

Any suggestions would be appreciated. Thanks.

[AlexET]

Hi @madhu_konduru, and welcome to the Apigee community!

Thank you for sharing such a detailed breakdown of your Apigee Hybrid setup and the specific error codes you are encountering.

We have seen your question, and we encourage our community members to share their knowledge. While the community dives in, you might also want to review the official Apigee Hybrid troubleshooting documentation for additional insights on runtime contract availability.

Thanks again for being a part of our community!

[madhu_konduru]

Hi @AlexET Thanks for your reply.

Based on the troubleshooting documentation, I reviewed the Synchronizer and Runtime logs and found that the Synchronizer is unable to fetch the runtime contract from the Apigee Control Plane. The logs report errors such as “failed to get contract from control plane”, “Upstream contract poll returned 404”, and “Runtime Contract not yet available in synchronizer”. Due to this, the Runtime components are unable to load the required version and are failing readiness checks.

Could you please take a look and help us understand what might be causing the contract synchronization failure? Any guidance or recommendations to resolve the issue would be greatly appreciated.