503 for /versions when deploying Docker image to flexible App Engine

Hi,

Our organization has several Google Cloud projects.

For one of these projects we experience an error running gcloud app deploy in all deployments with Docker image to flexible App Engine configuration.

Other projects work fine with similar configuration.

Is there something in the Google Cloud SDK preventing us from deploying?

Command
$ gcloud app deploy --image-url=[europe-docker.pkg.dev/[project]/images/backend:4.0.1582](http://europe-docker.pkg.dev/[project]/images/backend:4.0.1582) --version=4-0-1582 ./app.yaml --verbosity debug

app.yaml

service: backend-api
env: flex
runtime: custom

resources:
  cpu: 2
  memory_gb: 4
  disk_size_gb: 30

automatic_scaling:
  min_num_instances: 3
  max_num_instances: 9
  cool_down_period_sec: 180
  cpu_utilization:
    target_utilization: 0.12
  target_concurrent_requests: 17

liveness_check:
  path: "/_ah/start"
  check_interval_sec: 10
  timeout_sec: 8
  failure_threshold: 3
  success_threshold: 1
  initial_delay_sec: 360

readiness_check:
  path: "/_ah/warmup"
  check_interval_sec: 60
  timeout_sec: 10
  failure_threshold: 5
  success_threshold: 1
  app_start_timeout_sec: 360

Output

DEBUG: (gcloud.app.deploy) HttpError accessing <[https://appengine.googleapis.com/v1/apps/[project]/services/backend-api/versions?alt=json](https://appengine.googleapis.com/v1/apps/[project]/services/backend-api/versions?alt=json)>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 11 Aug 2025 10:59:13 GMT', 'server': 'ESF', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000', 'transfer-encoding': 'chunked', 'status': '503', 'content-length': '122', '-content-encoding': 'gzip'}>, content <{
  "error": {
    "code": 503,
    "message": "The service is currently unavailable.",
    "status": "UNAVAILABLE"
  }
}
>
This may be due to network connectivity issues. Please check your network settings, and the status of the service you are trying to reach.
Traceback (most recent call last):
  File "/usr/bin/../lib/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 998, in Execute
    resources = calliope_command.Run(cli=self, args=args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/../lib/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 934, in Run
    resources = command_instance.Run(args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
...
>
ERROR: (gcloud.app.deploy) HttpError accessing <[https://appengine.googleapis.com/v1/apps/[project]/services/backend-api/versions?alt=json](https://appengine.googleapis.com/v1/apps/[project]/services/backend-api/versions?alt=json)>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 11 Aug 2025 10:59:13 GMT', 'server': 'ESF', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000', 'transfer-encoding': 'chunked', 'status': '503', 'content-length': '122', '-content-encoding': 'gzip'}>, content <{
  "error": {
    "code": 503,
    "message": "The service is currently unavailable.",
    "status": "UNAVAILABLE"
  }
}

The logs from App Engine if this helps

(Version number is upgraded and different service name but the error is the same)

EDIT: Additional info.
The working project has:

locationId: europe-west

and the failing project has:

locationId: europe-west2

The problem appeared last Thursday August 7th. It was working fine uptil then.

Hi @Christian.L,

Did you already try waiting a few minutes and then redeploying? Sometimes, App Engine just needs a bit to catch up before it starts responding properly. If it still fails, check Cloud Logging to see the details.

If you see “No service found” or “permission denied,” it probably means your Endpoints service configuration hasn’t been deployed yet, or the Service Management/Cloud APIs are turned off. In that case, run gcloud endpoints services deploy and make sure those APIs are enabled.

If the log says the service exists but there’s no configuration for the version, find the correct service configuration ID, update your app.yaml with it, and redeploy. That should get things running again.

For the complete documentation, you may refer here.

We are experiencing the exact same issue (> 24hrs, us-central1) with projects that have not had any configuration changes. I don’t mean to hijack the thread, just wanted to add another data point.

The 503 troubleshooting link does not look to be relevant for this particular error message (“The service is currently unavailable.”), and the only log available is what has been previously added in this thread. Unfortunately that doesn’t offer much help for addressing this on our end.

We have been having the same issue for at least 48 hours.
We ran:
gcloud app --quiet deploy app.yaml --promote --stop-previous-version --verbosity=debug

We got:

ERROR: (gcloud.app.deploy) HttpError accessing <https://appengine.googleapis.com/v1/apps/elanza-production/services/default/versions?alt=json>: response: <{‘vary’: ‘Origin, X-Origin, Referer’, ‘content-type’: ‘application/json; charset=UTF-8’, ‘date’: ‘Tue, 12 Aug 2025 18:45:30 GMT’, ‘server’: ‘ESF’, ‘x-xss-protection’: ‘0’, ‘x-frame-options’: ‘SAMEORIGIN’, ‘x-content-type-options’: ‘nosniff’, ‘alt-svc’: ‘h3=“:443”; ma=2592000,h3-29=“:443”; ma=2592000’, ‘transfer-encoding’: ‘chunked’, ‘status’: ‘503’, ‘content-length’: ‘122’, ‘-content-encoding’: ‘gzip’}>, content <{
“error”: {
“code”: 503,
“message”: “The service is currently unavailable.”,
“status”: “UNAVAILABLE”
}
}

We also found this message (with the retries) in the log explorer / service account logs:

{
  "protoPayload": {
    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
    "status": {
      "code": 4,
      "message": "Server deadline (28.82012245937949s) expired."
    },
    "requestMetadata": {
      "callerIp": "52.205.186.45",
      "requestAttributes": {
        "time": "2025-08-12T18:43:06.319807Z",
        "auth": {}
      },
      "destinationAttributes": {}
    },
    "serviceName": "appengine.googleapis.com",
    "methodName": "google.appengine.v1.Versions.CreateVersion",
    "authorizationInfo": [
      {
        "resource": "apps/{project}/services/default/versions/20250812t184224",
        "permission": "appengine.versions.create",
        "granted": true,
        "resourceAttributes": {},
        "permissionType": "ADMIN_WRITE"
      }
    ],
    "resourceName": "apps/{project}/services/default/versions/20250812t184224",
    "serviceData": {
      "@type": "type.googleapis.com/google.appengine.v1.AuditData",
      "createVersion": {
        "request": {
          "parent": "apps/{project}/services/default",
          "version": {
            "id": "20250812t184224",
            "automaticScaling": {
              "cpuUtilization": {
                "targetUtilization": 0.7
              },
              "maxTotalInstances": 10,
              "minTotalInstances": 2
            },
            "resources": {
              "cpu": 1,
              "memoryGb": 4
            },
            "runtime": "vm",
            "betaSettings": {
              "cloud_sql_instances": "{project}:europe-west3:{project}",
              "module_yaml_path": "app.yaml",
              "vm_runtime": "nodejs"
            },
            "env": "flex",
            "flexibleRuntimeSettings": {
              "operatingSystem": "ubuntu22",
              "runtimeVersion": "20"
            },
            "entrypoint": {
              "shell": ""
            }
          }
        }
      }
    },
    "resourceLocation": {
      "currentLocations": [
        "europe-west3"
      ]
    }
  },
  "insertId": "1c7czd42x2",
  "resource": {
    "type": "gae_app",
    "labels": {
      "zone": "",
      "version_id": "20250812t184224",
      "project_id": "{project}",
      "module_id": "default"
    }
  },
  "timestamp": "2025-08-12T18:42:37.502763Z",
  "severity": "ERROR",
  "logName": "projects/{project}/logs/cloudaudit.googleapis.com%2Factivity",
  "receiveTimestamp": "2025-08-12T18:43:07.315453534Z"
}

Thanks for the suggestions.
However this is a consistent error we have started experiencing on an existing service and reproduced on a new service both in europe-west2.

We have waited and retried for several hours due to our understanding 503 errors can appear intermediately due to outages in Google internal services.

We are having exactly the same issue (since Aug 7 now in us-central1). We’ve spent 40+ person hours trying to find the issue, including back-and-forth with Google Support (we purchased a support contract just for this issue).

We just received a notice from Google Cloud support letting us know this is indeed an issue on their end. They promised they are working on it and they would update us no later than Aug 20.

1 Like

We’ve faced the same issue on our side and have reached out to Google Support.

Engineering was involved and it turns out the issue was on appengine side. We just confirmed this is now fixed for us. I suggest you try again now and see if this is resolved, otherwise don’t hesitate to open a support case in the Google Cloud Console.

We (Nate and I) can confirm that it started working for us now for at least one of our appengine services it was not working for before. With no change on our side.

Can confirm our services are deploying successfully now as well. Thanks @Nate and @devodev for the GCP support follow-ups.