GKE Gateway controller + ClusterIssuer configured for http01 create a solver pod that works but with a HTTPRoute that does not attach to the gateway

Google Cloud Compute Infrastructure
1

Describe the bug:

I’m using the GKE Gateway controller on a GKE cluster with cert-manager installed via the official chart. I’m using:

  • 1 gateway in the integration namespace called integration looking like this:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-staging
    networking.gke.io/addresses: ""
    networking.gke.io/backend-services: /projects/000000000000/regions/europe-west3/backendServices/gkegw1-b35h-integration-gw-serve404-80-um3p00uxr7bf,
      /projects/000000000000/regions/europe-west3/backendServices/gkegw1-b35h-integration-gw-serve500-80-w8426ytw9ck4
    networking.gke.io/firewalls: /projects/000000000000/global/firewalls/gkegw1-b35h-l7-default-europe-west3
    networking.gke.io/forwarding-rules: /projects/000000000000/regions/europe-west3/forwardingRules/gkegw1-b35h-integration-integration-q41c4s983fgr
    networking.gke.io/health-checks: /projects/000000000000/regions/europe-west3/healthChecks/gkegw1-b35h-integration-gw-serve404-80-um3p00uxr7bf,
      /projects/000000000000/regions/europe-west3/healthChecks/gkegw1-b35h-integration-gw-serve500-80-w8426ytw9ck4
    networking.gke.io/last-reconcile-time: "2026-03-09T14:52:15Z"
    networking.gke.io/lb-edge-extensions: ""
    networking.gke.io/lb-route-extensions: ""
    networking.gke.io/lb-traffic-extensions: ""
    networking.gke.io/ssl-certificates: ""
    networking.gke.io/target-http-proxies: /projects/000000000000/regions/europe-west3/targetHttpProxies/gkegw1-b35h-integration-integration-os2neyue7uw5
    networking.gke.io/target-https-proxies: ""
    networking.gke.io/url-maps: /projects/000000000000/regions/europe-west3/urlMaps/gkegw1-b35h-integration-integration-os2neyue7uw5
    networking.gke.io/wasm-plugin-versions: ""
    networking.gke.io/wasm-plugins: ""
  name: integration
  namespace: integration
spec:
  addresses:
  - type: NamedAddress
    value: gke-gateway-address
  gatewayClassName: gke-l7-regional-external-managed
  listeners:
  - allowedRoutes:
      namespaces:
        from: All
    hostname: example.com
    name: http
    port: 80
    protocol: HTTP
  - allowedRoutes:
      namespaces:
        from: All
    hostname: example.com
    name: https
    port: 443
    protocol: HTTPS
    tls:
      certificateRefs:
      - group: ""
        kind: Secret
        name: tls-cert-example-com
        namespace: integration
      mode: Terminate
status:
  addresses:
  - type: IPAddress
    value: 1.1.1.1 (changed ofc)
  conditions:
  - lastTransitionTime: "2026-03-09T14:48:40Z"
    message: The OSS Gateway API has deprecated this condition, do not depend on it.
    observedGeneration: 40
    reason: Scheduled
    status: "True"
    type: Scheduled
  - lastTransitionTime: "2026-03-09T14:48:40Z"
    message: ""
    observedGeneration: 40
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2026-03-09T14:49:14Z"
    message: ""
    observedGeneration: 40
    reason: Programmed
    status: "True"
    type: Programmed
  - lastTransitionTime: "2026-03-09T14:49:14Z"
    message: The OSS Gateway API has altered the "Ready" condition semantics and reserved
      it for future use.  GKE Gateway will stop emitting it in a future update, use
      "Programmed" instead.
    observedGeneration: 40
    reason: Ready
    status: "True"
    type: Ready
  - lastTransitionTime: "2026-03-09T14:49:14Z"
    message: ""
    observedGeneration: 40
    reason: Healthy
    status: "True"
    type: networking.gke.io/GatewayHealthy
  listeners:
  - attachedRoutes: 0
    conditions:
    - lastTransitionTime: "2026-03-09T14:48:40Z"
      message: ""
      observedGeneration: 40
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    - lastTransitionTime: "2026-03-09T14:48:40Z"
      message: ""
      observedGeneration: 40
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2026-03-09T14:49:14Z"
      message: ""
      observedGeneration: 40
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2026-03-09T14:49:14Z"
      message: The OSS Gateway API has altered the "Ready" condition semantics and
        reserved it for future use.  GKE Gateway will stop emitting it in a future
        update, use "Programmed" instead.
      observedGeneration: 40
      reason: Ready
      status: "True"
      type: Ready
    name: http
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
  • 1 ClusterIssuer:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-staging
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    solvers:
    - http01:
        gatewayHTTPRoute:
          parentRefs:
          - group: gateway.networking.k8s.io
            kind: Gateway
            name: integration
            namespace: integration
            sectionName: http
      selector:
        dnsNames:
        - example.com

The external regional ALB is correctly provisioned, the gateway is marked as programmed and its status looks ok. The HTTPRoute that gets automatically created by the ClusterIssuer also looks ok. Except that the HTTP01 challenge remains forever stuck in the “pending” status with the reason “Waiting for HTTP-01 challenge propagation: wrong status code ‘404’, expected ‘200’”.

I debugged further by creating an additional HTTPRoute object in the same namespace referring to an actual app (outside of this debugging step, I do not have any HTTPRoute in place because I first want to test the behavior of the provisioning of certificates). Sure enough, minutes after, the route actually does work and I can reach the app from outside the cluster, as expected.

Something else I’ve tried was to port-forward through the solver pod and do a GET call on the token endpoint like this: http://localhost:8089/.well-known/acme-challenge/token -H “Host: example.com” → sure enough, this also works.

When I do a curl on the token endpoint from outside the cluster I get “fault filter abort” returned, which afaik means nothing is available on that route.

→

Expected behaviour:
The challenge is not stuck in the “pending” status and the certificate gets correctly provisioned. The HTTPRoute is also attached to the Gateway.

Steps to reproduce the bug:

  1. Have a GKE standard cluster with Gateway API enabled
  2. Install cert-manager
  3. Create a Gateway in a given namespace with the “gke-l7-regional-external-managed” gateway class name and annotate it with cert-manager.io/cluster-issuer: letsencrypt-staging
  4. Configure a ClusterIssuer named letsencrypt-staging (e.g. with the staging endpoint) with a HTTP01 solver referring to the previously created Gateway object
  5. See the challenge stuck in the pending status

Anything else we need to know?:

  • “Attached routes” show 0 on the Gateway but I didn’t check if that was different when I manually tested a user-defined HTTPRoute object connected to an app running in the namespace.

Environment details:

  • Kubernetes version: v1.33.5-gke.2392000
  • cert-manager version: 1.19.4 (gateway api enabled)

I had created a github issue in the cert-manager repository but seems like the issue does not come from their side but rather from the GKE Gateway controller? Has anybody faced a similar issue with a similar setup?