Free, trusted SSL Certificates for Apigee hybrid ingress on GKE

Google Cloud Apigee Knowledge Hub
1

If you followed the installation instructions for Apigee hybrid for GKE and own the domain you used for your Cloud DNS zone, you already have all the necessary components to provision trusted SSL certificates for your API endpoints. Having certificates that are issued by a trusted CA is important, especially when exposing public APIs, as the certificates establish a chain of trust and prove domain ownership.

This post quickly outlines the necessary steps to configure an Cert-Manager issuer for Let’s Encrypt and how to use the automatically provisioned certificates on your Apigee hybrid ingress.

To recap the Apigee hybrid installation instructions, we assume you have provisioned and configured the following:

  • Cloud DNS with a managed zone e.g. apigee.example.com as per the Apigee hybrid installation instructions.
    TLDR; Cloud DNS is a managed DNS Service in Google Cloud
  • NS record for your domain as described here.
    TLDR; This ensures that your Cloud DNS Zone is discoverable
  • Cert-Manager installed as per the Apigee hybrid installation instructions.
    TLDR; Cert manager is a kubernetes operator for certificate management

The approach described below is only one possible option to automatically provision trusted certificates for the Apigee hybrid ingress. Other options include:

  • Using GCP External HTTPS load balancers with Google managed SSL certificates and Cloud Armor.
  • Using Cert-Manager as described below without Cloud DNS for runtime installations outside of GCP. In this case the Cert-Manager issuer would have to be changed to reflect the domain setup.

Authorizing Cert-Manager for Cloud DNS

In order to allow Cert-Manager to configure the necessary records in Cloud DNS to obtain a trusted certificate, we need to authorize it using a GCP service account. The following commands create a service account in GCP and add its key as a Kubernetes secret.

gcloud iam service-accounts create dns01-solver --display-name "dns01-solver"

gcloud projects add-iam-policy-binding $PROJECT_ID \
   --member serviceAccount:dns01-solver@$PROJECT_ID.iam.gserviceaccount.com \
   --role roles/dns.admin

gcloud iam service-accounts keys create key.json \
   --iam-account dns01-solver@$PROJECT_ID.iam.gserviceaccount.com

kubectl create secret generic clouddns-dns01-solver-svc-acct \
   --from-file=key.json -n istio-system

Cert-Manager Configuration

Because we already installed Cert-Manager and set up a Cloud DNS zone as part of the Apigee hybrid installation, we only need to create a Kubernetes resource to declare a Cert-Manager issuer based on Cloud DNS entries:

cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
  name: cloud-dns-issuer
  namespace: istio-system
spec:
  acme:
    email: cert-admin@your-domain.com
    server: [https://acme-v02.api.letsencrypt.org/directory](https://acme-v02.api.letsencrypt.org/directory)
    privateKeySecretRef:
      name: cloud-dns-issuer-account-key
    solvers:
    - dns01:
        clouddns:
          project: $PROJECT_ID
          serviceAccountSecretRef:
            name: clouddns-dns01-solver-svc-acct
            key: key.json
EOF

And then create a Certificate resource to start the request for a trusted certificate for our API ingress gateway:

cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: cert-manager-secret-$PROJECT_ID-default
  namespace: istio-system
spec:
  secretName: cert-manager-secret-$PROJECT_ID-default
  issuerRef:
    name: cloud-dns-issuer
  commonName: '*.apigee.example.com' # Change this
  dnsNames:
  - apigee.example.com     # Change this
  - '*.apigee.example.com' # Change this
EOF

After a few minutes the status of your certificate resource will change to “ready”:

kubectl get Certificate cert-manager-secret-$PROJECT_ID-default -n istio-system

And your TLS secret is ready to be used:

kubectl get secret cert-manager-secret-$PROJECT_ID-default -n istio-system

Using the trusted certificate in Apigee hybrid

First off, we need to locate the ApigeeRouteConfig that corresponds to the ingress we would like to add our new certificate to:

kubectl get apigeerouteconfigs.apigee.cloud.google.com -n apigee

We then replace the secret in the ApigeeRouteConfig to point to our newly created and Cert-Manager controlled secret:

cat <<EOF | kubectl apply -f -
apiVersion: apigee.cloud.google.com/v1alpha1
kind: ApigeeRouteConfig
metadata:
  name: $ROUTE_CONFIG_NAME_FROM_BEFORE # Change this
  namespace: apigee
spec:
  selector:
    app: istio-ingressgateway
  connectTimeout:
    300
  tls:
    mode: SIMPLE
    secretNameRef: "istio-system/cert-manager-secret-$PROJECT_ID-default" # Change this
EOF

Note that the Apigee hybrid quickstart in Apigee DevRel is now automatically creating let’s encrypt certificates. The steps above are still needed for manual installations.

Edit (8/9/2021): Note that the Apigee hybrid quickstart in Apigee DevRel is now automatically creating let’s encrypt certificates. The steps above are still needed for manual installations. You can also use an L7 load balancer with your Apigee hybrid deployment and use trusted google managed certificates directly on the load balancer as described in this article.

6 Likes
2

Awesome! Love this, Daniel.

3

This was super helpful, thanks Daniel! I had a problem that my sub-domain wasn’t properly configured to the Cloud DNS NS (as NS record in the Domain DNS config), but once that was corrected, everything worked perfectly.

4

The original post from Daniel states:

This post quickly outlines the necessary steps to configure a Cert-Manager issuer for Let’s Encrypt and how to use the automatically provisioned certificates on your Apigee hybrid ingress.

The original article is still relevant, but with hybrid 1.8, things are a little different. For one thing, the namespaces change. For another, the yaml changes slightly.

I’ll describe what I did to get this to work with Hybrid 1.8. The main differences:

  • I’m going to use the apigee namespace for the issuer, not the istio-system namespace
  • my cloud DNS is in a different GCP project than the kubernetes cluster. This is just how I have my cloud DNS set up, it is unrelated to the switch to hybrid 1.8.
gcloud config set project my-gcp-project-containing-gke
export PROJECT_ID=my-gcp-project-containing-gke
export DNS_PROJECT_ID=gcp-project-that-hosts-cloud-DNS

gcloud iam service-accounts create dns01-solver --display-name "dns01-solver" --project $PROJECT_ID

gcloud projects add-iam-policy-binding $DNS_PROJECT_ID \
   --member serviceAccount:dns01-solver@$PROJECT_ID.iam.gserviceaccount.com \
   --role roles/dns.admin

gcloud iam service-accounts keys create key.json \
   --iam-account dns01-solver@$PROJECT_ID.iam.gserviceaccount.com

kubectl create secret generic clouddns-dns01-solver-svc-acct \
   --from-file=key.json -n apigee

Then, the issuer. Moving to cert-manager.io/v1 , this must use cloudDNS as the name of the solver, not clouddns. I think the case is significant.

cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: cloud-dns-issuer
  namespace: apigee
spec:
  acme:
    email: dchiesa.cert-admin@google.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: cloud-dns-issuer-account-key
    solvers:
    - dns01:
        cloudDNS:
          project: $DNS_PROJECT_ID
          serviceAccountSecretRef:
            name: clouddns-dns01-solver-svc-acct
            key: key.json
EOF

Then the certificate:

cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: cert-manager-$PROJECT_ID-default
  namespace: apigee
spec:
  secretName: cert-manager-$PROJECT_ID-tls
  issuerRef:
    name: cloud-dns-issuer
  commonName: '*.apigee-hybrid.my-domain.com'
  dnsNames:
  - eval.apigee-hybrid.my-domain.com
  - '*.apigee-hybrid.my-domain.com'
EOF

That configuration assumes that I have my cloud DNS already resolving eval.apigee-hybrid.my-domain.com to the IP address of the TCP External Load Balancer for my hybrid cluster.

To check status of that update, I used

watch kubectl get Certificate -n apigee cert-manager-$PROJECT_ID-default  -o yaml

And after about 60 seconds it showed as “Ready”. Then, to get those warm-n-fuzzy feelings, check the secret:

kubectl get secret cert-manager-$PROJECT_ID-tls -n apigee -o yaml

And finally, check the ApigeeRouteConfig :

kubectl get apigeerouteconfigs.apigee.cloud.google.com -n apigee $ROUTE_CONFIG_NAME -o yaml

And update it with the appropriate new Secret:

cat <<EOF | kubectl apply -f -
apiVersion: apigee.cloud.google.com/v1alpha1
kind: ApigeeRouteConfig
metadata:
  name: $ROUTE_CONFIG_NAME
  namespace: apigee
spec:
  connectTimeout: 300
  selector:
    app: apigee-ingressgateway
  tls:
    mode: SIMPLE
    secretNameRef: cert-manager-$PROJECT_ID-tls
EOF

After that, I was able to invoke my hybrid ingress controller using 1-way TLS validation. In other words I was able to do:

curl https://api.hybrid-apigee.my-domain.com/basepath

…and the TLS was properly negotiated.