GCP HTTPS Load Balancer with Let’s Encrypt certificate
In this article we will create Let’s Encrypt certificate (CA certificate) and deploy it to GCP HTTPS load balancer. We will install cert-manager on the Google Kubernetes Engine (GKE) and use it to request Let’s Encrypt in order to issue the certificate for us.
Once we get the Let’s Encrypt certificate we then will create a Kubernetes ingress resource that will internally create the GCP HTTPS load balancer. We can actually use ManagedCertificate resource to instruct GCP to create the certificate (we call it Google managed certificate) for us but in this article we won’t. There’re some reasons to use Let’s Encrypt certificate (or other self managed certificates) instead of Google managed one:
- Google managed certificate does not allow us to use wildcard domain like “*.yourdomain.com”.
- Adding new domains to the existing Google managed certificate might not be easy. It needs more steps to do compare to the self managed certificate.
Note — This article is about deploying certificate to GCP HTTPS load balancer, it’s not about deploying certificate to Nginx ingress controller.
Prerequisites
This article won’t go too much in details about what the certificate is, what the Kubernetes ingress is. We will assume that the following knowledges are already in place:
- Helm — We will use it to deploy the Nginx ingress controller and Cert Manager on GKE.
- cert-manager — The tool that we use for obtaining the certificates from Let’s Encrypt issuer, but not limit to. We will not go too much about how to setup it but will provide the ready to use YAML files that make cert-manager able to talk to Let’s Encrypt and help Let’s Encrypt to perform DNS challenge with Cloud DNS.
- Cloud DNS — We will use Cloud DNS to manage the domain demonstrated in this article.
- Services in Google Cloud Platform (GCP) knowledges for example, firewall, GKE and HTTPS load balancer.
- We assume that you already have your own domain that you can add the “A” records. I will use my own its-software-services.com domain.
- Kubernetes knowledge.
Source Code
The source codes relate to this article are kept publicly in the GitHub repository here — https://github.com/its-knowledge-sharing/setup-gke-lb-lets-encrypt. I would recommend to clone the code and read them at the same time you read this article.
Create the GKE cluster
I’m assuming that we are familiar to the Google Cloud Platform (GCP) and already have the GCP account. See this script for more detail 01-setup-gke.bash.
gcloud container clusters create gke-lets-encrypt --zone us-central1-a --project ${PROJECT}Run the gcloud command above and wait for a while until the GKE cluster is created. Also, use command below to create the KUBECONFIG file to authenticate to our GKE cluster.
gcloud container clusters get-credentials gke-lets-encrypt --zone us-central1-a --project ${PROJECT}Create service account (SA)
To allow cert-manager to perform the DNS challenging with Let’s Encrypt, we will need to provide the SA for cert-manager to talk to Cloud DNS. The cert-manager will actually add the TXT record (value received from Let’s Encrypt) to domain we hosted in the Cloud DNS. This will make Let’s Encrypt trusts that we’re indeed the owner of the domain we want it to issue the certificate.
Run the command below to create the IAM service account (SA).
gcloud iam service-accounts create ${SA_NAME} --project ${PROJECT} --display-name="Service account for GKE"Once the SA is created, we will need to assign the role what this SA can do. For the sake of simplicity, we will grant it with the role roles/dns.admin. Please keep in mind that in the real life we should limit the access to only the operations it suppose to do. Please see this script 02-create-sa.bash for more details.
gcloud projects add-iam-policy-binding ${PROJECT} \
--member="serviceAccount:${SA_NAME}@${PROJECT}.iam.gserviceaccount.com" \
--project ${PROJECT} \
--role="roles/dns.admin"Create DNS zone in Cloud DNS
In this article we will use the subdomain demo1.its-software-services.com and test1.demo1.its-software-services.com for Let’s Encrypt to issue the certificate. We will create DNS managed zone “demo1” and later the “A” record will be put here. We don’t have any script to create the DNS zone, we need to create it manually as shown below.
Create external IP and “A” record
To make the outside world able to connect to our services on GKE, we will need to create the external IP address first and then map it to the domain demo1.its-software-services.com and test1.demo1.its-software-services.com. Later we will assign this IP address to the HTTPS load balancer.
The code snippet below demonstrates how we create the new IP address and map it to the domains mentioned earlier, the full script can be view here 03–1-setup-ip-dns.bash.
#!/bin/bashsource .env
IP_NAME=ingress-ip-1# Create IP
gcloud compute addresses create ${IP_NAME} \
--global \
--ip-version IPV4 \
--project ${PROJECT}IP=$(gcloud compute addresses describe ${IP_NAME} --global --project=${PROJECT} | grep 'address:' | cut -d':' -f2)gcloud dns record-sets transaction add ${IP} \
--name=${DOMAIN1} \
--ttl=300 \
--type=A \
--zone=${MANAGED_ZONE1} \
--project ${PROJECT}gcloud dns record-sets transaction add ${IP} \
--name="test1.${DOMAIN1}" \
--ttl=300 \
--type=A \
--zone=${MANAGED_ZONE1} \
--project ${PROJECT}
If everything is OK, we should see the created IP address and the domains mapped to it similar to the below picture. Please note that, you will need to use your own domain instead of its-software-services.com.
Deploy cert-manager to GKE
The easiest way we deploy cert-manager into Kubernetes is by using Helm. The cert-manager also provided it’s own Helm chart here https://charts.jetstack.io/. What we need to do is to create the Helm values file to customize what we need and run some Helm commands to get everything.
To simplify things, I wrote a cert-manager deployment script which can be viewed here 04–1-deploy-cert-manager.bash.
The snippet below demonstrates that we will need to create Kubernetes secret resource to hold the SA key file first. Then this secret will be used by cert-manager when invoking the Cloud DNS API (create the TXT records of the domains).
# Create service account secretgcloud iam service-accounts keys create ${KEY_FILE} --iam-account=${SA}kubectl delete secret ${SECRET} -n ${NS}
kubectl create secret generic ${SECRET} --from-file=${KEY_FILE}=${KEY_FILE} -n ${NS}helm repo add cert-manager https://charts.jetstack.io/
helm template cert-manager cert-manager/cert-manager \
-f cert-manager/cert-manager.yaml \
--version 1.7.1 \
--namespace ${NS} > tmp-cert-manager.yamlkubectl apply -n ${NS} -f tmp-cert-manager.yaml
CLUSTER_ISSUER_FILE=cert-manager/cluster-issuer.yaml
cp ${CLUSTER_ISSUER_FILE} ${CLUSTER_ISSUER_FILE}.tmp
sed -i "s#__PROJECT__#${PROJECT}#g" ${CLUSTER_ISSUER_FILE}.tmpkubectl apply -n ${NS} -f ${CLUSTER_ISSUER_FILE}.tmp
If everything is OK, we should see the pods and secrets in the cert-manager namespace similar to the pictures shown below.
Deploy Nginx ingress controller to GKE
We will need to deploy Nginx ingress controller and we will use it to be the backend of the Google HTTPS load balancer. The ingress controller will act like the entry point to serve the traffics come from Google HTTPS load balancer, the traffics will then be routed to the backend services in GKE depended on how we configure the controller.
To setup the Nginx ingress controller, it will be similar to the cert-manager that we use Helm to deploy it. The script to deploy is here 05–1-deploy-nginx.bash. If everything is OK, we should see the result similar to the picture shown below.
Create Let’s Encrypt certificate
Now it is the time to create the certificate. By Kubernetes point of view, the certificate is just the Secret resource with type kubernetes.io/tls. We will create the Cluster resource as shown below in order for cert-manager to request Let’s Encrypt to issue the certificate.
The script to create this certificate can be found here 06–1-deploy-cert.bash. Please note that we create the Certificate resource in the name space ingress-nginx because we need the kubernetes.io/tls secret to be created in this namespace too.
If everything is OK we should see the certificate with READY status set to True 5 minutes after running the script 06–1-deploy-cert.bash. If not, please follow the troubleshooting guide — https://cert-manager.io/docs/faq/acme/.
Once the cert-manager has successfully created the certificate, we will see the correspond Secret resource in the same namespace as shown in the picture below. Later we will deploy this secret/certificate to GCP HTTPS load balancer, this will be done by creating the Ingress resource which will cover soon.
Create HTTPS load balancer via Kubernetes Ingress resource
Now it is time to deploy the certificate we created earlier to Google HTTPS load balancer. To create the load balancer, we will create the Ingress resource as show below.
I created the script here 06–2-deploy-ingress-1.bash. Please note that in the Ingress above we configured the Ingress to use the certificate from the demo-cert Secret resource.
Take a look the annotation kubernetes.io/ingress.global-static-ip-name, we will see that it refers to name ingress-ip-1 of IP address we created earlier. This is the way we assign the IP address to the load balancer.
Wait for no long than 5 minutes after running the 06–2-deploy-ingress-1.bash script, we should see the Ingress nginx-ing-1 as shown in the picture below. Please note that the IP address is the same as one that we created earlier.
We should see the HTTPS load balancer created in the Google Cloud console similar to below picture.
The Let’s Encrypt certificate we created earlier is deployed to load balancer as expect!!!
Try surfing the URL — https://test1.demo1.its-software-services.com/ (this may be the dead link as I will remove it later). Don’t worry about the “404 Not Found” page, this is because we don’t configure any backend service that Nginx should route the traffic to.
Add new subdomain
Let’s assume the situation that we need a new subdomain to be added into the certificate. Actually, we won’t be able to modify the existing certificate that has been issued, we need to modify the existing Certificate resource (no need to change the resource name) instead.
The new Certificate is shown below, note that there is the new subdomain test2 added.
We need to create the “A” record mapped to domain test2.demo1.its-software-services.com as well. This script 07–1-update-dns.bash will add the new “A” record into Cloud DNS. Please see the picture below.
Now update the existing Certificate demo-cert by running this script 08–1-update-letsencrypt-cert.bash. The READY status will change to False because cert-manager detects that the Certificate is changed and try to request the new one from Let’s Encrypt. It should be no longer than 5 minutes then we will get the new one.
We will need to wait for a while, may be 20 minutes for GKE cloud controller to detect the changes of the Secret demo-cert and propagate the new certificate to GCP HTTPS load balancer.
We eventually get the new Let’s Encrypt certificate that support test2 subdomain.
Certificate renewal
Let’s Encrypt normally issues the 90-days valid certificate. Fortunately, cert-manager will keep checking to see if the certificates soon to expire or not. If yes, it will resubmit the request for new certificate from Let’s Encrypt. Once the new certificate is available, the GKE cloud controller will detect this and will propagate the new certificate to GCP HTTPS load balancer.
Supports
Congratulation!!! if you’ve read the entire article and it is able to help you solve your issues. You can support me by:
- Follow me.
- Buy me a coffee via ADA address below if you want.
