GCP Infrastructure as Code with Terraform and Terragrunt

In this article we will see how we can provision GCP services by using Terraform, starting from creating the service account, creating VPC and subnet, creating Cloud NAT, configuring firewall rules and creating an example GCE instance. We will see how we can structure our Terraform codes into several folders to make them easy to manage. We will see how we use Terragrunt to simplify the Terraform configurations, it can help our Terraform configurations DRY.

Infrastructure as Code (IaC) is the idea to represent the state of our infrastructure in the form of configuration files and manage the changes via the files changing. I won’t explain the advantages why the IaC is being adopted as the today practice, there are many explanations in the internet for this. Terraform is one of the tools that bring the idea of IaC to the real life.

Source Code

The source codes relate to this article are kept publicly in the GitHub repository here — https://github.com/its-knowledge-sharing/gcp-terragrunt-demo. I would recommend to clone the codes and read them at the same time you read this article.

In this article we also create a GCE instance by using Terraform and use the custom Ubuntu OS image as instance OS image. This custom Ubuntu OS image is created by Packer and the code is here — https://github.com/its-knowledge-sharing/gce-image-packer.

Terragrunt

Terragrunt is the Terraform wrapper tool that helps to solve many Terraform’s pain points like keeping your configurations DRY, working with multiple Terraform modules and folders, and managing remote state.

We won’t go to much in details about how to use Terragrunt but instead will demonstrate by codes example in the GitHub repository mentioned earlier.

Files Structure

We place Terraform codes into several folders based on the GCP services as shown in picture below.

• 00–0-vpc, the code to create VPC and subnet.
• 00–1-sa, the code to create service account and role.
• 00–2-firewall, the code to configure the firewall rules.
• 00–3-nat, the code to create Cloud NAT for outbound internet connection.
• 00–4-route, the code to create the route to internet.
• 01–1-gce-manager, the code to create an example GCE instance that uses custom Ubuntu as OS image.
• modules/gce, the internal module for creating GCE instance.
• terragrunt.hcl, this is the shared configuration file uses for configuring Terraform codes in the folders mentioned above.

Please note that to run the example code in you environment, we will need to change the project name in terragrunt.hcl to match yours.

Infrastructure Provisioning

In order to use Terragrunt to provision the GCP services, we need to install Terragrunt and Terraform first. In this article we will run them in Cloud Shell (web browser base shell to interact with GCP) and assume that we already have them installed. In my shell environment, I downloaded and placed them in my local directory and point the PATH environment to.

For the sake of simplicity, I create a wrapper script infra.bash that will help to provision all the GCP services with a single command.

For the first time we run the script, we will need to invoke and pass the “init” parameter as similar to when we call “terraform init” command.

If we take a look the terragrunt.hcl, we will see that we use the Terraform remote state files configuration, we keep the state files in the GCS bucket. Terragrunt will create the GCS bucket for us if it does not exist.

Now run “infra.bash apply” command and wait for a while. If there is no error, we should see the new GCE instance created similar to picture below. Actually, not just only the GCE created, we should see a new VPC, Cloud NAT and firewall rules created too.

Provisioning Order

We might ask ourselves that how Terragrunt knows that what GCP service need to be provisioned before the another one. For example, how does it knows that it needs to create the VPC before creating the GCE instance?

Please take a look this file terragrunt.hcl in the folder 01–1-gce-manager, there are the “dependency” blocks and this is how Terragrunt knows the provisioning order it needs to perform.

Provisioning GCE instance

It’s the good idea to add more explanations about how we provision the GCE instance. In this article we create the internal module here gce.tf and the module once again refers to another external module.

The Terraform external module to create GCE instance can be found here — https://github.com/its-software-services-devops/tf-module-gcp-vm, it is the one that I created it long time ago and using it in many projects.

Custom Ubuntu OS image

As mentioned earlier, this article uses the Custom OS image when provisioning the GCE instance. There are many advantages to use custom OS image instead of each GCE instance to have its own Terraform provisioner script. The provisioning speed is the first reason for me to use the custom OS image. Think about the situation like if there are 10 GCE instances to be created.

Please take a look the code here — https://github.com/its-knowledge-sharing/gce-image-packer to see how we use Packer to create GCE custom OS image.

If you want to create your own custom OS image and keep it in your own GCP project, please modify the project_id in this file variables.hcl.

To simplify the thing, I created another shell script “run.bash” that internally calls the “packer” command.

The easiest way to run the script is to clone it into your Cloud Shell environment and run the “run.bash”. The Packer binary file will need to be downloaded into your Cloud Shell environment too before running the script.

Cleanup

If everything is done and we want remove them we created earlier, we can simply do it by just by calling “infra.bash destroy” command.

Supports

Congratulation!!! if you’ve read the entire article and it is able to help you solve your issues. You can support me by:

• Follow me.
• Share my articles.
• Buy me a coffee via ADA address below if you want.