Automating Terraform with GitHub Actions
Terraform
Terraform is a marvellous open-source for infrastructure provisioning and management — infrastructure-as-code (IaC). With the help of Terraform now we can code our infrastructure in a declarative manner using blocks, arguments, and expressions, so no more click, click on UI and remembering what resources you’ve created; terraform is here to save your day. Terraform manages our infra via state file and its vendor-independent.
Terraform is one of the most demanding tools of the DevOps industry. I’ve been working and exploring Terraform since January 2020 and really it’s helping me a lot on daily basis infrastructure tasks.
GitHub Actions (GA)
GitHub Actions is an automation tool offered by GitHub which offers CI/CD, it’s one of the fastest-growing tools because of its speedy deployment, more reliable security, less overhead and easy-to-write workflow (pipeline). For in-depth knowledge, have a look at GA Docs.
With GitHub Marketplace, automation and deployment with GitHub Actions become so easy. GitHub Marketplace is a place where developers around the universe publish tools that allow us to create better and more custom workflow.
GA team gradually making it more mature with every update. It is one of my favourite tools for CI/CD after GitLab CI.
Create GCP Service Account
To let terraform provision infrastructure on GCP, we’ve to configure the Google Cloud SDK in the GitHub Actions environment.
Create one GCP Service Account. I’ve given the Project Owner role because I’m considering terraform the only resource which can be provisioning all/any resource(s). You can narrow the permission scope according to you.
Once done, create a Service Account key (JSON format).
Add GitHub Secrets
Once keys got generated, it is time to put it as a Secret on the GitHub repo so it can be supplied on runtime.
From Repo Settings➞Secrets➞New repository secret, create two secrets.
- GCP_SA_KEY: the value of the SA Key that you’ve downloaded
- GCP_SA_EMAIL: service account email address
Once you add these creds Terraform will able to provision resources on GCP.
Workflow
GitHub Repo: https://github.com/r4rohan/terraform-with-cicd
For the blog purpose, I’ve written Google Compute Engine terraform module and passing configuration value from examples folder.
Whenever a developer pushes TF code on GitHub, GA workflow will be triggered and start running necessary TF commands. The workflow is divided into 2 jobs — one dedicated to plan and another one apply.
Only main branch is allowed to provision infrastructure in the cloud, feature branches are only for terraform plan where you can see desired configurations.
Before running terraform plan, GA will run terraform init and terraform fmt through the script and before running terraform apply, it will run terraform init script only. Tf plan will generate tf_plan file/artifact which will be further used by tf apply. actions/upload-artifact@v2 and actions/download-artifact@v2 allows uploading (in tf plan job) and downloading (in tf apply job) or in short sharing of artifacts b/w jobs.
Terraform apply is manual.
As per my knowledge, GitHub Actions does not have manual approval other than the environment way. I searched a lot, but no luck; so I’ve gone with GitHub Environments where I’ve configured tf_apply as my environment and add myself as a reviewer. I’ve done this just to prevent any infrastructure mess.
terraform_apply:
name: 'Terraform Apply'
runs-on: ubuntu-latest
needs: terraform_plan
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
environment:
name: tf_applyConfigure it from Setting➜Environments.
If any reader does know a better way or know that GA provides manual approval capability other than environment, please highlight this para and let me know, love to update.
There is one more side job that puts our workflow job status on Slack kinda post-checkout.
Slack Integration
To allow GitHub Actions to send alert to Slack, we have to create an app and add Slack Webhook in GitHub Actions secret.
From your Slack Dashboard:
Create a New App ➜ App Name and Choose Slack Workspace ➜ Incoming Webhooks — Toggle Activate Incoming Webhooks ➜ Add New Webhook to Workspace ➜ Select Slack Channel.
Once you add your app to the Slack channel, you’ll see a Webhook URL, copy it and create a new SLACK_WEBHOOK GitHub Secret from Settings.
We are using this secret in the slack_notification job:
- name: Apply Slack Notification
uses: rtCamp/action-slack-notify@v2
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
env:
SLACK_CHANNEL: captain-alert
SLACK_COLOR: ${{ needs.terraform_apply.result }}
SLACK_TITLE: ${{ github.repository }}
SLACK_MESSAGE: ${{ github.event.head_commit.message }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
SLACK_FOOTER: 'Terraform Github Actions Alert'I’ve added two Slack Notification step, one is dedicated to tf_apply and another one is dedicated to tf_plan. This slack notification step is mandatory step means, which alert us on the success of terraform and failure as well.
Main Points
- GCS bucket is serving as terraform backend.
- Workflow offers concurrency which means only one workflow can be run at a time; I’ve done this to prevent our terraform state from locking and getting corrupted.
- GitHub secrets are being used to pass GCP Service Account credentials safely on runtime.
- Terraform Plan generates a plan file which is further used by terraform apply. This is done to prevent uninformed changes b/w plan and apply.
- Terraform code must be properly formatted which is considered a good practice else terraform format validation will throw an error and the pipeline would get stopped.
- Manual Approval before applying terraform apply stage.
Once approved, terraform will provision resource on GCP and notify you via Slack Alerts.
Destroy Resource(s)
To delete any resource or let suppose you’ve created two VM, out that one you want to destroy; just remove/comment the module(s) which you want to destroy.
# module "palpatine" {
# source = "../modules/gce"
# suffix = local.suffix
# gcp_project_id = var.gcp_project_id
# vpc_network_name = "default"
# instance_name = "palpatine"
# network_tags = ["http-server", "https-server", "ssh"]
# }Voila! This is how you can incorporate Terraform with GitHub Actions 🎉🎉🎉
Actions I’ve used: Upload Artifact, Download Artifact, Checkout, Setup GCloud and Slack Notification
I’ll try to incorporate resource provisioning via Terraform to different GCP environment project very soon. With that, you can provision infra with a single GA workflow in dev, stage and prod GCP projects with manual approvals and even different values for each ENVs.
