XenonStack Recommends

Enterprise Digital Platform

Infrastructure as Code (IaC) using Terraform | Complete Guide

Gursimran Singh | 21 August 2022

Subscription

XenonStack White Arrow

Thanks for submitting the form.

Introduction 

Earlier, it was a challenging and hectic job to manage IT infrastructures. The System Administrator had to manually manage all the underlying services, hardware, and software needed for the entire application system to work.

In recent years, the IT industry has seen a revolution in terms of tools and technologies that work towards automating the process of deploying and managing the Infrastructure. And this is what we term it as Infrastructure as Code or IaC.

Writing Infrastructure in configuration files or code is referred to as Infrastructure as code. This code acts as the single source of truth for all the components required in the Infrastructure, such as Load Balancers, Virtual Machines, Gateways, etc.
Various IaC tools are available, each solving its purpose, but now the question comes, which tool will best suit your goal.
Some of the popular IaC tools are:-

  1. Terraform
  2. Puppet
  3. Ansible
  4. Chef
  5. Cloudformation

What is Terraform?

Terraform is a tool used for changing, building, and versioning infrastructure efficiently and safely. It is used for managing existing and popular service providers or custom in house solutions. The components running for a single application or entire datacenter Configuration files describe to Terraform. An execution plan is being generated by Terraform, which defines what it will do to reach the desired state and then executes it to build the described infrastructure. By the time configuration changes, Terraform can determine what changed and create incremental execution plans which can be applied. Infrastructure Terraform manages including low-level components such as compute instances, and networking, as well as high-level components such as DNS entries, SaaS features, etc.
  • Infrastructure as code.
  • It is a tool to manage virtual server life cycles (AWS, VMWare, etc.)
  • It is a tool to manage supporting services (DNS, Email)
  • It is a tool to manage system services (MySQL, PostgreSQL)
  • Configuration files can be HCL or JSON.
  • Created by Hashicorp (Vagrant et al.)
  • Written in Go

A tool that lets you manage, build, and version your Infrastructure using configuration files. Click to explore about, A Complete Guide to Terraform

Why Terraform is important?

The main goal of DevOps is to perform more efficient software delivery and to make delivery efficient and quick. We need some tools like Terraform, which help companies with infrastructure as a code and automation. It is changing the DevOps world by changing the way infrastructure is managed, and makes faster and more efficient for executing DevOps. There exist technologies like Ansible, Chef, or Puppet for automating and provisioning of software. Terraform also uses the same law “infrastructure as code,” but it targets on automation of infrastructure itself. Complete cloud infrastructure like instances, IPs, volumes, networking can be defined easily.

Why should you use Terraform to manage your Infrastructure?

Terraform is an orchestrator and not an automation tool

What do we mean by that? Automation focuses on a single task, whereas orchestration means creating a workflow and combining multiple automation tasks.

Terraform follows a declarative approach and not a procedural

In a declarative approach, you will tell what you need and not how it is to be done. Just say what you want in your Infrastructure, and terraform will manage all the necessary steps to get the things done.

Multiple Provider support

Terraform comes with the feature to provide support for multiple public cloud service providers like Google Cloud Platform, AWS, Azure.

How to manage your Infrastructure as Code using Terraform?

Terraform is a tool that lets you manage, build, and version your Infrastructure using configuration files. It is a convenient tool that enables you to deploy a wide range of resources such as physical servers, networking interfaces, Load balancers on various platforms such as AWS, GCP, and Azure.
The same code needs to be replicated in different environments, ensuring that the same Infrastructure is applied everywhere.
For example, say you are working with AWS as your cloud provider and would like to spin up several EC2 instances of a specific type. You would define the type and number of instances in a configuration file, and Terraform would use that to communicate with the AWS API to create those instances. The same file could then be used to adjust the configuration, for example, increasing or decreasing the number of instances.
Because Infrastructure typically includes many more components than just compute instances, we use a Terraform feature called modules to combine multiple infrastructure components into large, reusable, and shareable chunks.

What are the features of Terraform?

The features of Terraform are listed below:

Infrastructure as a Code

A high-level configuration syntax is a use for describing support. It allows a blueprint of our datacenter to be versioned and treated it you would any other code. Also, infrastructure can be shared and re-used.

Execution Plans

It has a “planning” step from which it generates an execution plan. This execution plan shows what Terraform will do when they apply. This eliminates any surprises when Terraform manipulates infrastructure.

Resource Graph

A graph of all resources builds by Terraform, and parallelize the creation and modification of any non-dependent resources. By doing so, Terraform builds infrastructure as efficiently as possible, and operators get insight into dependencies in their infrastructure.

Change Automation

With minimal human interaction, complex changesets can be applied to infrastructure. What Terraform will change and in what order can be known by the previously mentioned execution plan and resource graph, which helps in avoiding many possible human errors.

CI/CD mechanisms are applied to test server configuration, server image building, environment provisioning and the mixture of environments. Click to explore about, How to Implement Infrastructure as Code in CI/CD Pipeline?

How does Terraform work?

Terraform is split into two parts, terraform core and terraform plugins. Terraform Core communicates with the terraform plugin. Terraform plugins expose an implementation for specific services such as AWS, bash.

What is the Terraform Core?

It’s a binary written in Go programming language. The compiled binary corresponds to CLI terraform.
Terraform Core is responsible for :
Reading the configuration files, i.e., IaC.
State management of various resources.
Construction of resource graph.
Execution of plan.
Communication with plugins.

What are the terraform Plugins?

Terraform Plugins are executable binaries written in Go.

Requirements for Terraform setup

Terraform binary
Configurations files (in which you write IaC)
State file

Terraform configuration files (*.tf)

You write every terraform configuration in files with an extension as .tf. You’ll be mostly dealing with resources in configuration files. Resources are the services or components of your Infrastructure, such as Load Balancers, Virtual machines, etc. And you pass various arguments in a resource you want to set up.
For example:

main.tf
resource "aws_instance" "web" {
ami = data.aws_ami.ubuntu.id
instance_type = "t3.micro"
tags = {
Name = "HelloWorld"
}
}

The above configuration file main.tf, defines a resource, i.e., EC2 instance in AWS. Inside the resource block, we have passed the desired arguments as:
ami- AMI id to be issued for our EC2.
Instance_type
Tags - tags for our EC2
This is how we can define other resources, as well.
We can now use terraform apply to deploy this resource on AWS.

 

What is Terraform state?

The most crucial part of terraform is the terraform state file(terraform.tfstate), which stores all the information related to the deployed Infrastructure and mapping between various components. Terraform then refers to this state file while deploying Infrastructure to check dependencies among resources.

Why is a state file required?

While deploying resources, for example, EC2, as in the example above, you don't specify the ARN or instance ID for the EC2. It gets dynamically assigned, so terraform needs to store all such information so that internally it can detect changes if you update the instance_type for the EC2 in the future.
State file is the single source of truth for your Infrastructure, so you need to keep this very safe and ensure that your team always refers to the latest version of terraform.tfstate file to avoid conflicts. By default, state file is stored locally, but keeping it locally is only beneficial if just one developer is working on managing the Infrastructure. However, the best practice is to keep terraform.tfstate file in some remote backend such as S3 bucket.
Following example shows the backend configuration as S3 bucket.

terraform {
backend "s3" {
bucket = "mybucket"
key = "path/to/my/key"
region = "us-east-1"
}
}

 

Backend Initialisation

An initialization step needs to be performed every time there is a change in backend configuration; the command used is terraform init.
This command does the following:
Initializes backend
Configures terraform modules (will cover later).

Terraform Workflow

Write IAC, i.e., configuration files.
See what changes it will reflect using terraform plan.
Deploy those changes using terraform apply

Terraform plan

The plan will represent a list of changes that your IAC will make. It doesn't actually affect anything, and it just shows what new will be created/deleted/modified. Therefore, it is safe to use terraform plans multiple times.

Interpreting Terraform plan

Resources can be easily interpreted by their names. For example,
Syntax: <Resource type>.< resource name>
Example, aws_instance.this.
Resources inside a module can be interpreted by:
Syntax: module.<module_name>.<Resource type>. < resource name>
Example module.my_virtual_machines.aws_instance.this
Terraform plan shows five actions for the resources:
[+] Add - Creation of new resource.
[ - ] Delete - Removal of existing resource.
[~] Modify in-place - Existing resources will be modified, and some new parameters will be added to them. Terraform plans will also show you which new parameters will be modified/ added.
[- / +] Terraform will delete and re-create the same resource. This happens when it's not possible to modify a parameter in place.

Sample Terraform Plan

An execution plan has been generated and is shown below.

20:10:18 Resource actions are indicated with the following symbols:
20:10:18 + create
20:10:18
20:10:18 Terraform will perform the following actions:
20:10:18
20:10:18 # aws_cloudwatch_log_group.log_group_containers[0] will be created
20:10:18 + resource "aws_cloudwatch_log_group" "log_group_containers" {
20:10:18 + arn = (known after apply)
20:10:18 + id = (known after apply)
20:10:18 + name = "/eks/dev-scipher-fx-eks-cluster-2/containers"
20:10:18 + retention_in_days = 14
20:10:18 + tags = {
20:10:18 + "Environment" = "dev"
20:10:18 + "Service" = "EKS Cluster"
20:10:18 + "Terraform" = "true"
20:10:18 }
20:10:18 }
20:10:18
20:10:18 # aws_cloudwatch_log_group.log_group_host[0] will be created
20:10:18 + resource "aws_cloudwatch_log_group" "log_group_host" {
20:10:18 + arn = (known after apply)
20:10:18 + id = (known after apply)
20:10:18 + name = "/eks/dev-scipher-fx-eks-cluster-2/host"
20:10:18 + retention_in_days = 14
20:10:18 + tags = {
20:10:18 + "Environment" = "dev"
20:10:18 + "Service" = "EKS Cluster"
20:10:18 + "Terraform" = "true"
20:10:18 }
20:10:18 }

 

In the above example, the plan shows:
Two new Cloudwatch groups being created in AWS.
Log groups retention_in_days being set to 14 days
Tags being added to log groups

Terraform apply

On applying a given plan, terraform locks the state. It will perform all the actions stated in the planning stage and deploy those resources. Some resources might take time; for example, AWS DocumentDB may take up to 15 minutes. Until that, terraform will block all other operations.
On completion, terraform updates the state file.

Terraform Modules

All the configuration files together with a folder are called a module. Modules allow us to create a reusable code that can be referred to again with required parameters. Consider it as just like functions in any programming language where we just have to call the process anywhere and pass the necessary set of parameters.
Using a module in a configuration file is similar to using resources as we have seen above; the only difference is the clause “module.”
module "moduleName" {
source = "module/path"
\CONFIG
/
}

The source path tells where the module can be found.
CONFIG block consists of a set of variables or arguments which can be passed to the module and belongs to the module.
For example:
If the main module has a variable log_retention_period, we can pass its value differently for different environments such as:
For Dev environment:
module "cloud_watch_log_groups" {
source = "module/path"
Log_retention_period = 14}

Similarly for Prod environment:
module "cloud_watch_log_groups" {
source = "module/path"
Log_retention_period = 21
}

Thus, as you can see, we can reuse the same module to deploy Infrastructure for multiple environments.

What are the advantages of Terraform?

  • Orchestration, not merely Configuration: For server configuration, all the configuration management tools were created, and to manage and to install the software already existed in the server / virtual machine where their primary goal.
  • Terraform focuses more on server provisioning. When the complete cloud infrastructure is considered as code, and all the parameters were combined in a configuration file, all the members of the team can easily collaborate on them as they would do any other code.
  • Multi-Provider - The most versatile feature of Terraform is that it supports multi-cloud orchestration like AWS, Azure, OpenStack, etc. and also premises deployments. It is beneficial when using two different resources from two various cloud providers at the same time.
  • Immutable Infrastructure - When using Chef, Puppet, Salt runs any software updates on servers; this often leads to configuration drift when differences in the configuration lead to bugs that lead to security breaches. Terraform solves this issue by using an immutable infrastructure approach in which every change in configuration leads to a separate configuration snapshot which means de-provisioning the old one and deployment of a new one. Updating the development environment by this way goes smooth and bug-free and returning the old configuration is easy like rolling back to a specific version.
  • Syntax - HCL (HashiCorp Configuration Language) is a custom language that is used by Terraform.
  • Dry Runs - Terraform uses a command known as “terraform plan,” which creates an execution plan; by using this execution plan, we can check whether the set of changes meets the expectation without performing any changes to real resources or state. For example, by running the terraform plan before committing a change to version control, check whether it will work as expected or not.
  • Client-only architecture - Terraform eliminates the need for additional checks for provisioning the infrastructure, which leverages the cloud provider’s API. In Ansible, this is done by connecting through SSH but with limitations. Terraform works on APIs and have a wide variety of options, which helps to make it more secure, reliable, and easy to use.
  • Super Portability - There are only a single tool and single language for describing the infrastructure which is used for multiple cloud providers. The problem of migrating to vendors is not a problem.
Infrastructure as Code is infrastructure management in a descriptive model, which helps to create infra and manage it with source code. Click to explore about, IaC Principles for Kubernetes Configuration Management

What are the use cases of Terraform?

The Use Cases of Terraform are listed below:

Heroku App Setup

Heroku is a PaaS for hosting web applications. Developers create an application and then attach add-ons like database or email provider. Its best feature is the ability to scale the number of dynos or workers elastically. Terraform is used for codifying the setup required for a Heroku application, ensuring that all necessary add-ons are available. But it can go even further -
  • Configuring DNSimple to set a CNAME,
  • By setting Cloudflare as CDN for the app.

Multi-Tier Applications

One of the pervasive patterns is N-tier architecture. A widespread 2-tier architecture is a pool of web servers that uses a database tier. More tiers can be added for API servers, caching servers, routing meshes, etc. To scale tiers independently and provide a separation of concerns, this pattern is used. Terraform is an optimal tool for building and managing these infrastructures. Every tier is described as a collection of resources and automatically handles the dependencies between each tier. Before the web servers are started, the database tier is available will ensure by Terraform and that the load balancers are aware of the web nodes. By using Terraform, each tier can be scaled easily by modifying a single count configuration value because the creation and provisioning of a resource are codified and automated, elastically with load becomes trivial.

Self-Service Clusters

In a specific organizational size, to manage a large and growing infrastructure becomes very challenging for a centralized operations team. Instead, to make “self-serve” infrastructure becomes more attractive, using tooling provided by the central operations team allowing product teams to manage their infrastructure. By using Terraform, the ability of how to build and scale a service can be codified in a configuration. To enabling customer teams to use the configuration as a black box Terraform configurations can be shared within an organization and to manage their services Terraform is used as a tool.

AI enabled the IT Infrastructure to be flexible, intangible and on-demand. Click to explore about, AI in IT Infrastructure Management

Terraform comparison with Chef, Ansible, Puppet and Salt?

On a machine that already exists, the configuration management tools install and manages. Terraform does the same, and also it allows us to focus on bootstrapping and initializing resources. Configuration management tools can also be used along with Terraform to configure things inside a virtual machine. Terraform makes the configuration management tool to be used to set up a resource once it has been created by using provisioners. When using Docker containers for running applications, these are self-sufficient and will contain the whole configuration of the application. Tools like Chef, Puppet, Ansible, etc. is not needed. But still, need something to manage infrastructure because the container will run anyway in the top of a server/virtual machine. Terraform is used to create infrastructure for containers to run on. Tools such as Chef, Ansible, Puppet, etc. are used as IAS or Infrastructure as Code, but Terraform is best for this because it can even maintain the state of infrastructure.


How to implement Terraform using AWS services?

There are following steps to implement Terraform (By using AWS)

Setting up AWS account

(Infrastructure across different types of cloud providers like AWS, Google Cloud, Azure, DigitalOcean, and many others) Cloud hosting services provided by AWS were reliable and scalable. AWS is one of the most popular cloud infrastructure providers.

Installing Terraform

Find the supported package of Terraform for system and download it. Terraform is download as a zip archive, then unzip the package. After installing, verify the installation by opening a terminal session and checking that Terraform is available. Deploying a single server - HCL language is used for writing Terraform code in files with extension “.tf.” HCL is a declarative language that describes the infrastructure which is wanted, and Terraform will find out how to create it. Infrastructure across different platforms or providers can be created by Terraform like AWS, Google Cloud, Azure, DigitalOcean, and many others. Configure the provider want to use is the first step for using Terraform. Deploying a single web server: In the next step, run a web server on this instance.

Deploying a cluster of web servers

To running a single server can be a good start, but a single server can be a single point of failure. If the server overwhelmed by heavy traffic or the server crashes, users could no longer access the site. To eliminate this run, a cluster of servers, which are routing around servers that go down and on the base of traffic, adjust the size of cluster up or down. There is a lot of work for managing such a cluster manually. Fortunately, AWS will take care of this by using the Auto Scaling Group (ASG).

Deploying a load balancer

One more problem to solve before launching the ASG: that there are many Instances, for this need a load balancer to distribute traffic across all of them. There is a lot of work for creating a load balancer that is highly available and scalable. AWS will let take care of this by using an Elastic Load Balancer (ELB) Clean up - After doing experimenting with Terraform, remove all the created resources, so AWS doesn’t charge for them. Terraform keeps track of created resources, and cleanup is a breeze.

Java vs Kotlin
Our solutions cater to diverse industries with a focus on serving ever-changing marketing needs. Click here for AI Transformation for Infrastructure and Operations

Command-line interface for Terraform?

The command-line interface (CLI) is used for controlling Terraform.
  • apply - Use for changes or builds infrastructure
  • console - Use for interactive console
  • destroy - Use for impair Terraform – managed infrastructure
  • fmt - Use for rewriting config files into canonical format
  • get - Use for downloading and installing modules for configuration
  • graph - Use for creating visual graph of the Terraform resources
  • import - Use for importing existing infrastructure into Terraform
  • init - Use for initializing existing or new Terraform configuration
  • output - Use for reading output from a state file
  • plan - Use for generating and showing an execution plan
  • providers - Use for printing tree of providers used in the configuration
  • push - Use for uploading this Terraform module to Terraform Enterprise to run
  • refresh - Use for updating local state file against real resources
  • show - Use for inspecting Terraform state or plan
  • taint - Use for manually marking resources for recreation
  • untaint - Use for manually unmarking a resource as tainted
  • validate - Use for validating terraform files
  • version - Use for printing Terraform version
  • workspace - Use for workspace management

What are the best practices of Terraform?

Code structure

Writing terraform code better having several files split logically like this:

  • main.tf - call modules, locals, and data-sources to create all resources
    variables.tf - contains declarations of variables used in main.tf
    outputs.tf - includes outputs from the resources created in main.tf
    terraform.tfvars: will be used to pass values to variables.

Remote backend

Use a remote backend such as S3 to store tfstate files. Backends have two main features: state locking and remote state storage. Locking prevents two executions from happening at the same time. And remote state storage allows you to put your state in a remote yet accessible location.

Naming conventions

General conventions

Use _ (underscore) instead of - (dash) in all: resource names, data source names, variable names, outputs.
Beware that existing cloud resources have many hidden restrictions in their naming conventions.
Only use lowercase letters and numbers.

Resource and data source arguments

Do not repeat resource type in resource name (not partially, nor completely):
Good: resource "aws_route_table" "public" {}
Bad: resource "aws_route_table" "public_route_table" {}
Include tags argument, if supported by resource as the last real argument, followed by depends_on and lifecycle, if necessary. All of these should be separated by a single empty line.

ReadME

Generate README for each module with input and output variables

Name
Description
Type
Default
Required

instance_count
Number of EC2 instances
String
1
NO
welcome_page
Content to be displayed at home page
String
-
YES

A Comprehensive Approach

To learn more about Configuration management and Compliance and governance and provisioning IT infrastructure we recommend following steps :