Infrastructure as Code using Terraform - IaC Complete Guide

Introduction to Terraform

Earlier, it was a challenging and hectic job to manage IT infrastructures. The System Administrator had to manually manage all the underlying services, hardware, and software needed for the entire application system to work.

In recent years, the IT industry has seen a revolution in terms of tools and technologies that work towards automating the process of deploying and managing the Infrastructure. And this is what we term it as IaC.

Writing Infrastructure in configuration files or code is referred to as Infrastructure as code. This code acts as the single source of truth for all the components required in the Infrastructure, such as Load Balancers, Virtual Machines, Gateways, etc.
Various IaC tools are available, each solving its purpose, but now the question comes: which tool will best suit your goal.
Some of the popular IaC tools are:-

1. Terraform
2. Puppet
3. Ansible
4. Chef
5. Cloud formation

The Infrastructure a Code helps DevOps teams use technologies to manage and configure infrastructure. Taken From Article, Principles of Infrastructure as Code

What is Terraform?

1. Infrastructure as code.
2. It is a tool to manage virtual server life cycles (AWS, VMWare, etc.)
3. It is a tool to manage supporting services (DNS, Email)
4. It is a tool to manage system services (MySQL, PostgreSQL)
5. Configuration files can be HCL or JSON.
6. Created by Hashicorp (Vagrant et al.)
7. Written in Go

Why Infrastructure as Code is important?

The main goal of DevOps is to perform more efficient software delivery and to make delivery efficient and quick. We need some tools like it, which help companies with infrastructure as a code and automation. It is changing the DevOps world by changing the way infrastructure is managed and making it faster and more efficient for executing DevOps. There exist technologies like Ansible, Chef, or Puppet for automating and provisioning software. It also uses the same law, “infrastructure as code,” but it targets the automation of infrastructure itself. Complete cloud infrastructure like instances, IPs, volumes, and networking can be defined easily.

Why should you use it to manage your Infrastructure?

The below mentioned are the usages of it:

An orchestrator and not an Automation Tool

What do we mean by that? Automation focuses on a single task, whereas orchestration means creating a workflow and combining multiple automation tasks.

It follows a Declarative approach and not a Procedural

In a declarative approach, you will tell what you need and not how it is to be done. Just say what you want in your Infrastructure, and it will manage all the necessary steps to get things done.

Multiple Provider Support

It comes with the feature to provide support for multiple public cloud service providers like Google Cloud Platform, AWS, and Azure.

Kubernetes also follows IaC principles which mean all of our deployment is written in the form of code. Taken From Article, IaC Principles for Kubernetes Configuration Management

How to manage your Infrastructure as Code using Terraform?

It is a tool that lets you manage, build, and version your Infrastructure using configuration files. It is a convenient tool that enables you to deploy a wide range of resources, such as physical servers, networking interfaces, and Load balancers, on various platforms such as AWS, GCP, and Azure. The same code needs to be replicated in different environments, ensuring that the same Infrastructure is applied everywhere.

For example, say you are working with AWS as your cloud provider and would like to spin up several EC2 instances of a specific type. You would define the type and number of instances in a configuration file, and it would use that to communicate with the AWS API to create those instances. The same file could then be used to adjust the configuration, for example, increasing or decreasing the number of instances. Because Infrastructure typically includes many more components than just compute instances, we use a feature called modules to combine multiple infrastructure components into large, reusable, and shareable chunks.

What are the features of Infrastructure as Code?

The features are listed below:

Infrastructure as a Code

A high-level configuration syntax is used for describing support. It allows a blueprint of our data center to be versioned and treated it you would any other code. Also, infrastructure can be shared and re-used.

Execution Plans

It has a “planning” step from which it generates an execution plan. This execution plan shows what it will do when it is applied. This eliminates any surprises when it manipulates infrastructure.

Resource Graph

A graph of all resources is built by it and parallelizes the creation and modification of any non-dependent resources. By doing so, it builds infrastructure as efficiently as possible, and operators get insight into dependencies in their infrastructure.

Change Automation

With minimal human interaction, complex changesets can be applied to infrastructure, what it will change, and in what order can be known by the previously mentioned execution plan and resource graph, which helps in avoiding many possible human errors.

How does Infrastructure as Code work?

It is split into two parts: core and plugins. Its core communicates with its plugin. Its plugins expose an implementation for specific services such as AWS bash.

What is its Core?

It’s a binary written in Go programming language. The compiled binary corresponds to the CLI terraform.
The core is responsible for :

Reading the configuration files, i.e., IaC.
State management of various resources.
Construction of resource graph.
Execution of plan.
Communication with plugins.

What are the Plugins?

Plugins are executable binaries written in Go.

Requirements for Terraform setup

Terraform binary
Configuration files (in which you write IaC)
State file

Terraform configuration files (*.tf)

You write every configuration in files with an extension as .tf. You’ll be mostly dealing with resources in configuration files. Resources are the services or components of your Infrastructure, such as Load Balancers, Virtual machines, etc. And you pass various arguments in a resource you want to set up.
For example:

main.tf
resource "aws_instance" "web" {
ami = data.aws_ami.ubuntu.id
instance_type = "t3.micro"
tags = {
Name = "HelloWorld"
}
}

The above configuration file main.tf, defines a resource, i.e., EC2 instance in AWS. Inside the resource block, we have passed the desired arguments as:
ami- AMI id to be issued for our EC2.
Instance_type
Tags - tags for our EC2
This is how we can define other resources, as well.
We can now use it apply to deploy this resource on AWS.

What is Terraform state?

The most crucial part of it is its state file (terraform.tfstate), which stores all the information related to the deployed Infrastructure and mapping between various components. It then refers to this state file while deploying Infrastructure to check dependencies among resources.

Why is a state file required?

While deploying resources, for example, EC2, as in the example above, you don't specify the ARN or instance ID for the EC2. It gets dynamically assigned, so it needs to store all such information so that internally, it can detect changes if you update the instance_type for the EC2 in the future.

State file is the single source of truth for your Infrastructure, so you need to keep this very safe and ensure that your team always refers to the latest version of terraform.tfstate file to avoid conflicts. By default, the state file is stored locally, but keeping it locally is only beneficial if just one developer is working on managing the Infrastructure. However, the best practice is to keep file in some remote backend such as S3 bucket.

Following example shows the backend configuration as S3 bucket.

terraform {
backend "s3" {
bucket = "mybucket"
key = "path/to/my/key"
region = "us-east-1"
}
}

Backend Initialisation

An initialization step needs to be performed every time there is a change in backend configuration; the command used is it into.
This command does the following:
Initializes backend
Configures its modules (will cover later).

Workflow

Write IAC, i.e., configuration files.
See what changes it will reflect using its plan.
Deploy those changes using it and apply

Plan

The plan will represent a list of changes that your IAC will make. It doesn't actually affect anything, and it just shows what new will be created/deleted/modified. Therefore, it is safe to use it plans multiple times.

Interpreting its Plan

Resources can be easily interpreted by their names. For example,
Syntax: <Resource type>.< resource name>
Example, aws_instance.this.
Resources inside a module can be interpreted by:
Syntax: module.<module_name>.<Resource type>. < resource name>
Example module.my_virtual_machines.aws_instance.this
Its plan shows five actions for the resources:
[+] Add - Creation of new resource.
[ - ] Delete - Removal of existing resource.
[~] Modify in-place - Existing resources will be modified, and some new parameters will be added to them. It plans will also show you which new parameters will be modified/ added.
[- / +] Terraform will delete and re-create the same resource. This happens when it's not possible to modify a parameter in place.

Sample Plan

An execution plan has been generated and is shown below.

20:10:18 Resource actions are indicated with the following symbols:
20:10:18 + create
20:10:18
20:10:18 Terraform will perform the following actions:
20:10:18
20:10:18 # aws_cloudwatch_log_group.log_group_containers[0] will be created
20:10:18 + resource "aws_cloudwatch_log_group" "log_group_containers" {
20:10:18 + arn = (known after apply)
20:10:18 + id = (known after apply)
20:10:18 + name = "/eks/dev-scipher-fx-eks-cluster-2/containers"
20:10:18 + retention_in_days = 14
20:10:18 + tags = {
20:10:18 + "Environment" = "dev"
20:10:18 + "Service" = "EKS Cluster"
20:10:18 + "Terraform" = "true"
20:10:18 }
20:10:18 }
20:10:18
20:10:18 # aws_cloudwatch_log_group.log_group_host[0] will be created
20:10:18 + resource "aws_cloudwatch_log_group" "log_group_host" {
20:10:18 + arn = (known after apply)
20:10:18 + id = (known after apply)
20:10:18 + name = "/eks/dev-scipher-fx-eks-cluster-2/host"
20:10:18 + retention_in_days = 14
20:10:18 + tags = {
20:10:18 + "Environment" = "dev"
20:10:18 + "Service" = "EKS Cluster"
20:10:18 + "Terraform" = "true"
20:10:18 }
20:10:18 }

In the above example, the plan shows:
Two new Cloudwatch groups being created in AWS.
Log groups retention_in_days being set to 14 days
Tags being added to log groups

Terraform Apply

On applying a given plan, it locks the state. It will perform all the actions stated in the planning stage and deploy those resources. Some resources might take time; for example, AWS DocumentDB may take up to 15 minutes. Until that, it will block all other operations.
On completion, it updates the state file.

Modules

All the configuration files, together with a folder, are called a module. Modules allow us to create a reusable code that can be referred to again with the required parameters. Consider it as just like functions in any programming language where we just have to call the process anywhere and pass the necessary set of parameters.
Using a module in a configuration file is similar to using resources, as we have seen above; the only difference is the clause “module.”

module "moduleName" {
source = "module/path"
\CONFIG
/
}

The source path tells where the module can be found.
CONFIG block consists of a set of variables or arguments that can be passed to the module and belong to the module.
For example:
If the main module has a variable log_retention_period, we can pass its value differently for different environments, such as:
For Dev environment:

module "cloud_watch_log_groups" {
source = "module/path"
Log_retention_period = 14}

Similarly for the Prod environment:
module "cloud_watch_log_groups" {
source = "module/path"
Log_retention_period = 21
}

Thus, as you can see, we can reuse the same module to deploy Infrastructure for multiple environments.

What are the Advantages of Infrastructure as Code?

1. Orchestration, not merely Configuration: For server configuration, all the configuration management tools were created, and to manage and to install the software already existed in the server / virtual machine where their primary goal.
2. It focuses more on Aerver Provisioning: When the complete cloud infrastructure is considered as code, and all the parameters were combined in a configuration file, all the members of the team can easily collaborate on them as they would do any other code.
3. Multi-Provider - The most versatile feature of it is that it supports multi-cloud orchestration like AWS, Azure, OpenStack, etc. and also premises deployments. It is beneficial when using two different resources from two various cloud providers at the same time.
4. Immutable Infrastructure - When using Chef, Puppet, Salt runs any software updates on servers; this often leads to configuration drift when differences in the configuration lead to bugs that lead to security breaches. It solves this issue by using an immutable infrastructure approach in which every change in configuration leads to a separate configuration snapshot which means de-provisioning the old one and deployment of a new one. Updating the development environment in this way goes smooth and bug-free and returning the old configuration is easy, like rolling back to a specific version.
5. Syntax - HCL (HashiCorp Configuration Language) is a custom language that is used by it.
6. Dry Runs - It uses a command known as “terraform plan,” which creates an execution plan; by using this execution plan, we can check whether the set of changes meets the expectation without performing any changes to real resources or state. For example, by running a plan before committing a change to version control, check whether it will work as expected or not.
7. Client-only Architecture - It eliminates the need for additional checks for provisioning the infrastructure, which leverages the cloud provider’s API. In Ansible, this is done by connecting through SSH, but with limitations. It works on APIs and has a wide variety of options, which helps to make it more secure, reliable, and easy to use.
8. Super Portability - There is only a single tool and single language for describing the infrastructure that is used for multiple cloud providers. The problem of migrating to vendors is not a problem.