Secrets no one tells you about Infrastructure as Code w/ Terraform

Overview

Hey Guys, Welcome to this blog. In this blog, I will cover all the concepts you need to understand to get started with Terraform. If you are a beginner, this blog will help you understand some of the core architectural concepts of Terraform.

Terraform is an open-source Infrastructure as Code software tool which is used for building, managing & versioning application infrastructures safely and efficiently. Terraform lets you define both cloud and on-prem resources in human-readable configuration files that you can version control, reuse, and share. Before I talk about Terraform you need to understand what is Infrastructure as Code or IaC.

What is Infrastructure as Code

Consider a scenario where you want to configure a server in a virtual instance to host your application.

So on a typical day, you would manually create the instance, configure the private network, open the required ports, configure load balancing, etc. And if you are an advanced user maybe you will do all of this from the Terminal using the CLI of the cloud provider you are using. In this case, all of the features you are using in your application are provided by a cloud provider like AWS, and collectively all these features become the infrastructure for your application. Now, imagine that you need to deploy the same application over 100s of times in different virtual machines with various networks and multiple storage options. so If you try to set up those 100s of servers manually, boi you are fucked 💀.

It would take you days to get the setup done. Also, consider the complexity involved in this and the possibility of you making errors, which adds up to further delays and unnecessary debugging. Not just that, you might be required to build the same environment for different use cases like development, QA, staging, production, and whatnot. So it is not practical to build the infrastructure manually for real-world applications.

Aaand, this is where the beauty of Infrastructure as Code comes into the picture. With the help of Infrastructure as Code or IaC You can write configuration files that will do all of this process automatically without any manual intervention.

So once you run the configuration file, it will automatically configure all the cloud servers for you. This is also a lot safer because there’s no human error, and you can run the configuration file as many times as you want. It is way faster than the manual process because you can set up the entire infrastructure in a matter of minutes. But realistically it depends on the complexity of the requirement.

Just like it can create the infrastructure for you, it can also update and delete the same infrastructure. And that's the general concept behind Infrastructure as Code or IaC.

Tools available in IaC

Now all the tools that are available under Infrastructure as Code, can be divided into two parts - Cloud Provider & Cloud Agnostic.

In Cloud Provider you have tools like AWS CloudFormation, Google Cloud Deployment Manager and Azure Resource Manager etc.

And then tools like Chef, Puppet, and Terraform are cloud agnostic. That means these tools are independent and you can use them in any combination. Anyway, they all pretty much let you specify the infrastructure in code and then create or destroy the infrastructure for you as needed. So you just need to configure the different aspects of the infrastructure like network, storage, security, and deployment.

Advantages of using IaC

Now that we covered what is IaC and the types of tools available under IaC. Let's see what are the advantages you get by using IaC.

Consistent Deployments

One of the biggest advantages is consistent deployments.

Say you have three environments, dev, staging, and production. Whatever you need to do to complete the deployment, needs to be done three times as you have three different environments. Also, you have to make sure that the setup of all three environments, is always the same because you need to minimise the risk of unexpected failure. That’s why Consistency is one of the main advantages of IaC, as it will make sure that all the environments are the same.

Because the infrastructure of all three environments is created by writing a single configuration file and you are executing that same file in different environments. Now obviously the different environment values can be set as per the different naming conventions of those different environments.

Infrastructure validation before deployment

Now there is a very important concept present in Terraform called Infrastructure validation before deployment.

Say you have written some scripts to create a virtual server in the cloud, and after the deployment of the script, you get to know that something is wrong with the server that you created. Now obviously you can delete it and recreate the deployment again. But the problem is you have to pay for it again. So you have to be very sure that your script is configured correctly.

Now, in IaC, you get to know what exactly your infrastructure would look like, even before the deployment. The benefit of this is that if there is any error in the code, you can validate and fix that error before doing the actual deployment. This will reduce deployment errors drastically.

And also you can test your infrastructure code using static code analysis if you are into that.

Deployment Speed

There’s another advantage of Using IaC which is deployment speed. You can legit create 100s of deployment servers just by running a single config file or script. This makes your software deployment process faster with fewer errors and helps the application run smoothly. Overall, the whole process becomes more efficient and increases your productivity to the next level.

Reusability

Another advantage is, that you can reuse the configuration files across different environments. You can scale the actual number of resources or servers just by changing the configuration values in the code. This helps lower the cost of infrastructure management, coz you don't need to spend more money on hiring people to create or manage all the deployment manually.

So Terraform uses its own configuration language known as HashiCorp Configuration Language or HCL to create the config files. You can also use JSON though. Terraform has a planning step, where it generates an execution plan. The execution plan shows what Terraform will do when you call terraform apply. This lets you avoid any errors when deploying the application. When you create or modify multiple resources, Terraform builds a graph of all your resources first. Using that graph Terraform can build infrastructure as efficiently as possible.

Now let's talk about some architectural differences between Terraform and others

Mutable vs Immutable Architecture

when you build an Application Infrastructure, you also need to understand the differences between Mutable vs Immutable architecture. Chef and Puppet are configuration management tools which install and manage software on a machine that already exists. It follows a mutable infrastructure pattern. Mutable pattern means that the servers are continually modified in place. If you have a newer version of your application, you just upgrade the server. If there is a change needed, you tweak the server configuration. Basically mutable infrastructure approach is more of a traditional approach where there used to be a team of people maintaining the servers.

But Terraform provides support for immutable infrastructure. In immutable pattern, these servers are never modified.

For every configuration change, a new server is created and the data & traffic is redirected to that new server. And once the new server is verified, the old ones are decommissioned or deleted. With immutable infrastructure approach, the deployment process becomes more predictable because there are no chances of a failed upgrade or of running into an unexpected state.

Procedural style vs Declarative style

Chef and Ansible follow a procedural style, where you write code that specifies step by step how to achieve some desired end state. In other hands, Terraform, CloudFormation, SaltStack, and Puppet uses a declarative style, where you write code that specifies your desired end state, and the IAC tool itself does the actual work to achieve that state. Procedural code does not fully capture the state of infrastructure, and it limits the reusability of the code. But with Terraform's declarative approach, the code always represents the latest state of your infrastructure. This makes it easy to create reusable code as you don't have to manually look for the current state.

Master and Agents

Chef, Puppet, and SaltStack require a master server for storing the infrastructure state. You interact with the master server and the master server distributes the required updates on the agent machines.

But on the other side, Terraform is primarily masterless. It communicates with cloud providers using the cloud provider's APIs so you don't need to install any agent software, unlike Chef, Puppet, or SaltStack.

The agent software, for example, the Chef client, Puppet agent, Salt Minion, is pre-installed on each server and typically runs in the background on each server, agent software is responsible for installing the latest configuration management updates.

State Files

Terraform follows a stateful management approach. That means it tracks the resources it is managing in the cloud. The state of the infrastructure contains all the information about the provisioned infrastructure and its different configurations.

And the way Terraform uses state files is that it saves the configuration in a state file. When you create an infrastructure using Terraform, it stores all the config files in a state file. Then Terraform uses the state file to map real-world resources of your configuration, keeping track of metadata, and improving performance for large infrastructures.

The state is stored by default in a local file named 'terraform.tfstate'. But it can also be stored remotely, which works better in a team environment. Using state file, Terraform identifies if the resources are created by Terraform or not. It keeps a metadata in this state file. And whenever you change the configuration, it automatically determines which part of the configuration is already created, and which needs to be changed with the help of state files.

These features allows you to make incremental changes in the future by simply changing the same configuration file to match your desired end state.

Here is a sample state file for reference. Basically, it contains the information as structured text file. I will cover this in an upcoming video. That's it about the overview & advantages of Terraform over others.

That's it about Terraform and Infrastructure as Code. Hope you get to learn something new. Thanks for sticking out till this end. See you in another blog.\

Byeyeyeyeyeyey...