State File in Terraform

The state file in Terraform is essential for tracking the infrastructure it has created. It plays a crucial role in updating and destroying existing infrastructure.

Since the state file stores comprehensive details about the infrastructure, it can also contain sensitive information such as secrets, passwords, and API tokens. This makes the terraform.tfstate file a critical and sensitive asset to manage carefully.

Understanding the Scenario

Imagine a DevOps team of five members working on a Terraform project maintained in a GitHub repository. To update the infrastructure, access to the state file is necessary. However, storing the state file openly in the repository is not advisable. In a production and collaborative environment, the state file must be shared, maintained, and accessed securely by all developers.

To address these concerns, Terraform Remote Backend is used.

Terraform Remote Backend

Using a remote backend is transformative when working with Terraform in teams or at scale.

Remote backends, such as AWS S3, Azure Blob Storage, or Terraform Cloud, offer several benefits:

• Concurrent Collaboration: Multiple team members can work on the same project simultaneously without conflicts.
• Centralised State: The state is stored in a shared location accessible to all team members, ensuring consistency.
• Versioning and History: Some remote backends offer versioning and history tracking, making it easier to roll back changes if needed.

One significant challenge in a team environment is maintaining synchronisation and preventing concurrent modifications. State locking is crucial in collaborative environments to avoid concurrent operations conflicting and causing issues.

State locking ensures that only one operation can modify the state at a time, preventing conflicts and ensuring smooth management of infrastructure by multiple users or automation systems.

Tutorial: Setting Up a Terraform Remote Backend

Step 1: Create a S3 bucket to store terraform.tfstate file and a Dynamodb Table to store Lock IDs

You can create a s3 bucket and dynamodb table with the following main.tf

provider "aws" {

 region = "us-east-1"

}

resource "aws_s3_bucket" "terraform_state" {

 bucket = "terraform-state-backend"

 force_destroy = true

}

resource "aws_dynamodb_table" "terraform_lock" {

 name           = "terraform_state"

 read_capacity  = 5

 write_capacity = 5

 hash_key       = "LockID"

 attribute {

   name = "LockID"

   type = "S"

 }

 tags = {

   "Name" = "DynamoDB Terraform State Lock Table"

 }

}

S3 Bucket

DynamoDB Table

Step 2: Creating your project files

To keep it simple, we will be launching an EC2 Instance

ec2.tf

provider "aws" {

   region = "us-east-1"

}

 ami           = <AMI ID>

resource "aws_instance" "example" {

 instance_type = <INSTANCE_TYPE>

 subnet_id     = <SUBNET_ID>

 key_name      = <KEYPAIR_NAME>

}

backend.tf

terraform {

 backend "s3" {

   bucket = "terraform-state-backend"

   key    = "terraform.tfstate"

   region = "us-east-1"

   dynamodb_table = "terraform_state"

 }

}

Step 3: Creating Infrastructure

Now we are ready to begin with infrastructure deployment

terraform init

It is a good practice to review what you are going to create, go through the plan before applying

terraform plan

And when you are sure to move ahead with the changes finally, we apply

terraform apply --auto-approve

Now you can view the terraform.tfstate file is successfully created in the S3 bucket and terraform will automatically fetch it from the bucket for further activity with the infrastructure.

terraform.tfstate in bucket

With S3 you can control access to this sensitive file. This solves 2 of our issues — restricted access and security.

This is how you can set up a Terraform Remote Backend for secured and restricted access to your Terraform state files.

Thank you for reading!