Terraform 'for' Expression: How to Dynamically Provision Infrastructure

What is Terraform 'for' Expression

Terraform "For Expression" is widely used, particularly in Terraform modules. A [.code]for[.code] expression allows you to create complex type values by transforming other complex type values.

This feature is not only beneficial in modules but also in your Main Infrastructure Code. Many infrastructure engineers are unaware of how to utilize these expressions and often end up with excessive repetitive code. In this guide, I will demonstrate an example to help you understand the advantages of using this expression, enabling you to write clean code and avoid repetitive declarations.

Terraform 'for' Expression Use Case with AWS

Imagine a scenario where we need to provision multiple AWS EC2 Instances but prefer not to place them within an Auto Scaling Group for management purposes. Additionally, we aim to incorporate a Load Balancer to evenly distribute inbound traffic among these multiple instances.

Getting Started

1. Let’s start by creating a local variable to specify how many nodes we want to deploy:

locals {
  nodes_count = 40
}

2. Let’s create the EC2 Instances based on the count. We will be using the Community Terraform Module to deploy the EC2 Instances.

module "lab_nodes" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "5.0.0"


  count = local.nodes_count


  name = "node-${count.index}"
}

3. Let’s create the Load Balancer and register our instances under our target group. We will be using the Community Terraform Module to deploy an Application Load Balancer. If you check the examples of usage of this module, you will see that each target has its own map specifying the Instance ID and the port. For example:

      targets = {
        my_ec2 = {
          target_id = aws_instance.this.id
          port      = 80
        },
        my_ec2_again = {
          target_id = aws_instance.this.id
          port      = 8080
        }
      }

Now, consider the scenario where we have already created 40 instances, and now we need to generate similar maps for each of these instances. Can you imagine the sheer size and repetition of the code in such a case? Not only would it be impractical, but it would also lack scalability. Now, let's picture a situation where the instance count needs to be increased from 40 to 100. Just think about the number of lines you would have to add in order to accommodate this change.

To avoid this to happen, we will be creating our ALB using the following code:

module "lab_alb" {
  source  = "terraform-aws-modules/alb/aws"
  version = "8.6.0"


  name               = "lab-alb"
  load_balancer_type = "application"


  target_groups = [
    {
      name             = "lab-tg"
      backend_protocol = "HTTP"
      backend_port     = 80
      target_type      = "instance"
      targets = {
        for i in range(local.nodes_count):
        i => {
          target_id = module.lab_nodes[i].id
          port      = 80
        }
      }
    }
  ]
}

Using 'for_each' instead of 'count'

1. In some cases it is better to use for_each instead of count, especially when you care about ordering. Also, [.code]for_each[.code] works based on maps or sets, so, we will be dynamically building a map from our nodes count.

locals {
  nodes_count = 40
  nodes_map = {
    for i in range(local.nodes_count):
      "node-${i}" => {}
  }
}

2. Let’s deploy the EC2 Servers:

module "lab_nodes" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "5.0.0"


  for_each = local.nodes_map


  name = each.key
}

3. Finally, let’s deploy our ALB:

module "lab_alb" {
  source  = "terraform-aws-modules/alb/aws"
  version = "8.6.0"


  name               = "lab-alb"
  load_balancer_type = "application"


  target_groups = [
    {
      name             = "lab-alb"
      backend_protocol = "HTTP"
      backend_port     = 80
      target_type      = "instance"
      targets = {
        for i, k in local.nodes_map :
        i => {
          target_id = module.lab_nodes[i].id
          port      = 80
        }
           }
    }
  ]
}

Here, we iterate over the local.nodes_map using a for loop. For each node, we dynamically assign the target_id as the corresponding EC2 instance ID, obtained from the module.lab_nodes output. Using dynamic code, we can effortlessly configure the ALB to target the appropriate instances based on the nodes_map.

Terraform 'for' Expression Code Explanation

In this Terraform code snippet, the for expressions are used to dynamically generate resources based on the specified logic. Let's break down what each section is doing:

1. [.code]locals[.code] block:‍

• [.code]nodes_count = 40[.code]: This defines a local variable [.code]nodes_count[.code] with a value of 40.
• [.code]nodes_map[.code]: This defines a local variable [.code]nodes_map[.code] as a map.
• The [.code]for[.code] expression [.code]for i in range(local.nodes_count)[.code] is used to iterate over a range of numbers from 0 to [.code]nodes_count - 1[.code].
• Inside the [.code]for[.code] expression, [.code]"node-${i}" => {}[.code] creates key-value pairs where the key is a string [.code]"node-${i}"[.code] and the value is an empty object [.code]{}[.code]. This effectively generates a map of 40 elements with keys like [.code]"node-0"[.code], [.code]"node-1"[.code], ..., [.code]"node-39"[.code].

2. ‍[.code] module "lab_nodes"[.code] block:

• This module creates EC2 instances using the Terraform AWS module [.code]terraform-aws-modules/ec2-instance[.code].
• The [.code]for_each = local.nodes_map[.code] meta argument tells Terraform to create an instance for each element in the [.code]nodes_map[.code] map.
• [.code]name = each.key[.code] sets the name of each instance to the key of the corresponding element in [.code]nodes_map[.code].

3. ‍[.code]module "lab_alb"[.code] block:

• This module creates an Application Load Balancer (ALB) using the Terraform AWS module [.code]terraform-aws-modules/alb[.code].
• [.code]name[.code] and [.code]load_balancer_type[.code] are basic configuration attributes.
• [.code]target_groups[.code] defines the target groups for the ALB.
• The [.code]for[.code] expression [.code] for i, k in local.nodes_map[.code] is used to iterate over each key-value pair in [.code]nodes_map[.code].
• Inside the [.code]for[.code] expression, [.code]i[.code] represents the key (e.g., [.code]"node-0"[.code], [.code]"node-1"[.code]) and [.code]k[.code] represents the corresponding value (an empty object in this case).
• The [.code]targets[.code] block within [.code]target_groups[.code] defines the targets for the target group.
• The [.code]for[.code] expression [.code]for i, k in local.nodes_map[.code] is used again to iterate over each key-value pair in [.code]nodes_map[.code].
• Inside the [.code]for[.code] expression, [.code]i[.code] represents the key (e.g., [.code]"node-0"[.code], [.code]"node-1"[.code]) and [.code]k[.code] represents the corresponding value (an empty object).
• [.code]i => { target_id = module.lab_nodes[i].id, port = 80 }[.code] specifies the target ID and port for each target based on the corresponding EC2 instance created by the [.code]module.lab_nodes[.code] module.

In summary, the for expressions in this code snippet are used to generate a dynamic number of EC2 instances and associate them with an ALB target group. The number of instances is determined by the [.code]nodes_count[.code] variable, and the instances are named according to the [.code]nodes_map[.code]. The ALB target group is configured to have targets corresponding to the EC2 instances created by the [.code]module.lab_nodes[.code] module.

Conclusion

In conclusion, the dynamic code showcased in this article demonstrates the power of Terraform's ability to handle dynamic infrastructure deployments. We can efficiently provision and configure resources by utilizing features like the for_each loop and dynamic mappings, making our deployments more flexible, scalable, and maintainable. Embracing dynamic code empowers us to build robust infrastructure as code solutions adapt to changing requirements and enable seamless collaboration among team members.

I strongly encourage you to learn and master the usage of these expressions in your code. Remember, there is always room for improvement. Investing time in crafting a well-written Terraform code will pay off in the future, as it will save you time and accelerate the progress of your infrastructure projects.

References:

Terraform For Expression Official Documentation:

 https://developer.hashicorp.com/terraform/language/expressions/for

Terraform Community Module for EC2:

https://registry.terraform.io/modules/terraform-aws-modules/ec2-instance/aws/latest

Terraform Community Module for ALB:

https://registry.terraform.io/modules/terraform-aws-modules/alb/aws/latest