Terraform for_each: Examples, Tips and Best Practices

Why do we need Looping in Terraform?

When managing Infrastructure-as-code (IaC) with the Terraform CLI, one often encounters scenarios where multiple resources that are similar but not identical need to be separately created. 

This could range from deploying several instances across different availability zones, setting up multiple DNS records, or managing numerous user accounts. Writing out configurations manually for each resource becomes tedious and introduces a higher chance of errors and inconsistencies.

This is where looping in Terraform comes into play. Looping constructs, like the [.code]for[.code] expression, [.code]for_each[.code], and [.code]count[.code] meta-arguments, provide a way to generate similar resources dynamically based on a collection or count.

Meta-arguments and Expressions for Terraform Looping

Meta-arguments, in a nutshell, are unique arguments that can be defined for all Terraform resources, altering specific behaviors of resources, such as their lifecycle, how they are provisioned, and their relationship with other resources. 

Expressions in Terraform are used to reference or compute values within your infrastructure configuration (like dynamic calculations, data access, and resource referencing).

These are mainly used in a Terraform resource block or a module block.

There are five different types of meta-arguments for Terraform resources, but we are going to focus on expressions and meta-arguments that help in looping for Terraform resources:

1.[.code]for_each[.code] - a meta-argument used to create multiple instances of a resource or module.  As the name implies, the [.code]for_each[.code] argument takes a map or a set of strings, creating an instance for each item. It provides more flexibility than count by allowing you to use complex data structures and access each instance with a unique identifier.

2.[.code]count[.code] - a meta-argument allows you to create multiple instances of a resource based on the given count. This is useful for creating similar resources without having to duplicate configuration blocks. For example, if [.code]count=5[.code] for an EC2 instance resource configuration, Terraform creates five of those instances in your cloud environment.

3.[.code]for[.code] - a versatile expression for iterating over and manipulating collections such as lists, sets, and maps. The [.code]for[.code] expression can be used to iterate over elements in a collection and apply a transformation to each element, optionally filtering elements based on a condition.

For example:

locals {
  original_set = {1, 2, 3, 4, 5}
  even_set = {for i in local.original_set : i if i % 2 == 0}
}

This creates an even_set containing only the even numbers from [.code]original_set[.code].

‍for vs. for_each vs. count‍

Here are some key points that differentiate the [.code]for[.code] expression from the [.code]for_each[.code] and [.code]count[.code] meta arguments.

How does for_each work?

Let us take a real-world example to better understand the [.code]for_each[.code] meta-argument.

Say you have a map of instance configurations where the string value for each key is an identifier for the EC2 instance, and the value is another map containing the instance type and AMI ID.

#main.tf
variable "instances" {
  description = "Map of instance configurations"
  type = map(object({
    ami           = string
    instance_type = string
  }))
  default = {
    "amzlinux" = {
      ami           = "ami-02d3fd86e6a2f5122"
      instance_type = "t2.micro"
    },
    "ubuntu" = {
      ami           = "ami-0ce2cb35386fc22e9"
      instance_type = "t2.small"
    }
  }
}
resource "aws_instance" "servers" {
  for_each      = var.instances
  ami           = each.value.ami
  instance_type = each.value.instance_type
  tags = {
    Name = "env0-Server-${each.key}"
  }
}

Let’s break everything down:

• [.code]variable "instances"[.code] defines a map where each element represents an EC2 instance configuration. For instance, "amzlinux" and "ubuntu" are identifiers for these configurations, each specifying an AMI ID and an instance type.
• [.code]resource "aws_instance" "servers"[.code] uses [.code]for_each[.code] to iterate over each element in the [.code]var.instances[.code] map. For each element, it creates an EC2 instance with the specified AMI and instance type.
• [.code]each.key[.code] in this context refers to the key in the map (e.g., "amzlinux", "ubuntu"), which we use to uniquely name each instance with the [.code]Name[.code] tag.
• [.code]each.value.ami[.code] and [.code]each.value.instance_type[.code] access the nested values for each instance's configuration.

After successfully running the Terraform workflow (init->plan->apply), we have provisioned these two instances using [.code]for_each[.code].

‍

Collections for for_each

‍Maps

Maps are collections of key-value pairs. When using [.code]for_each[.code] with maps, each iteration gives you access to both the map key and the value of the current item. Maps are ideal when you need to associate specific attributes or configurations with unique identifiers.

#Example config
variable "instance_tags" {
  type = map(string)
  default = {
    "Role" = "Web-server"
    "Environment" = "Production"
  }
}
resource "aws_instance" "server_tags" {
  for_each = var.instance_tags
  # Other Configuration...
  tags = {
    "${each.key}" = "${each.value}"
  }
}

Sets

Sets are collections of unique values. When iterating over a set with [.code]for_each[.code], the value for [.code]each.key[.code] and [.code]each.value[.code] will be the same since sets do not have key-value pairs but just a list of unique values.

Sets are useful when you need to ensure uniqueness and don't require associated values.

#Example config
variable "availability_zones" {
  type = set(string)
  default = ["us-west-2a", "us-west-2b"]
}
resource "aws_subnet" "list_subnets" {
  for_each = var.availability_zones
  availability_zone = each.key
  # Other Configuration...
}

Lists

Directly, [.code]for_each[.code] cannot iterate over lists because lists do not provide a unique key for each item.

However, you can use the [.code]tomap()[.code] or [.code]toset()[.code] function to convert a list into a set or a map, allowing [.code]for_each[.code] to iterate over it.

#Example config
variable "availability_zones" {
  type = list(string)
  default = ["us-east-1a", "us-east-1b"]
}
resource "aws_subnet" "example" {
  for_each = toset(var.availability_zones)    #or tomap(var.availability_zones)
  availability_zone = each.key
  # Other configurations...
}

Practical Use Cases for for_each

1. Resource Chaining

Resource chaining involves creating dependencies between resources where the configuration of one resource depends on the output of another. This is common in infrastructure setups where certain resources must be provisioned sequentially.

Let us take a common scenario of setting up a VPC and deploying subnets within the VPC, to better understand resource chaining.

#variable "networks" {
  description = "VPC Network Map"
  type = map(object({
    cidr = string
  }))
  default = {
    "network1" = {
      cidr = "10.0.1.0/24"
    },
    "network2" = {
      cidr = "10.0.2.0/24"
    }
  }
}
resource "aws_vpc" "main" {
  for_each = var.networks
  cidr_block = each.value.cidr
  tags = {
    Name = "VPC-${each.key}"
  }
}
resource "aws_subnet" "subnets" {
  for_each          = var.networks
  vpc_id            = aws_vpc.main[each.key].id
  cidr_block        = each.value.cidr
  availability_zone = "us-west-2a"
  tags = {
    Name = "Subnet-${each.key}"
  }
}

In this example, each VPC is created based on the networks map, and then a subnet is created within each VPC. The [.code]aws_subnet[.code] resource uses the ID of the [.code]aws_vpc[.code] created in the same [.code]for_each[.code] loop, demonstrating resource chaining.

2. Tagging Resources Dynamically‍

Dynamic tagging allows you to assign metadata to resources based on their configuration or other dynamic inputs, improving resource management, billing, and automation.

We can take an example of dynamically tagging s3 buckets using [.code]for_each[.code]:

variable "buckets" {
  description = "Map of S3 bucket configurations"
  type = map(object({
    name = string
    tags = map(string)
  }))
  default = {
    "bucket1" = {
      name = "env0-app-logs",
      tags = {
        "Environment" = "Production",
        "Application" = "Logging",
      }
    },
    "bucket2" = {
      name = "env0-app-data",
      tags = {
        "Environment" = "Staging",
        "Application" = "DataStore",
      }
    }
  }
}
resource "aws_s3_bucket" "env0_app_bucket" {
  for_each = var.buckets
  bucket = each.value.name
  tags = each.value.tags
}

This setup dynamically applies tags to each S3 bucket based on the tags defined in the [.code]buckets[.code] map.

3. Deploying Resources to Multiple Regions

Deploying resources across multiple regions can enhance disaster recovery and reduce latency. [.code]for_each[.code] can be used to manage such deployments efficiently.

We can deploy s3 buckets to multiple regions using [.code]for_each[.code] like this example below.

variable "regions" {
  description = "Regions to deploy S3 buckets"
  type        = map(string)
  default     = {
    "us-east-1" = "env0-app-us-east-1",
    "eu-central-1" = "env0-app-eu-central-1"
  }
}
provider "aws" {
  alias  = "useast1"
  region = "us-east-1"
}
provider "aws" {
  alias  = "eucentral1"
  region = "eu-central-1"
}
resource "aws_s3_bucket" "multi_region_buckets" {
  for_each = var.regions
  bucket   = each.value
  provider = each.key == "us-east-1" ? aws.useast1 : aws.eucentral1
}

In this example, S3 buckets are created in both the [.code]us-east-1[.code] and [.code]eu-central-1[.code] regions, using separate provider instances for each region. The provider attribute dynamically selects the correct provider based on the region key from the regions map.

Advantages of using for_each‍

1. Dynamic Resource Management

[.code]for_each[.code] enables the dynamic creation, management, and destruction of resources based on collections (maps or sets). This dynamic approach allows infrastructure to adjust automatically to changes in the input data without requiring manual updates to the Terraform configuration.

For example, you can write an IaC for infrastructure provisioning to provision a dynamic number of S3 buckets based on a list of project names like the example below:

variable "project_names" {
  type = set(string)
  default = ["DisasterRecovery", "VPCNetworkSetup"]
}
resource "aws_s3_bucket" "project_bucket" {
  for_each = var.project_names
  bucket   = each.value
}

2. Conditional Resource Creation

Combined with Terraform's conditional expressions, [.code]for_each[.code] can be used to conditionally create resources based on specific criteria within the data it iterates over. This allows for more granular control over which resources are created, updated, or destroyed.

For instance, you can tailor your IaC in such a way that creates an S3 bucket only if [.code]create=true[.code]:

variable "bucket_configs" {
  type = map(object({
    name   = string
    create = bool
  }))
  default = {
    "bucket1" = { name = "env0-1", create = true },
    "bucket2" = { name = "env0-2", create = false }
  }
}
resource "aws_s3_bucket" "conditional_bucket" {
  for_each = { for k, v in var.bucket_configs : k => v if v.create }
  bucket   = each.value.name
}

3. Improved Code Reusability

Instead of duplicating resource blocks for each instance of a resource, [.code]for_each[.code] allows you to define a single resource (or a module) block that can be applied to each item in a collection (like a map or set).

This approach abstracts the common configuration elements into a single, parameterized block, where the specific details for each resource instance are dynamically derived from the collection it iterates over.

For example, you can incorporate [.code]for_each[.code] with the use of modules to keep your code DRY:

variable "environments" {
  type = map(string)
  default = {
    "dev"  = "ami-02d3fd86e6a2f5122",
    "prod" = "ami-0ce2cb35386fc22e9",
  }
}
module "dry_module_ec2" {
  for_each      = var.environments
  source        = "terraform-aws-modules/ec2-instance/aws"
  ami           = each.value
  instance_type = "t2.micro"
}

Commonly Asked Questions/FAQs‍

Q. Can I use Terraform for_each and count for the same resource?

No, [.code]for_each[.code] and [.code]count[.code] cannot be used together within the same resource or module block. You must choose one based on your use case. 

[.code]count[.code] is used to create a specified number of identical resources. for_each iterates over items in a map or set to create multiple resources with different configurations.

‍Q. What is the use of count.index in Terraform?

The [.code]count.index[.code] in Terraform is a built-in variable that starts with a current zero-based index of the resource created in a block where the count meta-argument is used. 

It's primarily used when you're creating multiple instances of a resource with count and need to differentiate between these individual instances, with a numeric identifier.

‍Q. Can for_each be used with modules?

Yes, [.code]for_each[.code] can be used with Terraform modules, enabling you to create multiple instances of a module based on the items in a map or a set. When using for_each with a module, you can define a collection (map or set) containing the values you want to iterate over.

‍Q. Can count be used to conditionally create a resource?

Yes, [.code]count[.code] can be used to create a resource in Terraform conditionally. You can specify whether to create a resource based on a condition by leveraging the count meta-argument. For example, if the condition evaluates to true, create a single instance of the resource, and prevent the resource creation if it evaluates to false.

‍Q. Is it possible to migrate from count to for_each?

Yes, it is possible to migrate from count to [.code]for_each[.code] in Terraform, but the process requires careful planning and execution. You can check here for detailed information on migrating [.code]count[.code] to [.code]for_each[.code].

‍