OpenTofu v1.8: A Detailed Look at the Upcoming Release

OpenTofu is heading fast to its next stable release, and you can already start testing it yourself using the files provided here. 

As with the previous version, which introduced the long-awaited state file encryption, this release also introduces highly anticipated features, as we continue to chart our roadmap based on community feedback.

In this post, I’ll dive into these features, discuss their use cases, and explore the challenges they will help solve for OpenTofu users.

Variables Support in Backend Configurations

One of the most requested features for Terraform CLI over the years has been the ability to use variables in the backend configuration block. This flexible feature helps keep your code DRY and coherent while minimizing the chance of issues caused by various workarounds, which I’ll discuss below.

To demonstrate the type of issues this new feature solves, let's consider an example of Terraform stacks hosted across multiple workspaces (dev, staging, production,etc) with different regions for compliance reasons. 

While using variables to set AWS regions in resources is straightforward, it previously wasn't possible to do the same for backend configurations.

And so, while setting AWS regions with variables worked seamlessly for the following provider configurations:

variable "region" {
  default = "us-east-1"
}

provider "aws" {
  region = var.region
}

For the backend, this wasn’t possible:

terraform {
  backend "s3" {
    bucket = "my-state-bucket"
    key    = "my-key"
    region = var.region // THIS DOESN’T WORK
  }
}

Existing (Meh) Workarounds 

Where there is a will, there’s a way. Over time, several workarounds emerged to deal with the lack of variable backend support – each with its own drawback. Here are a few I used or encountered over the years:

Pipeline Flags

Using [.code]-backend-config[.code] flags to specify backend details during [.code] init[.code] runs. For instance, in the case of the example above:

tofu init -backend-config="region=eu-west-1"

Dynamic Configuration Files

Another workaround would be to use separate backend configuration files, dynamically generated by the pipeline:

tofu init -backend-config="my.eu-west-1.backend.tf"

Both of these solutions technically work. However, they are cumbersome and prone to errors, requiring the user to set the correct configuration, thus increasing the risk of potential issues and errors.

In this case, for instance, they would require you to provide specific configuration overrides in the [.code]tofu init[.code] command. You would have to manually input the correct backend alongside your variable. A mismatch could result in running with the wrong state, which could be destructive.

Restructured Configuration

Lastly, you could organize your configuration folders based on backend configurations. 

Here, we put our actual configuration inside a module called “my-main-module”, and have separate main.tf files for each region. In each main.tf there would be a different backend configuration (per region) which would be called the “my-main-module” module, with folders looking something like this:

.
├── regions/
│   ├── us-east-1/
│   │   └── main.tf
│   └── eu-west-1/
│       └── main.tf
└── my-main-module/

This approach would make the backend configuration static and replicable with minimal changes. However, it also introduces unnecessary complexity, adding an extra layer of inner modules that muddle the readability of both the plan and apply logs. 

This approach is not dynamic and requires constant tinkering. For instance, supporting another region would require adding new code and folders to the existing structure.

A Better Option

Now, let’s take a look at how the code will look with OpenTofu 1.8 and backend configurations that now support interpolations using variables and locals:

variable "region" {
  default = "us-east-1"
}

provider "aws" {
  region = var.region
}

terraform {
  backend "s3" {
    bucket = "my-state-bucket"
    key    = "my-key"
    region = var.region // This now works
  }
}

Simple, straightforward, and DRY. Plus, you can also use [.code]locals[.code] to ensure that your backend configuration is consistent and correct based on the workspace:

locals {
  region_per_workspace = {
    dev-us = "us-east-1"
    dev-eu = "eu-west-1"
  }
  region = local.region_per_workspace[terraform.workspace]
}

terraform {
  backend "s3" {
    bucket = "my-state-bucket"
    key    = "my-key"
    region = local.region
  }
}

This setup ensures that your backend configuration matches the [.code]var.region[.code], preventing misconfigurations and potential resource destruction.

Moreover, the support for variables in backend configurations also extends to credentials and other dynamic parameters. 

For instance, using [.code]assume_role[.code] for different AWS accounts is just one of many things that can now be simplified, like so:

terraform {
  backend "s3" {
    bucket = "my-state-bucket"
    key    = "my-key"
    region = "us-east-1"
    assume_role = {
      role_arn = "arn:aws:iam::${var.account_id}:role/role-name"
    }
  }
}

Importantly, support for variables and locals was added not just for backend configuration, and you can now use them in module sources and in encryption blocks.

For more information, see the OpenTofu release blog post

Now, with this out of the way, let’s discuss the other new features I’m excited about, starting with the new .tofu files.

Introducing .tofu Files 

In short, the addition of .tofu files offers an alternative way to run OpenTofu configurations in conjunction with legacy Terraform code, stored in .tf files. 

This opens the door to multiple benefits, such as enabling module authors to leverage OpenTofu-specific features without breaking compatibility with Terraform. 

It also helps new users test OpenTofu-specific features, facilitating a smooth transition from Terraform to OpenTofu and ensuring that configurations remain reliable and easy to maintain during this transition.

Let’s break it down:

1. Streamlining Modules Support for OpenTofu Features

Any module compatible with both Terraform and OpenTofu can only use features supported or shared by both frameworks.

So far, this has not been an issue, but looking forward we plan to have OpenTofu add new language features, which Terraform might not add. 

With .tofu files, module authors can easily benefit from these OpenTofu additions without doing a lot of heavy lifting.

For example, let’s say you're the maintainer of a module, and you would like to deprecate a variable. 

Variable deprecation is a feature request that has been accepted in OpenTofu, and will probably be introduced in the following OpenTofu versions. Since variable deprecation would use a syntax that Terraform would not be familiar with, simply adding the “deprecation” field would not work when using the module from the Terraform CLI.

Let’s say that the module has the following variable definition:

// variables.tf
variable “region” {
  default = “us-east-1”
}

We would like to add a deprecation warning for that variable. For that, you could create a .tofu file that would define the variable with deprecation:

// variables.tofu
variable “region” {
  default = “us-east-1”
  deprecation = “The region field is deprecated, and will be removed in future versions. Please use the variable `X` instead.”
}

Here we have the existing variables.tf that has defined the “region” variable that we would want to deprecate. We’ve added a variables.tofu file with the same “region” variable declaration, with the addition of a deprecation warning.

When using Terraform it will only pick up the .tf files, and the module will still work as-is, without variable deprecation.

When using OpenTofu, the variables.tofu file would come into play, with variable deprecation inside, displaying the warning to the user.

2. Paving New Paths to Get Started

Making the full switch from Terraform to OpenTofu is generally very straightforward, requiring just a simple replacement of the binary.

However, with .tofu files, you can make it a more gradual transition and start selectively using OpenTofu-specific features while still being able to roll back to Terraform. Or, if you prefer, you can alternate between the OpenTofu stack and Terraform stack as you try things out.

One way or another, .tofu files open up additional options and ensure that a configuration is valid for both Terraform and OpenTofu at all times.

3. Keeping the Tooling Ecosystem Intact

OpenTofu and Terraform share a vast ecosystem, consisting of many third-party tools that help us write and manage our configuration – from basic editing with highlighting and auto-completes, to static code checks, cost estimation tools, etc. 

Similar to the above example for module authors, the recent addition of .tofu files is our way of ensuring ongoing cross-compatibility for all of these tools, with a practical and low-effort solution.

For example, thanks to .tofu files, a syntax highlighter can now easily associate code with OpenTofu configuration and have the encryption block supported and highlighted correctly, with proper auto-completes.

This is just one scenario of course, and it serves to show how the addition of .tofu files will keep the tooling ecosystem intact, in a way that will benefit all users and require minimal adjustment from tool authors and maintainers.