Recently, I had the opportunity to give a lightning talk at OpenTofu Day during KubeCon 2024. My session focused on three main strategies for managing OpenTofu configurations across multiple environments such as dev, integration, and production. These strategies included using separate configurations, leveraging workspaces, and utilizing the backend-config flag.

During the talk, I reviewed the benefits and challenges of each approach, showcasing how OpenTofu's Static Evaluation feature can improve and simplify these workflows. I also covered module versioning and demonstrated how Static Evaluation introduces greater flexibility and efficiency in managing configurations.

Whether you're looking to streamline processes, minimize code duplication, or make the most of OpenTofu's features, this session offers practical insights and examples to help you optimize your workflows.

Here’s the video recording of my session—I hope you find it useful!

‍

Transcript

Let me paint a familiar picture. You're starting a new project, and it's up to you to build a solid Infrastructure as Code configuration from scratch, using OpenTofu. As a developer who values good practices, you likely have specific requirements for managing your Infrastructure as Code. You need multiple environments—dev, integration, production—and a reliable promotion pipeline to ensure that code deployed to production was tested in previous stages.

You want to keep those environments isolated so nothing from development leaks into production. And, of course, you want a single codebase configured with the right variables for each environment. You'll need flexibility to enable environment-specific features when necessary, but you want to avoid duplicating code as much as possible. That's where the DRY principle comes in: Don't Repeat Yourself. Keep your code clean and maintainable.

I've been in your shoes, wrestling with keeping infrastructure clean and scalable across different environments. I'm part of the OpenTofu core team, a software engineer at env0, and I'm here today to share three and a half recipes to enhance your OpenTofu configuration. We don't have much time, so let's get cooking.

Recipe 1: OpenTofu Workspaces

Our first recipe uses the inherent workspaces feature of OpenTofu. OpenTofu supports the workspace command to create new workspaces and select existing ones. This approach allows for a single instance of your configuration, ensuring no duplication. It also provides a straightforward method to manage environment-specific features and configurations using workspace interpolation.

However, the main issue with this method is that all different environment states are stored in a single backend. This limits isolation between state files and can pose challenges when managing separate cloud accounts for each environment. You cannot run them with distinct credentials corresponding to the relevant cloud accounts. In my experience, I rate this recipe 5 out of 10. It's useful for supporting multiple development environments or ephemeral environments, but for managing distinct environments like development and production, it tends to fall flat.

Let's explore what else we have in stock with our second recipe. The instructions for this recipe are very simple. You just need to go through the right directory of the environment and run tofu plan and tofu apply.

However, when we look at the ingredients, we notice that there are duplicated configurations for each environment. Both the development and production environments are utilizing a shared module—the networking module—but each one receives different parameters. Each environment can also include its own auto.tfvars file, making it really easy to set different configurations for each environment.

While this results in a clear solution with effective modularization, it requires strict discipline. It's tempting to add resources directly to the duplicated configuration, risking configuration drift as code diverges between environments. As the number of environments grows, managing these duplicates becomes increasingly harder. I score this recipe 7 out of 10. It's suitable for small projects requiring extensive customization but not recommended for large projects with many contributors. It's worth mentioning tools like Terragrunt that can simplify this approach.

The next recipe involves blending the backend configuration into the cooking instructions. So first we start by running the command tofu init to prepare the kitchen and initialize our environment with the right backend. Once everything is set, we run tofu apply to deploy the infrastructure.

The ingredients for this recipe are quite similar to those of the workspace recipe, featuring a single step of configuration. However, we faced a challenge when attempting to use different variables for each environment. To achieve this, we need to pass an additional parameter to the tofu apply command. If we accidentally mix up the wrong backend configuration with an incorrect set of variables, it could lead to a hot mess in our deployed configuration.

Setting aside the risks of misconfiguration, this recipe allows us to isolate our state successfully while keeping a single instance without duplicates. So I give it an 8 out of 10, and I recommend it for a project that runs OpenTofu as part of automation, maybe as part of a CI/CD pipeline.

If your team runs OpenTofu locally, this can be a recipe for disaster.

The final recipe comes from the OpenTofu team. Thanks to the early evaluation feature in OpenTofu 1.8, the backend block now accepts variables and locals. Instead of injecting backend configuration directly into the tofu command, now we can run tofu init and tofu apply, adding the appropriate variable indicating your environment. This change reduces the risk of misconfiguration and makes feature management easier. After waiting for this feature, I confidently give it a perfect 10 out of 10.

Summary

Let's recap the scores:

• OpenTofu Workspaces: 5/10
• Directories with Shared Modules: 7/10
• Injecting Backend Configuration: 8/10
• Backend Configuration with Variables: 10/10

With these recipes, each with its strengths, experiment and find what suits your needs. Remember, you can adjust or mix and match flavors from different recipes. With the right approach, your infrastructure can be a masterpiece. Choose your flavor and get cooking with OpenTofu. Thank you.

Q&A Session

Q: So this was great. I do have a couple questions about having the backend config the variables. So let's say, hypothetically, I'm on a platform team, and we do backend injection. Kind of what you demonstrated, which you gave it, I think an eight out of ten. For variables, one of the things that we run into is that teams, we inject it for them. But from a local perspective they have no clue where their backend lives. And that's something that we actually handle for them. Could Vars be potentially a way to mitigate that if they're testing things locally?

A: I think so. I'm also like before we had this feature, I was also injecting the backend configuration into the command line. And I like I think it's when I used to look at like I as part of this doc, I went and I looked at our configurations and like, you really don't see a backend block. You have like the backend block, but it's empty and you don't understand what happens behind the scenes. And as you say, you don't really understand what's going on there. So yeah, I definitely agree with you that if we're using variables, we have the backend block. You have all the logic, like it's not a hidden logic that just the one who knows how to run OpenTofu knows exactly how it works. You see it, it's there. New developers that start working on your projects. They can see it and understand exactly how it works. So yeah, I think it's much more visual than previous approaches.

Q: I am curious what your opinion is on using remote workspaces similar to using, like, TerraKube or Terraform Cloud. You didn't mention anything like that. You're using s3's remote state, endpoint. But I'm curious if you have any experience using other tools and what your opinion is.

A: Yeah. So I work in env0 and we also have a feature of remote backend. And I like it's obviously a best practice to use a remote backend and not run the commands locally because you want your backend stored somewhere else. You want it to be, when you run, you want some kind of locking mechanism to ensure that problems won't happen.

And I really think that, like personally, as long as your backend is not stored locally and then you're all set if you want to use any other feature, if it makes your life easier, do that. But like, it's really up to you.

‍