Infrastructure as Code Is Not a Tool Choice — It Is a Discipline

The Difference Between Using IaC Tools and Practicing IaC

Most engineering teams that say they do Infrastructure as Code have Terraform files in their repository. A smaller fraction actually practice IaC in the sense that matters: their infrastructure is fully defined in code, their code is the source of truth, and the configuration in their cloud provider could be recreated from that code without manual intervention.

The gap between these two positions is wide, and it is where most infrastructure fragility lives.

The Source of Truth Problem

IaC works as a discipline only if the code is the definitive source of truth for what infrastructure exists. The moment an engineer makes a change in the AWS console and does not update the Terraform state to match, the code and the infrastructure have diverged. This is called drift.

Drift is insidious because it is invisible. The Terraform files look correct. The infrastructure works. The divergence only becomes apparent when you run terraform apply and it tries to modify or destroy something that was changed manually — at which point you have a production incident in the making.

Preventing drift requires:

No manual changes in the cloud console. This is a hard rule, not a guideline. It requires team buy-in and, ideally, IAM policies that restrict the ability to make manual changes in production.
Regular drift detection. Run terraform plan in CI/CD on a schedule and alert when there is unexpected drift. Some teams use automated drift remediation; others prefer drift alerts with manual review.
State locking. Use remote state backends (S3 + DynamoDB for AWS, GCS for GCP) with state locking enabled. Concurrent state modifications without locking produce state corruption — a scenario that is painful to recover from.

Module Design: Reuse Without Rigidity

Terraform modules are the unit of reuse. A well-designed module abstracts a meaningful infrastructure component — a VPC configuration, an EKS cluster, a managed database, an application service with all its supporting resources — behind a clean variable interface.

The failure mode of module design is over-abstraction. A module that tries to encapsulate every possible configuration option becomes harder to use than writing the Terraform directly. The goal is to abstract the right things — the things that are consistent across uses — while leaving the right things as variables.

Module versioning is non-optional for any module shared across environments or teams. A module update that breaks production because staging and production were using different versions is a configuration management failure. Use semantic versioning and pin module versions explicitly. Test module updates in non-production before applying to production.

State Management in Multi-Environment Systems

A single Terraform state for all environments is a liability. Changes to staging affect the same state file as production. A failed apply mid-way through a production change can leave state partially updated, requiring careful manual intervention.

Use separate state files per environment. The cleanest approach is separate state backends per environment, with a CI/CD pipeline that applies changes to staging first and requires explicit approval before applying to production.

Workspace-based approaches (using terraform workspace) are simpler to set up but share backend infrastructure. They work for smaller teams but become harder to manage as the number of environments grows.

Testing Infrastructure Code

Infrastructure code is code. It should be tested.

Linting and formatting: terraform validate and terraform fmt are the minimum. Integrate them into CI/CD — failing to format code is a merge blocker, not an optional style preference.

Static analysis: tools like tfsec, checkov, and trivy scan Terraform code for security misconfigurations — public S3 buckets, unencrypted databases, overly permissive security groups. These checks belong in the PR pipeline, not as a post-hoc audit.

Plan review in CI/CD: every PR that changes infrastructure should produce a terraform plan output visible in the PR. Engineers reviewing the PR should review the plan, not just the code. What Terraform will actually do is the relevant artifact — not what the code says it intends to do.

End-to-end testing: for critical infrastructure modules (VPC configurations, Kubernetes clusters, database setups), tools like Terratest allow you to provision real infrastructure in a test environment, run assertions against it, and tear it down. This is expensive in time and cost, but provides the only real guarantee that the module works as intended.

The Documentation That Actually Gets Read

IaC is self-documenting in a specific sense: the code describes what infrastructure exists, what its configuration is, and how it is connected. This is valuable, but incomplete.

What the code does not document: why a particular configuration decision was made. Why is this security group rule allowing this port range? Why is this database sized the way it is? Why was this particular subnet design chosen?

Architecture Decision Records (ADRs) — short, structured documents that record significant decisions, the context that drove them, and the options that were considered — provide the why that code cannot. Store them in the same repository as the infrastructure code. Treat them as first-class documentation, updated when decisions change.

Infrastructure as Code is an engineering discipline with the same quality standards that apply to application code: review processes, testing, linting, version control, and clear ownership. Teams that treat it as a tooling adoption rather than a practice change end up with Terraform files that describe infrastructure that no longer matches what is running — which is, in practice, no better than having no IaC at all.