Terraform

TL;DR

• Terraform is a declarative Infrastructure-as-Code tool: HCL describes desired state, and Terraform diffs that against a state file to compute a plan of changes to apply against real infrastructure.
• Writing HCL is the easy part — state management discipline (backend, isolation strategy, locking) is what determines whether a setup scales to hundreds of resources and dozens of engineers or becomes unmanageable.
• plan diffs config against the state file, not live infrastructure directly — a stale or corrupted state file produces a wrong plan even when the config itself is correct.
• for_each should be the default over count for anything created from a list or map; count is fine only for purely conditional creation.
• State must live in a remote backend with encryption and locking — local state, or state committed to Git, routinely leaks plaintext secrets.

Core Concepts

Providers, resources, and aliases

A provider configures access to an API (AWS, GCP, a SaaS platform); resources declare what should exist through that provider. Multi-region or cross-account designs use provider aliases to target more than one provider configuration from a single root module:

provider "aws" {
  region = "us-east-1"
  alias  = "primary"
}
provider "aws" {
  region = "eu-west-1"
  alias  = "dr"
}

resource "aws_s3_bucket" "primary_logs" {
  provider = aws.primary
  bucket   = "logs-us-east-1"
}
resource "aws_s3_bucket" "dr_logs" {
  provider = aws.dr
  bucket   = "logs-eu-west-1"
}

Modules

Modules encapsulate reusable patterns — inputs, outputs, resources. The rule that matters most for reusability: never declare a provider configuration inside a module. A hardcoded provider locks the module to one account/region/credential set; passing providers explicitly from the root (providers = { aws = aws.security }) is what lets the same module deploy into different accounts or regions without modification.

for_each vs. count

This distinction is checked in every senior Terraform review. count creates indexed resources — removing an item from the middle of the source list shifts every subsequent index, and Terraform reads that as those resources having changed identity, triggering unnecessary (and for something like an IAM user, potentially disruptive) destroy/recreate cycles:

# ❌ count — fragile, index-based identity
resource "aws_iam_user" "users" {
  count = length(var.user_list)
  name  = var.user_list[count.index]
}
# Removing index 0 shifts every subsequent user's index — Terraform
# tries to rename/recreate users 1, 2, 3...

# ✅ for_each — stable, key-based identity
resource "aws_iam_user" "users" {
  for_each = toset(var.user_list)
  name     = each.value
}
# Removing one user from the list only affects that one resource

count still has a legitimate, narrow use: purely conditional creation (count = var.enabled ? 1 : 0). For anything creating multiple resources from a list or map, for_each should be the default.

State

State is the file tracking the mapping between configuration and real infrastructure — resource IDs, attributes, dependencies. This file must never be committed to Git: it contains sensitive values (database passwords, generated secrets) in plaintext by default, and state files with real credentials have been accidentally pushed to public repos as a direct result. The structural fix isn’t “remember not to commit it” — it’s using a remote backend in the first place, which removes local state files from the picture entirely.

terraform {
  backend "s3" {
    bucket         = "mycompany-terraform-state"
    key            = "platform/networking/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-state-lock"
  }
}

Remote backends and locking

Remote backends (S3+DynamoDB, GCS, Terraform Cloud) give three things local state can’t: team collaboration, encryption at rest, and state locking — DynamoDB (for S3 backends) prevents two concurrent applies from corrupting the same state simultaneously.

Workspaces vs. directory-based isolation

Workspaces look purpose-built for environment separation (terraform workspace select prod) but are widely considered an anti-pattern for exactly that use case: workspaces share the same backend configuration, often the same state bucket, differing only by workspace name. A terraform destroy run against the wrong workspace is a real, catastrophic risk that’s structurally easy to trigger by mistake. Directory-based isolation — separate directories with genuinely separate backend keys per environment or component — makes that mistake much harder to make, because there’s no ambiguous “which workspace am I in right now” state to get wrong:

infrastructure/
├── networking/   → own state, own apply cycle
├── database/     → own state, own apply cycle
├── compute/      → own state, own apply cycle

This same directory-based split is also the direct fix for the “50MB state file, 8-minute plans, constant lock conflicts” problem: splitting a monolithic state into per-component state files (referencing each other’s outputs via terraform_remote_state) shrinks each individual plan’s scope proportionally and removes lock contention between unrelated components that no longer share a single state file and lock.

Terraform vs Ansible vs CloudFormation

Command / Configuration Reference

terraform init                     # download providers, configure backend
terraform validate                 # syntax/config validation, no API calls
terraform plan -out=tfplan         # compute diff vs. state, save plan
terraform apply tfplan             # apply a saved plan
terraform state list               # list resources tracked in state
terraform state mv <src> <dst>     # rename/move a resource in state without destroying it
terraform import <addr> <id>       # bring an existing resource under management
terraform workspace list           # list workspaces (avoid for env separation — see below)

# Remote state reference across components
data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "mycompany-terraform-state"
    key    = "platform/networking/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_instance" "app" {
  subnet_id = data.terraform_remote_state.networking.outputs.subnet_id
}

Common Pitfalls

• Pitfall: A state file with real credentials is accidentally committed to a public Git repository. Why: Local state (or state-in-Git) was used instead of a remote backend, and state stores sensitive values in plaintext by default. Fix: Use a remote backend with encryption at rest from the start of the project.
• Pitfall: count-based resource creation triggers unexpected, disruptive destroy/recreate cycles. Why: Removing an item from the middle of the source list shifts every subsequent index, changing resource identity. Fix: Use for_each keyed by a stable identifier for anything created from a list or map.
• Pitfall: A module can’t be reused across accounts without modification. Why: A provider configuration is hardcoded inside the module. Fix: Never declare providers inside modules; pass them explicitly from the root via a providers = {} block.
• Pitfall: terraform destroy is run against the wrong environment. Why: Workspaces share one backend and state bucket, differing only by name, making it easy to be in the wrong workspace context. Fix: Use directory-based isolation with distinct backend keys per environment instead of workspaces.
• Pitfall: A growing monolithic state file turns every plan into an 8-minute wait with frequent lock conflicts. Why: Every engineer’s plan/apply refreshes and locks the entire state file, regardless of which resources they’re touching. Fix: Split state by component/directory so plans and locks scope to only the relevant resources.

Interview Questions

• Explain exactly why for_each is safer than count for a list of IAM users, mechanically. — Tests whether the index-shift-vs-stable-key mechanism is genuinely understood.
• A 50MB state file takes 8 minutes to plan with constant lock conflicts. Walk through the fix. — Tests whether state-splitting via directory-based isolation is the reasoned, correct answer.
• Why do experienced Terraform practitioners avoid workspaces for environment separation? — Tests whether the shared-backend risk is understood, not just repeated as a rule of thumb.
• Why should provider configuration never live inside a reusable module? — Tests understanding of module reusability constraints.
• What’s the structural fix for state files leaking secrets, versus a one-time remediation? — Tests whether the candidate reaches for remote backends as a systemic fix, not just “don’t commit it.”