Terraform Fundamentals

Terraform is an infrastructure-as-code tool: you declare desired resources in HCL, and Terraform computes the difference between desired config, recorded state and real infrastructure, then applies the needed API calls through providers.

terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 5.0" }
  }
}

resource "aws_s3_bucket" "logs" {
  bucket = "acme-orders-logs"
}

• Core loop: init (download providers/modules, configure backend), plan, apply, destroy.
• Resources form a dependency graph from references; independent ones are created in parallel.

State is a JSON mapping from resource addresses in config to real object IDs plus their last known attributes. Terraform needs it to know what it manages, to compute diffs efficiently, and to track dependencies and metadata.

• Without state Terraform could not know which existing bucket corresponds to aws_s3_bucket.logs.
• State can contain sensitive values in plaintext.
• Never edit it by hand; use terraform state subcommands.

A remote backend shares state across the team and CI, supports encryption and versioning, and locks state during writes so concurrent applies cannot corrupt it.

terraform {
  backend "s3" {
    bucket         = "acme-tf-state"
    key            = "prod/orders/terraform.tfstate"
    region         = "eu-west-1"
    encrypt        = true
    use_lockfile   = true   # S3-native locking, Terraform 1.10+
  }
}

• Before 1.10, locking used a DynamoDB table (dynamodb_table = "tf-locks"), which is now deprecated in favour of S3 native lockfiles.
• Enable bucket versioning, block public access, restrict IAM.
• Split state by environment/component to limit blast radius.

plan refreshes real state, compares it with config and prints the proposed create/update/destroy actions without changing anything. In CI, save the plan and apply exactly that file so what was reviewed is what runs.

terraform init -input=false
terraform plan -out=tfplan -input=false
terraform apply -input=false tfplan

• Symbols: + create, ~ update in place, -/+ replace (destroy then create), - destroy.
• A saved plan can contain secrets; store it as a protected artifact.

Input variables parameterize a module, locals are internal computed names, and outputs expose values to the caller or the CLI.

variable "env" {
  type = string
  validation {
    condition     = contains(["dev", "stage", "prod"], var.env)
    error_message = "env must be dev, stage or prod."
  }
}

variable "db_password" {
  type      = string
  sensitive = true
}

locals { name = "orders-${var.env}" }

output "db_endpoint" { value = aws_db_instance.main.endpoint }

• Precedence (low to high): defaults, TF_VAR_name environment variables, terraform.tfvars, *.auto.tfvars, then -var and -var-file in command-line order.
• sensitive = true hides values in CLI output but not in state.

A module is a directory of Terraform files called with inputs and returning outputs. Every config is a root module that may call child modules.

module "orders_db" {
  source  = "terraform-aws-modules/rds/aws"
  version = "~> 6.0"
  identifier = "orders-${var.env}"
}

• Good modules: small, single purpose, typed variables with descriptions, minimal required inputs, useful outputs, version pinned.
• Avoid deep nesting and "god" modules with dozens of toggles.
• Version modules via Git tags or registry versions; pin in callers.

count creates indexed instances; for_each creates instances keyed by map keys or set elements. Removing an item in the middle of a count list shifts all later indexes and causes destroy/recreate; for_each keys are stable.

resource "aws_sqs_queue" "q" {
  for_each = toset(["orders", "payments", "emails"])
  name     = "${each.key}-${var.env}"
}

• for_each keys must be known at plan time (not derived from unknown resource attributes).
• Use count = var.enabled ? 1 : 0 for simple conditional resources.

Drift is real infrastructure differing from state/config because of manual console edits or other tools. terraform plan (which refreshes) shows it; plan -refresh-only shows drift alone without proposing config changes.

• Decide per change: revert it (apply), or adopt it by updating config (and apply -refresh-only to accept into state).
• Detect continuously with scheduled plans in CI that alert on non-empty diffs (-detailed-exitcode: 0 no change, 2 changes).
• Restrict console write access to reduce drift; use lifecycle { ignore_changes } only for fields legitimately managed elsewhere.

CLI workspaces give the same configuration and backend multiple named state files (terraform.workspace). They are suited to short-lived copies, not strongly isolated environments.

terraform workspace new staging
terraform workspace select staging

• Weak isolation: same backend credentials and code; easy to apply to prod by selecting the wrong workspace.
• For dev/stage/prod, prefer separate directories or root modules with separate state, backends and credentials, reusing shared modules; Terragrunt or per-env tfvars help.

Import attaches an existing real resource to a resource address in state without creating it. You must still write the matching configuration.

import {
  to = aws_s3_bucket.logs
  id = "acme-orders-logs"
}

resource "aws_s3_bucket" "logs" {
  bucket = "acme-orders-logs"
}

• Since Terraform 1.5, declarative import blocks go through plan and apply, and terraform plan -generate-config-out=gen.tf can draft config.
• The legacy CLI terraform import addr id changes state immediately without a plan review.
• After import, run plan until it shows no diff.

Any attribute a provider returns, including generated passwords and keys, is stored in plaintext in state, regardless of sensitive. Anyone with state read access can read them.

• Encrypt the backend (SSE-KMS), restrict access with IAM, enable versioning and audit logging.
• Avoid creating secrets in Terraform when possible: reference existing ones from Secrets Manager/Vault with data sources, or let the service generate them (RDS manage_master_user_password = true).
• Terraform 1.10+ ephemeral values and 1.11 write-only attributes (password_wo) keep values out of state where providers support them.
• Never put secrets in .tfvars committed to Git; use TF_VAR_* from the CI secret store.

lifecycle overrides default create/update/delete behaviour per resource.

resource "aws_db_instance" "main" {
  # ...
  lifecycle {
    prevent_destroy = true
    ignore_changes  = [engine_version]
  }
}

• prevent_destroy: plan fails if the resource would be destroyed; guard for databases and state buckets. It does not stop removal of the block itself.
• create_before_destroy: build the replacement first to avoid downtime (names/unique constraints must not collide).
• ignore_changes: skip drift on attributes managed elsewhere (autoscaling desired count, tags set by tooling).

With a backend that supports locking, the second run fails with "Error acquiring the state lock" and shows the lock holder ID, user and time. Without locking, both read the same state and the later write can overwrite the first, orphaning real resources.

• Never use -lock=false in normal operation.
• If a crashed run left a lock, confirm no process is running, then terraform force-unlock <LOCK_ID>.
• Serialize applies in CI (one pipeline per state, concurrency groups).

concurrency:
  group: tf-prod-orders
  cancel-in-progress: false

Stop and find the attribute marked # forces replacement; do not apply. Replacement of a stateful resource means data loss or downtime.

• Common causes: changing identifier, engine, subnet group, or a rename of the resource block in config.
• If it is a rename, use a moved block so state follows the new address.
• If an immutable attribute must change, plan a managed migration (snapshot, restore, cutover) rather than in-place replacement.
• Protect with prevent_destroy = true and deletion_protection = true.

moved {
  from = aws_db_instance.db
  to   = aws_db_instance.main
}

Run a plan to see the drift, then choose: revert it (apply as is) or accept it into code.

terraform plan -refresh-only
# if the manual change is the truth, update HCL to match, then:
terraform apply -refresh-only
terraform plan   # expect "No changes"

• Reverting could break whatever the manual change fixed (maybe an incident hotfix); ask first.
• Add scheduled drift detection and tighten IAM so console edits are rare.
• Inline ingress blocks vs separate aws_security_group_rule resources should not be mixed on the same group, as they fight.

Move resources between states rather than destroying them.

terraform state pull > backup.tfstate           # always back up first
terraform state mv -state-out=../network/terraform.tfstate \
  aws_vpc.main aws_vpc.main

• Alternative: removed block with lifecycle { destroy = false } in the old config plus import blocks in the new config (declarative, reviewable).
• Verify both states with plan showing no changes before any other change.
• Use terraform_remote_state or data sources so consumers find the moved resources.

removed {
  from = aws_vpc.main
  lifecycle { destroy = false }
}

Either read the producer's state with terraform_remote_state, or, preferably, look resources up with provider data sources, which decouples consumers from the producer's state layout and permissions.

data "aws_vpc" "main" {
  tags = { Name = "prod-main" }
}

data "terraform_remote_state" "network" {
  backend = "s3"
  config  = { bucket = "acme-tf-state", key = "prod/network/terraform.tfstate", region = "eu-west-1" }
}

• terraform_remote_state requires read access to the whole state, including its secrets.
• Alternatively publish IDs to SSM Parameter Store as an explicit interface.

Pin versions and upgrade deliberately: commit .terraform.lock.hcl, bump the constraint in a dedicated PR, read the provider upgrade guide for breaking changes, and review the plan in a non-prod environment first.

required_providers {
  aws = { source = "hashicorp/aws", version = "~> 5.60" }
}

terraform init -upgrade       # re-resolves within constraints and updates the lock file

• The lock file records hashes so every machine uses identical providers.
• Generate hashes for all CI platforms: terraform providers lock -platform=linux_amd64 -platform=darwin_arm64.

Separate root modules per environment (own backend key, own credentials) calling shared versioned modules.

infra/
  modules/
    service/        # reusable ECS/EKS service module
    database/
  envs/
    dev/   main.tf backend.tf dev.tfvars
    stage/ main.tf backend.tf stage.tfvars
    prod/  main.tf backend.tf prod.tfvars

• Promote changes dev -> stage -> prod through the same PR flow; prod applies need approval.
• Separate AWS accounts per environment is the strongest isolation.
• Keep env differences in tfvars (sizes, counts), not forked code.

Terraform records each resource in state as it succeeds, so state reflects completed work and the rest remains pending. Fix the cause (quota, permission, name collision) and re-run plan and apply; it continues.

• Resources that were created but errored during creation may be marked tainted (terraform untaint or let it replace).
• If a resource exists in the cloud but not in state (crash before write), import it.
• If the run was killed, check for a stuck lock and a errored.tfstate file locally, which can be pushed with terraform state push after review.

Layer checks from cheap to expensive.

terraform fmt -check -recursive
terraform validate
tflint
checkov -d .           # or trivy config / tfsec for policy and security
terraform plan -out=tfplan

• Native terraform test (1.6+) runs .tftest.hcl with command = plan or real apply in a sandbox.
• Terratest for integration tests, with ephemeral accounts.
• Policy as code (OPA/Conftest, Sentinel) on the JSON plan: terraform show -json tfplan.
• Post a plan comment to the PR; require approval before apply on main.

Forget it in state without destroying it. Modern way is a removed block; legacy way is terraform state rm.

removed {
  from = aws_s3_bucket.legacy
  lifecycle { destroy = false }
}

terraform state rm aws_s3_bucket.legacy

• Remove the resource block from config as well, otherwise the next plan wants to create it again.
• Document and hand ownership to another tool/state if it will continue to be managed.

Combine access control, encryption and avoiding secret creation inside Terraform.

resource "aws_db_instance" "main" {
  identifier                  = "orders-${var.env}"
  manage_master_user_password = true   # RDS stores it in Secrets Manager
  # ...
}

• Bucket: SSE-KMS with a dedicated key, versioning, block public access, deny non-TLS, MFA delete optionally.
• IAM: only the CI role and a small break-glass group can read state; developers use plan-only roles where possible.
• Applications fetch secrets at runtime from Secrets Manager, never via Terraform outputs.
• Audit with CloudTrail data events on the state bucket.

Use create_before_destroy and avoid fixed names that collide; use name_prefix so old and new coexist during the swap.

resource "aws_lb_target_group" "app" {
  name_prefix = "orders-"
  port        = 8080
  protocol    = "HTTP"
  vpc_id      = var.vpc_id

  lifecycle { create_before_destroy = true }
}

• Dependent resources (listeners) get updated to the new one before the old is destroyed because of the dependency graph.
• The attribute must be propagated: create_before_destroy applies transitively to dependencies of that resource.