Terraform remote state: backends, locking and migrating local state

A local terraform.tfstate works for one person on one laptop. As soon as a second person or a CI pipeline runs Terraform, you need the state in one shared place, protected against two runs writing it at the same time. That is what a backend with state locking provides.

What a backend does

The backend decides where state is stored. The default is local. Terraform ships a fixed set of built-in backends, including s3, azurerm, gcs, pg (PostgreSQL), consul, http and kubernetes; you cannot add backends as plugins. HCP Terraform is another option for remote state.

If the backend supports locking, Terraform locks the state automatically for every operation that could write it, and refuses to continue if it cannot get the lock. Check your backend’s documentation to confirm it supports locking.

Example: S3 with S3-native locking

terraform {
  backend "s3" {
    bucket       = "example-tfstate"
    key          = "network/prod/terraform.tfstate"
    region       = "eu-central-1"
    encrypt      = true
    use_lockfile = true
  }
}

• key is the object path of this project’s state. Give every project and environment its own key.
• encrypt enables server-side encryption of the state and lock files.
• use_lockfile turns on locking with a lock object stored next to the state file (in recent versions; it defaults to false).
• The older dynamodb_table argument locks through a DynamoDB table instead. The S3 backend docs mark DynamoDB-based locking as deprecated; both can be set while you migrate.

Turn on versioning for the state bucket, restrict access to the people and pipelines that run Terraform, and keep the bucket itself outside the configuration that uses it. Per the docs, the minimum permissions are s3:ListBucket on the bucket and s3:GetObject and s3:PutObject on the state object, plus s3:GetObject, s3:PutObject and s3:DeleteObject on the .tflock object when use_lockfile is on.

Backend blocks cannot use variables

A backend block cannot refer to variables, locals or data sources, and a configuration can have only one backend block. For values that differ per environment or should not be in Git, use partial configuration: leave the arguments out of the block and pass them to init.

# backend.tf
terraform {
  backend "s3" {}
}

# prod.s3.tfbackend
bucket       = "example-tfstate"
key          = "network/prod/terraform.tfstate"
region       = "eu-central-1"
use_lockfile = true

terraform init -backend-config=prod.s3.tfbackend

Migrate existing local state

Warning: state is the only link between your code and the real resources. Losing it means Terraform no longer knows what it manages. Back it up before every migration and check that nobody else is running Terraform on this project.

1. Make sure the working copy is clean: terraform plan should show no changes.
2. Back up the state: cp terraform.tfstate terraform.tfstate.backup-$(date +%F).
3. Add the backend block (or the empty block plus a backend config file).
4. Run terraform init -migrate-state and answer yes when Terraform asks to copy the existing state to the new backend.
5. Run terraform plan again. It must still show no changes. If it wants to create everything, the state did not arrive; stop and restore from the backup.
6. Once it works, delete the local state file from the working copy so nobody uses it by mistake.

terraform init -reconfigure is different: it switches to the new backend settings without copying state. Use it only when the new backend already has the right state.

Working with remote state day to day

terraform state list
terraform plan -lock-timeout=5m
terraform apply -lock-timeout=5m

-lock-timeout makes Terraform retry for a while if someone else holds the lock, which is useful in CI where runs can overlap. Avoid -lock=false: it removes the protection the backend gives you.

OpenTofu

OpenTofu uses the same backend blocks and the same init -migrate-state flow with the tofu command. It also supports encrypting state and plan files at rest, locally and in backends, configured in an encryption block inside terraform { }. Check the OpenTofu backend documentation for your version, since backend options can differ between the two tools.

Official documentation

• Backend block configuration overview
• Backend type: s3
• OpenTofu state and plan encryption