Architecture Drift Is a Systems Problem, Not a Documentation Problem

Three descriptions of the same system

Every cloud-native system is described in at least three places at once. The first is the intended architecture: the design that was agreed on, the boundaries that were drawn, the reasons behind them. It usually lives in a design review, a diagram, or the memory of the engineers who were in the room. The second is the infrastructure definition: the Terraform modules, Helm charts, Kubernetes manifests, and pipeline configuration that declare what should exist. The third is production reality: the resources actually running, the services actually calling each other, the permissions actually granted, and the traffic actually flowing.

These three descriptions are not versions of one truth. They are three separate truths, maintained by different tools, on different cadences, by different people. Nothing in the default toolchain forces them to agree. The intended architecture changes when a team decides to split a service. The infrastructure definition changes when someone opens a pull request. Production changes when a node is replaced, an autoscaler adds replicas, or an engineer adds a security group rule to unblock an incident at 2am. Each change is locally reasonable. The gap between them is drift.

Drift is not a bug in your process. It is the equilibrium state of a system that stores the same fact in three places without a mechanism to reconcile them.

Why documentation discipline cannot fix it

The common response to drift is to ask engineers to be more disciplined. Update the diagram when you change the system. Keep the runbook current. Treat documentation as part of the definition of done.

This fails for a structural reason. The cost of updating the documentation is paid by the individual engineer who makes the change, and it is paid immediately. The benefit is collective and delayed: it accrues to whoever reads the diagram six months from now, a person the author may never meet. Under any deadline pressure, the immediate individual cost wins. This is not a character flaw. It is how incentives behave when the person paying and the person benefiting are different.

You can see the same dynamic in code comments. Teams do not stay documented because engineers are lazy. They stay documented because tooling makes it cheaper to write the truth once than to maintain a copy. Generated API documentation works because it is derived from the thing it describes. Architecture documentation is not derived from anything. It is a parallel artefact, and parallel artefacts decay.

Drift is also a measurement problem

You cannot fix what you cannot see, and most teams materially underestimate their drift because they only look for it in two of the three places. A terraform plan compares the intended definition with production for resources Terraform manages. It is silent about everything else: resources created outside the state file, relationships between services, runtime configuration injected by a controller. And it says nothing at all about the first truth, the intended architecture, because no tool holds it.

Runtime reconciliation loops close part of this gap. Kubernetes controllers, for example, continuously compare desired state to observed state and act on the difference. That is a real and important mechanism. It is also narrow. A controller reconciles one resource kind against one declared specification. It does not tell you that a dependency described in a design review no longer exists, or that a service now reaches a data store it was never meant to touch.

Here is the trap that makes this harder than it looks. Teams often configure reconciliation to ignore exactly the fields where drift appears.

resource "aws_security_group" "api" {
  name = "api-sg"

  ingress {
    from_port = 443
    to_port   = 443
    protocol  = "tcp"
  }

  lifecycle {
    ignore_changes = [ingress]
  }
}

An ignore_changes block is often justified. Managed rules, autoscaling side effects, and third-party controllers all mutate resources in ways a plan cannot express. But each exception is a place where the definition stops being the source of truth and production quietly takes over. Enough of them, and the plan is green while the architecture is wrong.

Treating architecture as a maintained system

The alternative is to stop treating architecture as an artefact and start treating it as a maintained model: a single representation that is continuously compared against the infrastructure definitions and against production, with differences surfaced rather than assumed away.

The model does not have to be complete on day one. It has to be the thing everyone reads and the thing that gets updated, so that there is only one place the truth can live. That is a systems change, not a documentation change. The goal is not a better diagram. It is to make drift a detected, attributable event instead of a silent one, so that the gap between what you designed and what you run is a known quantity rather than a discovery during an incident or a security review.