Washington Center 2015: The ERAM Software Failure That Ground-Stopped the East Coast
A 2015 software upgrade at Washington Center retained controller-display data in memory instead of deleting it, eventually exhausting resources and disrupting East Coast air traffic.
A new convenience feature became a national air-traffic bottleneck
On 15 August 2015 the Washington Air Route Traffic Control Center in Leesburg, Virginia suffered a major automation failure that forced the FAA to reduce traffic through a large portion of East Coast airspace. Hundreds of flights were delayed or cancelled. The FAA later traced the incident to a recently installed ERAM software feature that let individual controllers create customized windows containing frequently referenced information.[1] The feature was operationally modest, but its memory behavior was not.
Small features can sit inside enormous blast radii
A user-interface enhancement may appear low risk, yet if it runs inside shared safety-critical infrastructure, its memory and lifecycle behavior can affect the whole service.
Deleted information was not actually removed from system memory
The FAA explained that when controllers changed or deleted their customized data, the information should have been fully removed. Instead, those changes remained in memory until the storage limit was filled.[1] Processing resources needed by the overall ERAM system were then consumed. This is conceptually similar to a memory leak: objects become unreachable from the user’s perspective while still occupying resources inside the system.
The interface said “deleted” while the runtime state said “retained”
That mismatch is especially dangerous in long-running systems because each harmless user action can accumulate invisible technical debt until a threshold is crossed.
Once shared resources filled, the automation platform could not sustain normal service
The Washington Center controls high-altitude routes across a large and heavily traveled region. When its flight-processing automation degraded, traffic had to be slowed, rerouted, or held because surrounding facilities could not safely absorb normal volumes. Initial reporting described widespread delays and cancellations while investigators were still determining the cause.[2] The outage illustrates how centralized infrastructure can amplify a local defect into network-wide disruption.
Backup procedures preserved safety but not normal throughput
Controllers used backup systems and procedures while FAA reduced arrival and departure rates. The agency reported that the region’s major airports still handled a substantial fraction of scheduled traffic, but well below normal capacity.[1] That outcome is exactly what degraded-mode design should prioritize: protect separation and situational awareness first, accept delay second.
A safe outage can still be economically severe
Resilience should be judged on multiple axes. The system prevented a safety catastrophe, yet the service disruption still propagated across airlines, airports, crews, and passengers.
The quickest mitigation was to disable the new function
FAA temporarily suspended the controller-customization feature, eliminating the immediate trigger while Lockheed Martin and the agency worked on a permanent fix.[1] This is an important operational pattern: when a new feature causes instability, rollback or feature disablement is often safer than diagnosing under live load. Critical infrastructure should therefore preserve the ability to deactivate optional functions without disabling the platform.
Testing failed to reproduce a cumulative production workload
The FAA said the contractor was examining why the problem had not been identified during testing.[1] The likely lesson is familiar to software teams: short functional tests can verify that a feature works without revealing that repeated use accumulates state. Long-duration soak testing, realistic user behavior, and resource telemetry are required to find defects whose effect grows slowly over hours or days.
State accumulation belongs in the test plan
Any feature that creates, modifies, or deletes persistent or cached data should be tested for resource stability after thousands of repetitions, not only for correctness after one action.
DOT later treated the outage as evidence that ERAM resilience needed more work
A 2018 DOT Inspector General audit identified the August 2015 failure as one of two serious ERAM incidents that significantly disrupted the National Airspace System.[3] The audit credited FAA with corrective actions but also found gaps in contingency testing and backup planning. Contemporary reporting documented nearly one thousand delays and cancellations tied to the event.[4] The incident therefore became part of a broader reliability discussion rather than an isolated software bug.
The incident also illustrates why observability must include resource ownership. A system can report total free memory while hiding which feature is consuming it and whether deleted user state is actually reclaimed. In critical services, per-feature quotas and leak detection can turn a platform-wide exhaustion event into a contained local failure. That is especially important when optional interface functions share the same process and memory pool as core routing logic.
Why Washington Center 2015 remains a useful software-failure case
This outage is valuable because the defect was mundane: retained data filled memory. Yet the environment was extraordinary: the software coordinated real aircraft across one of the world’s busiest airspaces. A simple lifecycle bug acquired national consequences because it lived inside shared critical infrastructure.[3][5]
The engineering lesson is to match testing depth to blast radius, not feature size. Optional interface features should have hard resource quotas, observability, and kill switches. Long-running services should continuously measure memory growth and rehearse fallback modes. Critical systems do not only fail through exotic algorithms; sometimes they fail because something that looked deleted was still there.
Works Cited
- 01FAA Statement — Automation Problems at Washington Center aviationpros.com
- 02
- 03
- 04
- 05Washington Technology — Software Problem Temporarily Grounds Lockheed Air Traffic Control System washingtontechnology.com
CodeHistory is a living archive. Citations document the evidence used for this edition; later evidence may refine the account.