AWS Kinesis 2020: The Scaling Event That Took Down Cognito and Other Services
AWS’s November 2020 Kinesis outage began with a modest capacity addition that pushed front-end servers over operating-system thread limits and cascaded into dependent AWS services.
A small capacity addition triggered a large control-plane failure
On November 25, 2020, Amazon Kinesis Data Streams began adding a relatively small amount of capacity in US-EAST-1. AWS later said the addition was the trigger but not the root cause.[1] Kinesis front-end servers maintain operating-system threads for communication with many back-end clusters. As the fleet changed, the number of threads required on front-end hosts crossed an operating-system limit. New thread creation failed, existing hosts became unhealthy, and the service entered a recovery problem in which the mechanism used for ordinary scale was now contributing to resource exhaustion.
The limit was local but the architecture made it systemic
Each host encountered a familiar operating-system resource ceiling, yet the same fleet design repeated that ceiling across many hosts, turning one capacity dimension into a service-wide constraint.
Kinesis front ends could not rediscover back-end clusters normally
AWS explained that the exhausted servers had difficulty communicating with the membership system used to discover back-end clusters.[1] Restarting front-end processes did not immediately help because they needed to rebuild large amounts of membership information while the fleet was impaired. Recovery therefore required reducing the number of threads each server needed and gradually restoring the front-end fleet. This is a recurring distributed-systems lesson: when metadata about the cluster is necessary for a node to rejoin, large-scale restart can be harder than steady-state operation.
Recovery paths need lower resource requirements than normal operation
A system that needs full-scale metadata, connections or dependencies before a node can become healthy may struggle to bootstrap after widespread failure.
Other AWS services depended on Kinesis in ways customers could not always see
Cognito, CloudWatch, EventBridge and other AWS services experienced impairment because they used Kinesis internally.[1] Some customers who never directly provisioned a Kinesis stream still encountered failures through those services. This is the hidden-dependency problem of cloud platforms: managed services are built from other managed services, so one foundational impairment can appear as authentication errors, delayed metrics, event-processing failures or console problems elsewhere.
Dependency maps must include provider-internal services
Customers can design around documented dependencies, but providers also need to reason about internal dependency fan-out when deciding where to place critical functions.
Cognito impact turned a streaming outage into an authentication outage
Amazon Cognito is used to authenticate and authorize users for applications.[2] During the Kinesis event, Cognito APIs in US-EAST-1 experienced elevated errors because of an internal Kinesis dependency. Authentication is especially sensitive because it sits at the entrance to otherwise healthy applications. When identity depends on a regional data-processing service, failure of that service can lock users out of workloads whose compute and storage remain available.
Foundational dependencies deserve stronger isolation
Identity, monitoring and control-plane functions should avoid sharing avoidable dependencies with ordinary workload paths because their failure makes diagnosis and recovery harder.
AWS changed both operating-system limits and architectural scaling behavior
AWS increased relevant resource limits and redesigned how Kinesis front ends obtain and maintain cluster membership so the number of operating-system threads would no longer grow in the same way with fleet scale.[1] The response attacked the failure class rather than only the observed threshold. Raising a limit buys time; changing the relationship between fleet size and per-host resources removes the scaling cliff. Good postmortems distinguish those two kinds of remediation.
Monitoring impairment made the outage harder for customers to understand
CloudWatch itself experienced problems during the Kinesis event, so customers could lose visibility while their workloads were already failing.[3] This is an operational anti-pattern seen in many cloud incidents. Monitoring must be resilient to the same dependency failures it is expected to expose. Providers increasingly design status, telemetry and support paths with separate failure domains because an outage without trustworthy observability is harder to contain than an outage with the same technical root cause.
Backoff and overload control are core distributed-systems safety mechanisms
AWS Builders Library guidance emphasizes retries with backoff, load shedding and controlling amplification during overload.[4] The Kinesis event showed why: when many unhealthy nodes restart or rediscover peers simultaneously, recovery traffic can become a new source of load. Systems need jitter, bounded concurrency and staged reintroduction so that repair does not recreate the resource exhaustion that caused the failure. Kinesis documentation also describes shard-based scaling, but internal metadata systems must scale alongside customer-visible capacity.[5]
Why AWS Kinesis 2020 belongs in software-failure history
The 2020 Kinesis outage belongs in software history because it shows how cloud scale can reveal resource relationships that remain invisible for years. A modest capacity addition crossed a per-host thread limit, which impaired membership discovery, which complicated restart, which propagated into services as varied as Cognito and CloudWatch. The lesson is not simply to raise operating-system limits. It is to inspect how per-node resources grow with global system size and to design recovery paths with less dependency and less coordination than normal operation. At hyperscale, local limits become architectural limits unless the design deliberately breaks that coupling.
Works Cited
- 01AWS — Summary of the Amazon Kinesis Event, November 25 2020 aws.amazon.com
- 02AWS Docs — Amazon Cognito docs.aws.amazon.com
- 03AWS Docs — Amazon CloudWatch docs.aws.amazon.com
- 04
- 05AWS Docs — Amazon Kinesis Data Streams docs.aws.amazon.com
CodeHistory is a living archive. Citations document the evidence used for this edition; later evidence may refine the account.