Nasdaq’s 2013 SIP Outage: The Software Failure That Halted U.S. Stock Trading
Nasdaq's 2013 SIP outage combined extreme reconnect traffic with a latent redundancy flaw, halting trading in Nasdaq-listed U.S. stocks for roughly three hours.
The SIP was a shared source of consolidated quote information
The UTP Securities Information Processor collected and disseminated consolidated quote data for Nasdaq-listed securities. On August 22, 2013, it stopped disseminating quote data, and Nasdaq coordinated a regulatory halt in Tape C securities.[2] Because participants rely on a common view of the market to trade fairly, losing consolidated quotations was not merely a data-vendor outage. Continuing normal trading risked information asymmetry.
Shared market data is critical infrastructure
The SIP sits between multiple exchanges and the public price-discovery process. A failure in that layer can force healthy trading venues to stop because the common information substrate is missing.
Repeated connect and disconnect cycles drove message rates far beyond tested capacity
Nasdaq’s postmortem said more than 20 connect/disconnect sequences from NYSE Arca consumed resources and generated bursts above 26,000 quote updates per port per second, compared with typical peaks below 1,000 and tested capacity around 10,000.[1] The overload also included inaccurate-symbol quotes and rejects. Real-world protocol behavior exceeded the assumptions represented by the capacity test.
Capacity tests need failure-shaped traffic
Normal peak load is not the only relevant workload. Reconnect storms, rejects, replay bursts, and recovery traffic can be much more concentrated than steady-state trading.
Overload exposed a latent software flaw in the redundancy mechanism
Nasdaq said the exceptional inputs degraded the SIP and revealed a latent software-code flaw that prevented built-in redundancy from failing over cleanly.[1] This is a recurring reliability pattern: overload does not simply make a system slower; it can force execution into rarely tested failover paths. A redundant system is only as good as the transition between primary and backup.
Failover must be tested under the conditions that trigger failover
Testing a standby during a quiet maintenance window proves little about whether it can take over while queues are full, peers are reconnecting, and messages are arriving out of order.
Technical recovery took about 30 minutes, but market reopening took much longer
Nasdaq said the SIP issue was technically stabilized by 12:24 p.m., yet trading in all Nasdaq-listed stocks did not fully resume until 3:25 p.m.[2] Regulators, exchanges, and participants used the intervening time to coordinate an orderly reopening. This distinction matters: recovery time includes business validation and ecosystem synchronization, not just the moment engineers declare a process healthy.
Safe restart is part of outage design
Financial systems need pre-agreed reopening procedures, message sequencing, market-data validation, and communication channels so recovery does not become an improvised negotiation.
The SEC used the incident to accelerate systems-resilience regulation
SEC Chair Mary Jo White called the interruption serious and linked it to broader work on safeguards for critical market systems.[3] Regulation Systems Compliance and Integrity, finalized in 2014, formalized stronger expectations around capacity, integrity, resiliency, testing, and incident handling for key market entities.[4]
Nasdaq subsequently proposed architectural and operational improvements
Nasdaq’s disclosures said it implemented measures to improve SIP resilience and later planned migration of the processor technology to its INET platform.[5] The response addressed not only raw capacity but architecture, disaster recovery, and coordination. Once a shared market service becomes systemically important, improving only the line of code that failed is too narrow.
The outage showed how inter-market protocols can produce nonlinear load
A participant reconnecting after trouble may resend state to restore correctness. If multiple attempts fail, each reconnect can create another burst exactly when the receiver is weakest. Protocols therefore need backoff, flow control, and admission limits for recovery traffic. Otherwise every participant behaves rationally while collectively overwhelming the shared processor.
Why the 2013 Nasdaq SIP outage belongs in software-failure history
The Nasdaq SIP outage belongs in software history because it was a failure of both capacity assumptions and redundancy behavior in infrastructure shared by an entire market segment. Abnormal reconnect traffic overwhelmed processing, a latent software flaw blocked clean failover, and the absence of reliable consolidated quotes forced a market-wide halt. The technical issue was repaired quickly, but coordinated reopening took hours. The enduring lesson is that critical shared systems must test overload and failover together, reserve capacity for recovery traffic, and treat restart coordination as part of the architecture. Financial software also needs recovery objectives expressed in business terms: which obligations can be delayed, which positions must remain visible, which customer actions must still be possible, and how operators prove that replayed state has not duplicated or omitted transactions. These questions turn disaster recovery from an infrastructure exercise into a control system for legal and economic commitments. The episode also shows why market-data systems need explicit overload policy. When a participant produces reconnect storms or malformed-symbol traffic, the processor should be able to throttle or isolate that input without sacrificing the rest of the consolidated feed. Such controls require careful fairness rules because every exchange expects equal treatment, but shared infrastructure cannot allow one unhealthy peer to consume enough capacity to make every other peer invisible. Resource isolation is therefore both a software-engineering and a market-governance problem.
Works Cited
- 01
- 02Nasdaq Trader — UTP SIP Issue on August 22, 2013 nasdaqtrader.com
- 03
- 04
- 05
CodeHistory is a living archive. Citations document the evidence used for this edition; later evidence may refine the account.