Snowflake and the Separation of Cloud Storage from Database Compute
Snowflake rebuilt the data warehouse for cloud infrastructure by separating durable storage from independently scalable compute clusters while coordinating them through shared metadata and services.
Cloud infrastructure weakened the assumption that storage and compute must scale together
Traditional shared-nothing analytical databases often partitioned data across nodes that each owned both processors and disks. That architecture can scale well, but changing cluster size may require moving data because compute capacity and storage placement are coupled. Snowflake’s founders treated elastic cloud infrastructure as an opportunity to rethink that relationship. Their 2016 SIGMOD paper described a database service that separated persistent storage from query compute while preserving the performance advantages of parallel execution.[1] Rather than attach every block of durable data permanently to one worker, Snowflake placed table data in cloud object storage and created elastic compute clusters that could access the shared data. This shift turned warehouse capacity into two independently managed dimensions: how much data is retained and how much compute is activated to process it.
The cloud changed database economics as well as hardware
Object storage offered durable, scalable capacity with a different cost model from compute instances. Separating the two let a warehouse consume CPU only when workloads required it while keeping data available independently.
Snowflake called the design multi-cluster shared data
The original paper described three architectural layers: data storage, virtual warehouses, and cloud services.[1] Durable data lived in cloud storage, query execution ran in shared-nothing compute clusters, and a services layer coordinated metadata, optimization, transactions, security, and infrastructure management. Snowflake called the combination a multi-cluster, shared-data architecture because many independent compute clusters could operate against one logically shared data repository. The design borrowed benefits from both shared-disk and shared-nothing traditions without being identical to either. Shared storage simplified data management and elasticity, while isolated compute clusters reduced contention between workloads. The cloud-services layer supplied the global coordination needed to make those pieces look like one database service.
Caching connected remote durable storage to fast query execution
Separating storage and compute introduces a performance problem: object storage is remote from the processors executing a query. Snowflake’s paper addressed this with local caching on compute nodes, allowing hot data to remain near the execution engine while the authoritative copy stayed in cloud storage.[1] Current Snowflake documentation still describes the architecture as centralized persisted data combined with MPP compute clusters and a coordinating services layer.[2] This detail is important because separation does not mean compute is stateless in every moment. Local disks can hold temporary data and caches, but those copies are disposable. A warehouse can be suspended without making the durable database disappear, and another warehouse can later read the same underlying tables.
Decoupling durability from locality made compute disposable
A cache can be lost and rebuilt; durable table data cannot. Keeping those responsibilities separate allows compute clusters to start, stop, resize, or multiply without redefining ownership of the base data.
Virtual warehouses made compute an independent resource
In Snowflake terminology, a virtual warehouse is a cluster of compute resources used for query execution and data-manipulation work.[3] Different warehouses can run against the same stored data while having their own CPU, memory, and local cache. This makes workload isolation a first-class concept: an ETL job can use one warehouse while interactive analysts use another. Scaling compute no longer necessarily requires copying or repartitioning the persistent data. That model also made cost visible in a new way because users could suspend warehouses when idle and size them according to the workload. The database became an elastic service composed from independently consumable resource layers rather than one fixed appliance.
Scale up and scale out became separate controls
A warehouse can be resized to give one query more resources, while multi-cluster configurations add parallel clusters to absorb concurrency. Those are different responses to different bottlenecks.
Independent clusters reduced interference among concurrent workloads
Snowflake’s documentation describes each virtual warehouse as an independent compute cluster that does not share compute resources with other warehouses.[2] Multi-cluster warehouses can add clusters when query concurrency rises.[3] This architecture addressed a long-standing warehouse problem: one large report or load job can consume resources needed by other users. Physical isolation makes it possible to assign workloads separate compute without creating separate copies of the database. The shared storage layer keeps the data logically unified while compute is multiplied around it. That combination became one of the most influential patterns in cloud analytics because it aligned technical isolation with organizational needs such as separate teams, service levels, and budgets.
Shared durable data enabled new forms of data sharing
Once multiple compute clusters can query the same stored data without each owning a private copy, the architecture can support sharing models that are difficult in node-coupled warehouses. Snowflake’s Secure Data Sharing documentation explains that consumers can query shared objects without the provider copying the underlying data into the consumer account; consumers pay for the compute used to query it.[4] That feature is a product capability built above access control and metadata, but it reflects the underlying architectural separation. Storage ownership and query execution are distinct enough that one data set can serve different compute contexts. The database becomes a managed coordination layer around shared durable information rather than a single cluster containing both the only copy of the data and the only CPUs permitted to read it.
Separation changed organizational architecture too
Teams could isolate compute budgets and workloads while still working from governed shared data. Infrastructure boundaries no longer had to map one-to-one onto copies of the warehouse.
The architecture became a reference point for cloud data systems
By the mid-2020s, separating storage from elastic compute had become a common design pattern in cloud data platforms. In 2026, ACM SIGMOD awarded the Snowflake Elastic Data Warehouse paper its Test-of-Time Award, citing its influence on research and industry and the design of modern cloud data systems.[5] The recognition is notable because the paper described a production architecture rather than a purely theoretical algorithm. Snowflake’s influence came from demonstrating that the cloud was not merely a cheaper place to host an existing warehouse. It could justify reorganizing the database around object storage, disposable compute, and managed services.
Why Snowflake belongs in database history
Snowflake belongs in database history because it made storage-compute separation a mainstream architectural principle for analytical databases. The 2016 design kept durable data in cloud storage, ran queries in elastic virtual warehouses, and coordinated the system through shared services.[1] Current documentation preserves that layered model, while features such as independent warehouses and zero-copy sharing extend the consequences of the separation.[2][4] The 2026 SIGMOD Test-of-Time Award confirms how widely the idea influenced later systems.[5] Modern cloud warehouses differ in implementation, but many now assume that compute should be able to scale, stop, and multiply without forcing the durable database to move with it.
Works Cited
- 01Dageville et al. — The Snowflake Elastic Data Warehouse snowflake.com
- 02Snowflake Documentation — Key Concepts and Architecture docs.snowflake.com
- 03Snowflake Documentation — Virtual Warehouses docs.snowflake.com
- 04Snowflake Documentation — About Secure Data Sharing docs.snowflake.com
- 05ACM SIGMOD — 2026 Test-of-Time Award sigmod.org
CodeHistory is a living archive. Citations document the evidence used for this edition; later evidence may refine the account.
Submit a research lead