Traditional data warehouses were built on one of two models. In a shared-disk architecture, all computers in a cluster access the same central storage device. This is simple, but can create a bottleneck. In a shared-nothing architecture, each computer node has its own private storage. This scales well but requires data to be shuffled between nodes, which can be complex and slow.

Snowflake uses a third way: a multi-cluster, shared data architecture built for the cloud. This hybrid model takes the best of both worlds. All data lives in a central storage repository, but multiple, independent compute clusters can access it simultaneously without competing for resources. It’s like having a single, massive library where different teams can set up their own reading rooms of any size, all accessing the same collection of books.

This architecture is organized into three distinct, independently scalable layers.

1. Database Storage: This is the foundation where your data lives. When you load data into Snowflake, it's converted into an optimized, compressed, columnar format and stored in cloud storage (like Amazon S3 or Google Cloud Storage) in your account's region.
2. Query Processing: This is the muscle layer. Queries are executed by independent compute clusters called virtual warehouses Each warehouse is an isolated set of resources (CPU, memory) that doesn't interfere with others. You can have one for your data science team, another for financial reporting, and a third for data loading, all accessing the same data without stepping on each other's toes.
3. Cloud Services: This is the brain. This layer coordinates the entire system. It handles query optimization, security, transaction management, and metadata. When you run a query, the services layer figures out the most efficient plan to access the data and tells the virtual warehouse what to do. It’s completely managed by Snowflake, so you don’t have to worry about it.

Snowflake doesn't just dump your files into cloud storage. When you ingest data, Snowflake automatically splits it into small, immutable chunks called micro-partitions. Each micro-partition is typically 50 to 500MB of uncompressed data, stored in that optimized columnar format.

For every micro-partition it creates, Snowflake's Cloud Services layer collects and stores a rich set of metadata. This isn't just the file name; it's a detailed catalog of what's inside.

This metadata is the secret sauce behind Snowflake's performance. It allows the query optimizer to know exactly what data exists in which micro-partitions without ever having to read the files themselves.

This rich metadata enables a powerful optimization technique called pruning. When you run a query with a filter, Snowflake first consults the metadata to figure out which micro-partitions could possibly contain relevant data. It then instructs the virtual warehouse to scan only those partitions, ignoring the rest.

Imagine you have a massive table of sales data spanning ten years, stored in thousands of micro-partitions. If you run a query like this:

Snowflake's optimizer will look at the MIN/MAX metadata for the sale_date column in every micro-partition. It will instantly realize that only a handful of them contain data from July 4th, 2024. The query engine will then completely skip the thousands of other partitions that fall outside that date range. This drastically reduces the amount of data that needs to be read and processed, making queries incredibly fast.

To further optimize this, you can define a clustering key for a table. This tells Snowflake to try to store data with similar key values together in the same micro-partitions. For the sales table, clustering by sale_date would physically group the data by date, making range-based queries even more efficient.

The true breakthrough in this architecture is the complete separation, or decoupling, of storage and compute. Because the compute clusters (virtual warehouses) don't store any data themselves, they can be scaled independently and instantly.

This has massive implications:

• Performance: A data loading process running on one warehouse will never slow down a critical dashboard being used by executives on another warehouse.
• Cost: You can scale compute resources up to handle a massive workload, and then scale them down to zero when they're not needed, paying only for what you use. Storage costs are separate and are typically much lower.
• Flexibility: Different teams can use warehouses sized perfectly for their needs. A small warehouse for ad-hoc queries, a large one for machine learning model training.

By combining a multi-cluster, shared-data model with intelligent storage and metadata management, Snowflake provides a platform that is both powerful and elastic. This architecture is the foundation for everything else you can do in Snowflake.