No history yet

Storage and MVCC Internals

The Life of a Row

In PostgreSQL, what you see as a row in a table is internally managed as a "tuple." Each tuple lives on a data page, the fundamental 8KB block of storage on disk. Understanding the lifecycle of a tuple is key to grasping how PostgreSQL handles data modifications and concurrency.

When you INSERT a new row, PostgreSQL creates a new tuple and writes it to a data page. Along with your data, it stores a few hidden system columns. The most important for concurrency are xmin and xmax.

  • xmin stores the transaction ID (XID) that created this tuple version.
  • xmax is initially set to 0. If the tuple is deleted or updated, xmax will be set to the XID of the transaction that performed that operation.

PostgreSQL's storage engine uses an append-only architecture. This means it never modifies data in place. When you run an UPDATE, you're not changing the existing tuple. Instead, PostgreSQL performs two steps:

  1. It marks the original tuple as no longer valid by setting its xmax to the current transaction's ID.
  2. It inserts a completely new tuple with the updated data. This new tuple's xmin is the current transaction's ID.

A DELETE is simpler: it just marks the target tuple as invalid by setting its xmax. The tuple isn't immediately removed from the page. In both cases, the old, invalidated tuple becomes a "dead tuple."

An UPDATE in PostgreSQL is essentially a DELETE followed by an INSERT under the hood. This creates two tuple versions: the old 'dead' one and the new 'live' one.

Managing Space and Visibility

These dead tuples stick around, consuming valuable disk space. If a table experiences frequent updates and deletes—a high churn rate—it can become filled with these obsolete versions. This phenomenon is called table bloat. As bloat increases, queries that scan the table have to wade through more and more useless data, slowing down performance. Indexes also suffer from index bloat, as they retain entries pointing to dead tuples.

To manage this, PostgreSQL uses two important data structures: the (FSM) and the (VM).

The FSM keeps track of pages that have free space available for new tuples. When you insert data, PostgreSQL consults the FSM to quickly find a suitable location, avoiding a costly sequential scan of the table's pages.

The Visibility Map (VM) is a bitmap with two bits per data page. The first bit indicates whether a page is "all-visible," meaning it contains no dead tuples and all its tuples are visible to every active transaction. This allows for a performance optimization called an index-only scan. The second bit tracks whether a page has been frozen, which is related to preventing transaction ID wraparound issues.

Lesson image

Concurrency and Its Challenges

This entire system of managing different tuple versions is called (MVCC). It's what allows a transaction reading from a table to not block a concurrent transaction writing to it. Each transaction operates on a "snapshot" of the database. This snapshot determines which tuples are visible based on their xmin and xmax values relative to the transaction's own ID and start time.

The main advantage of using the MVCC model of concurrency control rather than locking is that in MVCC locks acquired for querying (reading) data do not conflict with locks acquired for writing data, and so reading never blocks writing and writing never blocks reading.

This elegant system has a critical dependency: garbage collection. The VACUUM process is responsible for cleaning up dead tuples. However, it can only remove tuples that are not visible to any active transaction. A single, long-running transaction can prevent VACUUM from cleaning up dead tuples created after it began. This is because the old tuples must be preserved in case that long transaction needs to see them.

This can stall garbage collection for the entire system, leading to rapid table and index bloat, especially in high-churn environments. Performance degrades as queries must sift through an ever-increasing number of dead tuples. Monitoring for long-running transactions is therefore crucial for maintaining a healthy PostgreSQL database.

Another serious risk is (XID). PostgreSQL uses a 32-bit integer for transaction IDs, which can be exhausted. To prevent this, old transaction IDs are "frozen," marking them as always visible in the past. VACUUM is responsible for freezing old tuples. If a long-running transaction prevents vacuuming for too long, the database can run out of assignable XIDs and will be forced to shut down to prevent data corruption. This is a catastrophic failure, so PostgreSQL has autovacuum settings designed to aggressively prevent it, but poor configuration or extreme transaction patterns can still pose a risk.

Understanding these internal mechanisms—how tuples are versioned, how space is managed, and how concurrency is controlled—is essential for diagnosing performance issues and managing a PostgreSQL database at scale.