No history yet

Transaction Logging Mechanisms

The Log-First Rule

The 'A' in ACID stands for atomicity, the all-or-nothing guarantee for database transactions. But how does a database keep this promise if the power cuts out mid-operation? The answer lies in a robust bookkeeping strategy called Write-Ahead Logging, or WAL.

Write-Ahead Logging (WAL)

Transactions do not write directly to disk.

Rule:

Log first, data later

The core principle is simple but powerful. Before a database ever modifies its main data files on disk, it first writes a description of the intended change to a separate file: the transaction log. This log is an append-only, sequential record of every action. If a crash occurs, the database doesn't have to guess what was happening. It can consult the log, which acts as the definitive source of truth, to either complete or roll back incomplete work, thus preserving atomicity.

How It Works

To understand WAL, you need to picture three key locations: the data files on disk (the permanent database), the transaction log on disk (the permanent record of changes), and the buffer cache in memory (a temporary workspace).

When a transaction begins, the database doesn't immediately write to the data files. Instead, it copies the relevant data pages from disk into the buffer cache. All modifications happen here first. A page that has been changed in memory but not yet saved to the data file is called a 'dirty page'.

Here’s the critical step: before that dirty page can be written to the main data file, a log record describing the change must be written to the transaction log on disk. This log record contains all the information needed to either redo or undo the change.

To keep everything in order, each log record is stamped with a unique, ever-increasing Log Sequence Number (LSN). These numbers act like a timeline. Every data page in the buffer cache also tracks the LSN of the most recent log record that describes a change to it. This ensures the database can verify that the log record has been safely written to disk before it allows the corresponding data page to be written.

Buffer Management and Recovery

Databases use specific rules to decide when to move data between memory and disk. These rules, known as buffer management policies, involve a trade-off between performance and simplicity of recovery.

  • Steal vs. No-Steal: A 'Steal' policy allows the buffer manager to write a dirty, uncommitted page to disk. This is efficient, as it frees up memory, but it means the database might contain changes from transactions that haven't finished. If a crash happens, these changes must be undone. A 'No-Steal' policy is simpler—it keeps all uncommitted changes in memory—but can be inefficient and require a huge buffer cache.

  • Force vs. No-Force: A 'Force' policy requires all changes made by a transaction to be written to the data file on disk the moment the transaction commits. This is simple but slow, as it can cause many small, random disk writes. A 'No-Force' policy is much faster, as it allows a committed transaction's dirty pages to remain in the buffer cache and be written out later in a more efficient batch. Most high-performance databases use a 'No-Force' approach.

Most modern relational databases use a Steal and No-Force policy. It offers the best performance, but it also creates the most complex recovery scenario: the database on disk may contain uncommitted changes that need to be undone, and it may be missing committed changes that need to be redone.

This is where the log records become essential. A famous algorithm called (Algorithm for Recovery and Isolation Exploiting Semantics) uses the log to restore consistency after a crash in a system with a Steal/No-Force policy. It works in three phases:

  1. Analysis: It scans the log from the last checkpoint to identify which transactions were active at the time of the crash and which pages were dirty in the buffer cache.
  2. Redo: It starts from an appropriate point in the log and reapplies all changes, even those from uncommitted transactions. This brings the database to the exact state it was in at the moment of the crash.
  3. Undo: It then scans backward through the log and reverts the changes made by any transactions that were identified as incomplete during the analysis phase. This is done by applying special 'undo' log records. The result is a database containing only the effects of successfully committed transactions.

Ultimately, Write-Ahead Logging and the policies that govern it are a masterclass in managing trade-offs. By accepting a more complex recovery process, databases can achieve significantly higher performance during normal operation, all while upholding the crucial promise of atomicity.