No history yet

Process and Memory Architecture

The PostgreSQL Orchestra

When you start a PostgreSQL server, you're not just launching a single program. You're kicking off a whole team of specialized processes that work together. The conductor of this orchestra is a process called the Postmaster, which is the first to start and the last to shut down. Its main job is to listen for incoming client connections and manage the lifecycle of all other PostgreSQL processes.

When a client application wants to connect to the database, the Postmaster authenticates the request. If successful, it performs a fork(), creating a new child process known as a backend process (or postgres process). This new process inherits the Postmaster's configuration and then handles all communication for that one client connection. This process-per-connection model provides excellent isolation. A crash in one backend process won't take down the entire database server.

Besides the Postmaster and backend processes, several other crucial background workers are running. The WAL Writer is responsible for writing transaction logs to disk, ensuring durability. The Checkpointer periodically writes modified data from memory to the data files on disk, creating a known good state for recovery. Finally, the Background Writer trickles modified data to disk in the background, reducing the workload spikes for the Checkpointer.

Shared and Local Memory

All these processes need to coordinate, and they do so through a large segment of shared memory allocated when PostgreSQL starts. This shared space is the central hub for data and transaction information, allowing different processes to work on the same data without constantly reading from and writing to disk, which is much slower.

Think of shared memory as a communal workbench where all the specialized workers can access and modify the project's main components.

The largest and most important part of this shared area is the Shared Buffer Pool. This is PostgreSQL's main cache. When you request data from a table, PostgreSQL fetches the required data pages from disk and stores a copy in the shared buffer pool. If another process needs the same data, it can be read directly from this fast, in-memory cache instead of from the slow disk. This is a fundamental part of what makes database operations fast.

Another key area is the WAL Buffer. To ensure data is never lost, PostgreSQL uses a technique called Write-Ahead Logging. Before any change is made to the actual data files, a record of that change is first written to a log. These log records are temporarily stored in the WAL Buffer before being flushed to permanent storage by the WAL Writer. This guarantees that even in a server crash, the database can be restored to a consistent state by replaying these logs.

Finally, the Commit Log (CLOG) is a small but vital part of shared memory that tracks the status of every transaction: was it committed or aborted? This allows PostgreSQL to quickly determine which transactions are visible to others, which is the cornerstone of its concurrency control system.

Lesson image

While shared memory is for collaboration, each backend process also gets its own private memory space. This is used for operations specific to the query it's currently running. The most significant of these is work_mem.

This parameter defines the amount of memory a backend process can use for operations like sorting data (for an ORDER BY clause) or for hash tables used in complex joins. If an operation requires more memory than work_mem allows, PostgreSQL will spill the excess data to temporary files on disk, which dramatically slows down the query. Setting this value correctly is a key part of performance tuning.

Tuning Memory

Understanding this architecture is crucial for tuning PostgreSQL. The main configuration file, postgresql.conf, contains parameters that control the size of these memory areas. For example, shared_buffers sets the size of the Shared Buffer Pool, and work_mem controls the local memory for sorting.

PostgreSQL settings like work_mem, shared_buffers, and maintenance_work_mem must align with workload needs.

There are other important local memory settings too. maintenance_work_mem is a special, larger pool of memory reserved for maintenance tasks like creating an index or running VACUUM. Since these tasks are infrequent but memory-intensive, they get their own budget. temp_buffers are used for caching temporary tables within a session.

Properly configuring these values involves a trade-off. Allocating too much memory to shared_buffers might leave too little for individual processes' , leading to slow queries that spill to disk. Conversely, setting work_mem too high can be wasteful, as memory allocated to one backend is not available to others. A hundred active connections, each with a large work_mem, could quickly exhaust a server's RAM.

Memory AreaScopeControlled ByPurpose
Shared Buffer PoolShared (Global)shared_buffersCache data and index pages.
WAL BufferShared (Global)wal_buffersTemporarily store transaction log records.
CLOGShared (Global)(Internal)Track transaction commit status.
Work MemoryLocal (Per-process)work_memSorting, hashing, and complex join operations.
Maintenance Work MemoryLocal (Per-process)maintenance_work_memMemory for VACUUM, CREATE INDEX, etc.

By understanding how these processes and memory structures interact, you can make informed decisions when configuring your database, leading to a system that is both stable and performant.

Quiz Questions 1/6

What is the primary role of the Postmaster process in PostgreSQL?

Quiz Questions 2/6

Which component of PostgreSQL's shared memory acts as the primary cache for data pages read from disk?