Why should people on non-NUMA platforms be denied access to memory-layout information that’s already available under the hood? : Bertrand Drouvot, AWS Open Source Databases.
Every buffer PostgreSQL loads into its shared buffer pool lives somewhere in physical memory, a specific OS page, a specific NUMA node, a specific alignment boundary. For years, understanding that layout required either NUMA-aware hardware or a willingness to pay the steep cost of pg_buffercache_numa.
In PostgreSQL 19, that changes, and a new view pg_buffercache_os_pages lands in the pg_buffercache extension (version 1.7), giving every PostgreSQL user a lightweight, platform-independent window into how shared buffers map onto the operating system’s memory pages. Describe these terms: NUMA, TLB. No NUMA support required. No performance tax for what you don’t need. This post traces the full journey from a mailing-list idea to a committed feature, and explains exactly what it means for you as a DBA, developer, or performance engineer.
The Problem: Two Worlds, One Gap
PostgreSQL manages its buffer pool as a flat array of BLCKSZ-sized blocks in shared memory. On the other side of the fence, the OS manages memory in pages, typically 8 KB (normal pages) or 2 MB (huge pages).
When these two sizes don’t align cleanly, a single database buffer can straddle two OS pages. This has real consequences:
- On NUMA systems, a straddled buffer can pull from two different memory nodes → latency spikes.
- With huge pages, misalignment can force the OS to fall back to small pages for edge buffers → TLB pressure.
- During configuration tuning (BLCKSZ, huge pages, PG_IO_ALIGN_SIZE), there was no easy way to verify alignment without NUMA hardware.
The Old Tool: pg_buffercache_numa
Introduced in PostgreSQL 18 (commit ba2a3c2302f), pg_buffercache_numa exposed an os_page_num field that told you exactly which OS page each buffer lived on. But there was a hard gate:
| Requirement | Reality |
| Needs libnuma | Linux-only |
| pg_buffercache_numa is slow | Queries all NUMA metadata per buffer |
| Requires NUMA-enabled system | Fails silently on macOS, FreeBSD, Windows |
Bertrand Drouvot’s observation was simple and sharp: the os_page_num calculation itself has nothing to do with NUMA. It’s pure arithmetic, buffer address, OS page size, done
The Proposal: os_page_num for Everyone
On April 10, 2025, Bertrand opened the thread on pgsql-hackers:
So why not also add this information (os_page_num) in pg_buffercache? It would make this information available on all systems, not just NUMA-enabled ones. It would help understand the memory layout implications of configuration changes such as database block size, OS page size (huge pages, for example).
The initial idea was to add os_page_num as a new column directly to pg_buffercache. Tomas Vondra and Nathan Bossart both agreed on the value, but flagged it as a new feature that had to wait for PG19, past the PG18 freeze.
The Design Journey: Six Months of Refinement
April & July 2025: The First Patch (v1–v2)
Bertrand submitted v1 as a column addition to pg_buffercache. On returning to the thread in July for the commitfest, v2 introduced two things:
- A shared macro
GET_MAX_BUFFER_ENTRIES - A helper function
get_buffer_page_boundaries()
These were extracted from pg_buffercache_numa_pages() to reduce code duplication, an early sign of good engineering hygiene.
July 2025: Cardinality Problem: New View (v3–v7)
Tomas Vondra raised a critical architectural objection:
Adding columns is fine, but it seems rather wrong to change the cardinality. The view is meant to be a 1:1 mapping for buffers, but now suddenly it’s 1:1 with memory pages.
On systems with huge pages, most buffers map 1:1 to OS pages, but edge buffers that straddle a page boundary would produce two rows for one buffer. That would silently break monitoring queries like COUNT(*).
The fix? A dedicated new view: pg_buffercache_os_pages
Bertrand submitted v3 with this approach. The review cycle continued through v7, addressing:
- Documentation wording (
os_page_numdescription capitalization) - Test placement (role-check tests moved into the main
pg_buffercache.sqltest file) - Addition of
CHECK_FOR_INTERRUPTS()(after commiteab9e4eadded it to otherpg_buffercachefunctions)
Reviewer Mircea Cadariu verified the results on a system with a 16 KB OS page size and 8 KB block size and observed the expected two-buffers-per-page output:
postgres=# SELECT current_setting('block_size');
current_setting
-----------------
8192
(1 row)
postgres=# select * from pg_buffercache_os_pages;
bufferid | os_page_num
----------+-------------
1 | 0
1 | 1
2 | 2
2 | 3
3 | 4
3 | 5
4 | 6
4 | 7
5 | 8
5 | 9
6 | 10
6 | 11
7 | 12
7 | 13
8 | 14
8 | 15
9 | 16
9 | 17
10 | 18
10 | 19The Architectural Overhaul (v8–v9, Final)
In November, committer Michael Paquier performed a deep review and identified the remaining issue: the code for pg_buffercache_os_pages() was still largely a copy-paste of pg_buffercache_numa_pages().
His proposed Option 3 was elegant:
“Rename
pg_buffercache_numa_pages()topg_buffercache_os_pages()— which takes a boolean to decide whether NUMA data collection runs. Switch the existingpg_buffercache_numaview to call it withtrue. The newpg_buffercache_os_pagesview calls it withfalse.”
This yielded the final patch series:
| Patch | Purpose |
|---|---|
0001 | Introduce GET_MAX_BUFFER_ENTRIES and get_buffer_page_boundaries |
0002 | Add internal pg_buffercache_numa_pages_internal() with boolean NUMA flag |
0003 | Add pg_buffercache_os_pages view + rename internal function + rewire NUMA view |
Michael committed the result after final tweaks (fixing typedefs.list documentation corrections, etc.).
What You Can Do With It
Verify Buffer Alignment Under Huge Pages
If you’re running with huge_pages = on (2 MB pages) and BLCKSZ = 8192, you should ideally see every buffer on exactly one OS page. But due to PG_IO_ALIGN_SIZE alignment, some buffers may straddle page boundaries:
postgres=# SELECT
pages_per_buffer,
COUNT(*) AS buffer_count
FROM (
SELECT bufferid, COUNT(*) AS pages_per_buffer
FROM pg_buffercache_os_pages
GROUP BY bufferid
) subq
GROUP BY pages_per_buffer
ORDER BY pages_per_buffer;
pages_per_buffer | buffer_count
------------------+--------------
2 | 16384
(1 row)Changing the alignment from PG_IO_ALIGN_SIZE to 2 MB in BufferManagerShmemInit() collapses this to a single row — a powerful diagnostic for both developers and operations teams.
Join With pg_buffercache for Full Context
Since both views are keyed on bufferid, you can join them:
postgres=# SELECT
b.relfilenode,
b.relblocknumber,
b.isdirty,
p.os_page_num
FROM pg_buffercache b
JOIN pg_buffercache_os_pages p USING (bufferid)
WHERE b.relfilenode IS NOT NULL
ORDER BY p.os_page_num, b.relblocknumber;
relfilenode | relblocknumber | isdirty | os_page_num
-------------+----------------+---------+-------------
1262 | 0 | f | 0
1262 | 0 | f | 1
1260 | 0 | f | 2
1260 | 0 | f | 3
1259 | 0 | f | 4
1259 | 0 | f | 5
1259 | 1 | f | 6
1259 | 1 | f | 7
1259 | 2 | f | 8
1259 | 2 | f | 9
1259 | 3 | f | 10
1259 | 3 | f | 11
1259 | 4 | f | 12
1259 | 4 | f | 13
1259 | 5 | f | 14
1259 | 5 | f | 15
33003 | 24186 | f | 16
33003 | 24186 | f | 17
33003 | 53787 | f | 18
33003 | 53787 | f | 19
1259 | 8 | f | 20
1259 | 8 | f | 21
1259 | 9 | f | 22
1259 | 9 | f | 23
33003 | 53788 | f | 24
33003 | 53788 | f | 25
33003 | 53789 | f | 26
33003 | 53789 | f | 27
33003 | 24187 | f | 28
33003 | 24187 | f | 29View Reference:
postgres=# \d pg_buffercache_os_pages
View "public.pg_buffercache_os_pages"
Column | Type | Collation | Nullable | Default
-------------+---------+-----------+----------+---------
bufferid | integer | | |
os_page_num | bigint | | |Access: Requires pg_monitor membership (same as pg_buffercache).
Cost: Very low — no NUMA syscalls, no mincore(). Pure shared-memory pointer arithmetic.
How it connects to the Broader PG19 Story?
This feature is a perfect example of the incremental refinement culture in PostgreSQL development:
It also connects directly to:
- pg_buffercache_numa (PG18) — the sibling view, now sharing the same internal function
- pg_get_shmem_allocations_numa — a parallel Bertrand fix improving wording in that function’s comments
- Huge Pages tuning (huge_pages = on, huge_page_size) — where buffer alignment analysis is most impactful
- BLCKSZ customization — non-default block sizes (4 KB, 16 KB, 32 KB) interact differently with OS pages
Conclusion:
PostgreSQL 19’s pg_buffercache_os_pages makes shared-buffer memory-layout information accessible without requiring NUMA support. It helps DBAs and developers investigate buffer-to-OS-page mappings and understand memory alignment.
This enhancement demonstrates how small improvements in observability can make PostgreSQL internals easier to understand and troubleshoot. It also reflects PostgreSQL’s commitment to thoughtful design and cross-platform accessibility.
The key takeaway: Better memory visibility enables more informed investigation and tuning of PostgreSQL memory configurations.