Virtual Memory, the MMU, TLB & Paging

A physical address is a number that directly identifies a byte in RAM — it is the actual wire signal sent to the memory controller. If your machine has 8 GB of RAM, physical addresses run from 0x0000_0000 to 0x1_FFFF_FFFF.

A virtual address is what your program uses. When you declare int x = 42; and then take &x, you get a virtual address — a number in your process's private address space. Two completely different processes can both have a variable at virtual address 0x7fff0000, and they will not interfere, because those virtual addresses map to different physical addresses.

Think of it like this: a hotel has physical room numbers (the real rooms: 101, 102, 103...). A guest booking system might call the same room "Deluxe Suite A", "Room 101", or "Unit 1-A" depending on the context. Virtual addresses are like those nicknames — the MMU is the front desk that looks up the real room number.

On a 64-bit x86 system (x86-64), virtual addresses are 64 bits wide, but current CPUs only implement 48 or 57 of those bits (meaning the usable virtual address space is 256 TB or 128 PB). Physical addresses are typically 46–52 bits wide, mapping up to 64 TB–4 PB of RAM. The virtual space is much larger than what fits in physical RAM — and that asymmetry is intentional and powerful.

// You can observe virtual addresses directly:
#include <stdio.h>
int global_var = 10;
int main(void) {
    int local_var = 20;
    printf("virtual addr of global: %p\n", (void *)&global_var);
    printf("virtual addr of local:  %p\n", (void *)&local_var);
    printf("virtual addr of main:   %p\n", (void *)main);
    // Run this twice simultaneously — same addresses, different physical backing!
    return 0;
}

Virtual memory was not invented for one reason — it solved four problems at once, which is why it became universal.

1. Isolation / Protection. Without VM, a buggy or malicious program could write to any byte of RAM, corrupting the OS or another process. With VM, each process's translations are private. Accessing an address that has no valid mapping causes a page fault — the MMU traps to the OS, which can kill the offending process cleanly. This is why a segfault kills your program, not the kernel.

2. Abstraction / Simplicity. Every program can be compiled assuming it starts at address 0 (or some fixed virtual base). You do not have to know where in physical RAM you will land. The linker, loader, and OS handle that.

3. Demand Paging / Overcommit. Your program declares a 1 GB array, but only ever reads 10 MB of it. Without VM, you'd need to allocate all 1 GB of physical RAM upfront. With VM, the OS allocates physical pages lazily — only when you actually touch them. This is why malloc(1_000_000_000) succeeds even on a machine with 512 MB free: the virtual mapping exists, but physical RAM is not committed until pages are accessed.

4. Sharing. Many processes run the same library (e.g., libc.so). Without VM, each would need its own physical copy. With VM, the same physical pages of libc can be mapped into many virtual address spaces simultaneously, with read-only protection. Memory-mapped files work the same way — the file's pages are shared between the filesystem cache and the process's address space.

// Observe demand paging: allocate 1 GB, check physical usage
// Run: watch -n 0.5 'ps aux | grep demand'
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#define GB (1024UL * 1024 * 1024)
int main(void) {
    char *p = malloc(GB);   // virtual pages mapped, NO physical RAM yet
    printf("Allocated 1 GB virtually. Press enter to touch it...\n");
    getchar();
    memset(p, 0, GB);       // NOW physical pages are faulted in
    printf("Done. Physical RAM used now. Press enter to exit.\n");
    getchar();
    free(p);
    return 0;
}

The MMU does not translate individual bytes — that would require an impossibly large lookup table (a 48-bit address space has 256 trillion possible addresses). Instead, memory is divided into fixed-size chunks called pages (virtual) and frames (physical). Translation maps a virtual page to a physical frame.

On x86-64 and AArch64, the default page size is 4 KiB = 4096 bytes = 2¹² bytes. This means the lowest 12 bits of a virtual address are the page offset — they identify which byte within the page — and they pass through the translation unchanged. Only the upper bits (the virtual page number, VPN) get translated to a physical frame number (PFN). The final physical address is (PFN << 12) | page_offset.

Why 4 KiB? It is a historical sweet spot: small enough that internal fragmentation (wasted space at the end of a page) is tolerable, large enough that the translation tables stay manageable. Modern systems also support huge pages (2 MiB on x86-64 with 4-level paging, or 1 GiB) for workloads like databases and HPC where TLB pressure from millions of 4K pages becomes a bottleneck.

You can query your system's page size with getconf PAGESIZE (usually 4096), or in C with sysconf(_SC_PAGESIZE). In the kernel, it is the constant PAGE_SIZE.

// Demonstrate page alignment
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(void) {
    long page = sysconf(_SC_PAGESIZE);  // typically 4096
    char *p = malloc(1);               // some small allocation
    // Extract page number and offset from the virtual address
    unsigned long addr = (unsigned long)p;
    unsigned long vpn  = addr >> 12;   // top bits: virtual page number
    unsigned long off  = addr & 0xFFF; // bottom 12 bits: offset within page
    printf("page size = %ld bytes\n", page);
    printf("addr = %lx  VPN = %lx  offset = %lx\n", addr, vpn, off);
    free(p);
    return 0;
}

The page table is the OS-maintained data structure that maps virtual page numbers to physical frame numbers. At its simplest, imagine a flat array: pte_t page_table[2^36] for a 48-bit address space with 4 KiB pages — you need 2^(48-12) = 2^36 ≈ 68 billion entries. Even at 8 bytes per entry, that is 512 GB just for the page table — clearly unworkable.

The solution is multi-level paging: a tree of tables, where each level covers part of the virtual address. x86-64 uses a 4-level page table (and optionally 5-level for larger address spaces):

• PGD — Page Global Directory (level 4, top)
• PUD — Page Upper Directory (level 3)
• PMD — Page Middle Directory (level 2)
• PTE — Page Table Entry (level 1, leaf)

A 48-bit virtual address is split as: [PGD:9][PUD:9][PMD:9][PTE:9][Offset:12]. Each level uses 9 bits to index into a 512-entry table (512 × 8 bytes = 4096 bytes = exactly one page — elegant!). Only the portions of the tree that are actually needed are allocated. A process that uses only 10 MB of memory only needs a tiny fraction of the full 4-level tree.

Each leaf PTE (Page Table Entry) contains:

• The PFN (physical frame number) — upper bits
• Present (P) bit — is this page currently in RAM?
• Read/Write (R/W) bit — can it be written?
• User/Supervisor (U/S) bit — can user-mode code access it?
• Accessed (A) bit — set by hardware when page is read
• Dirty (D) bit — set by hardware when page is written
• No-Execute (NX/XD) bit — is execution of code on this page allowed?

These bits are how the OS enforces memory protection. A stack page is R/W but not executable. Code pages are readable and executable but not writable. The kernel's own memory is marked Supervisor-only. Violating any of these causes a page fault.

/* In the Linux kernel (arch/x86/include/asm/pgtable_types.h),
   PTE flags look roughly like this: */
#define _PAGE_PRESENT   (1UL << 0)   /* page is in RAM */
#define _PAGE_RW        (1UL << 1)   /* read/write (0 = read-only) */
#define _PAGE_USER      (1UL << 2)   /* user-mode accessible */
#define _PAGE_ACCESSED  (1UL << 5)   /* hardware sets on access */
#define _PAGE_DIRTY     (1UL << 6)   /* hardware sets on write */
#define _PAGE_NX        (1UL << 63)  /* no-execute (NX bit) */

/* A typical user read-only code page PTE might look like:
   PFN=0x1A3B00 | _PAGE_PRESENT | _PAGE_USER
   A writable data page:
   PFN=0x2C4D00 | _PAGE_PRESENT | _PAGE_RW | _PAGE_USER */

The Memory Management Unit (MMU) is a hardware component (today integrated into the CPU core itself) that intercepts every memory access and translates the virtual address to a physical address before the access reaches the cache or RAM.

The MMU knows where the current process's top-level page table lives because the OS stores that physical address in a special register. On x86-64, this is CR3 (Control Register 3). On AArch64, it is TTBR0_EL1 / TTBR1_EL1 (Translation Table Base Registers). When the OS performs a context switch — swapping from process A to process B — it changes CR3 to point to B's page table. Instantly, every virtual address now means something different. This is how process isolation is hardware-enforced.

When the MMU receives a virtual address and there is no cached translation (more on TLB below), it performs a page table walk (also called a hardware table walk or page walk): it reads the PGD entry, then the PUD, then the PMD, then the PTE, from physical memory. Each step is a memory read. At the leaf PTE, it extracts the PFN, appends the page offset, and produces the physical address.

On x86-64, the MMU does all of this autonomously in hardware — you do not write any code for it. On some RISC architectures like MIPS and older SPARC, the hardware raises a TLB-miss exception and the software (the kernel) walks the table. RISC-V and ARM both have hardware walkers.

The OS interacts with the MMU by: (1) building and maintaining page tables in memory, (2) writing the root address into CR3/TTBR, (3) handling page faults when the MMU cannot complete a translation.

/* Reading CR3 (x86-64 kernel code — requires privilege level 0) */
#include <linux/types.h>
static inline unsigned long read_cr3(void) {
    unsigned long val;
    asm volatile("mov %%cr3, %0" : "=r" (val));
    return val;
    /* This physical address points to the PGD of the current process.
       Bits [12..52] are the PFN; bits [0..11] are control flags (PCID etc.) */
}
/* In user space you can't read CR3, but /proc/<pid>/maps shows the
   virtual layout, and /proc/<pid>/pagemap lets you find physical frames. */

Let's trace exactly what happens when a program reads a single byte from a virtual address on x86-64 with 4-level paging and 4 KiB pages.

Suppose the virtual address is 0x00007fff_deadbeef.

In binary (48 bits used):

 0000 0000 0000 0000  0111 1111 1111 1111  1101 1110 1010 1101  1011 1110 1110 1111
 |___________________________|_________|_________|_________|________________________|
         sign extension         PGD[9]    PUD[9]    PMD[9]    PTE[9]    Offset[12]

Decomposed:

• PGD index: bits [47:39] = 0b000_0000_01 = 1
• PUD index: bits [38:30] = 0b111_1111_11 = 511
• PMD index: bits [29:21] = 0b11_1101_111 = 495
• PTE index: bits [20:12] = 0b0_1010_1101 = 173
• Offset: bits [11:0] = 0xeef

Step 1: CPU reads CR3 → physical address of PGD (e.g., 0x1000_0000) Step 2: Physical read at 0x1000_0000 + 1*8 → PUD base address (e.g., 0x2000_0000) + flags Step 3: Physical read at 0x2000_0000 + 511*8 → PMD base address + flags Step 4: Physical read at PMD_base + 495*8 → PTE table base + flags Step 5: Physical read at PTE_base + 173*8 → leaf PTE: PFN | flags Step 6: Check Present bit. If 0 → page fault (see next concept). If 1 → continue. Step 7: Physical address = (PFN << 12) | 0xeef Step 8: Memory access proceeds to cache/RAM using physical address.

Without the TLB, every single memory access would require 4 extra memory reads just for the page walk — a 5x slowdown on memory-intensive code. The TLB is why this is fast in practice.

/* You can observe the virtual address decomposition in user space.
   This shows how Linux/x86-64 splits a virtual address: */
#include <stdio.h>
#include <stdint.h>
void decode_vaddr(uint64_t va) {
    /* x86-64 canonical form: bits [63:48] must equal bit 47 */
    uint64_t pgd = (va >> 39) & 0x1FF;
    uint64_t pud = (va >> 30) & 0x1FF;
    uint64_t pmd = (va >> 21) & 0x1FF;
    uint64_t pte = (va >> 12) & 0x1FF;
    uint64_t off = va & 0xFFF;
    printf("VA = 0x%016lx\n", va);
    printf("  PGD[%3lu] PUD[%3lu] PMD[%3lu] PTE[%3lu] off=0x%03lx\n",
           pgd, pud, pmd, pte, off);
}
int main(void) {
    int x;
    decode_vaddr((uint64_t)&x);
    decode_vaddr((uint64_t)main);
    return 0;
}

If the MMU had to walk the page table for every memory access, programs would be 4–5× slower. The hardware solution is the Translation Lookaside Buffer (TLB) — a small, extremely fast hardware cache that stores recent virtual→physical translations.

The TLB is inside the CPU, usually within the MMU itself, and is implemented as a fully-associative or set-associative content-addressable memory. A typical L1 TLB holds 64–128 entries. Modern CPUs have a hierarchy: L1 iTLB (instruction), L1 dTLB (data), and a larger L2 unified TLB. An Intel Core i7 might have 64 L1 dTLB entries + 256 L2 TLB entries.

On a memory access:

1. The CPU sends the VPN to the TLB.
2. TLB hit (common case, ~99%): The TLB returns the PFN in ~1 cycle. Done.
3. TLB miss: The hardware walker fires, walks the page table (4 memory reads), loads the new translation into the TLB (possibly evicting an old one), and retries.

This is why working set locality matters so much for performance. If your program touches more unique pages than the TLB can hold, you get constant TLB misses and the effective cost of every memory access rises dramatically. This is called TLB thrashing.

TLB Shootdowns are a critical multi-core concern: when the OS modifies a page table mapping (e.g., munmap, mprotect, fork copy-on-write), it must invalidate the stale TLB entries on every CPU core that might have cached that translation. This requires an inter-processor interrupt (IPI) — expensive! Large VMs and databases with many threads doing frequent mmap/munmap can see significant overhead from TLB shootdowns.

CR3 writes flush the entire TLB (on a context switch). Modern CPUs mitigate this with PCID (Process-Context IDentifiers), which tag TLB entries with a process ID so they survive context switches.

# You can observe TLB miss rates with Linux perf:
# perf stat -e dTLB-loads,dTLB-load-misses,iTLB-loads,iTLB-load-misses ./your_program

# Example output for a program with poor locality:
#   dTLB-loads:       48,392,711
#   dTLB-load-misses:  3,201,445  #  6.61% of all data TLB accesses missed

# For a matrix multiply comparing row-major vs column-major access:
# Row-major (cache + TLB friendly):
#   dTLB-load-misses:     12,443  #  0.01%
# Column-major (poor locality):
#   dTLB-load-misses:  4,821,002  #  8.2%  → significant TLB pressure

A page fault is a hardware exception raised by the MMU when it cannot complete an address translation. Page faults are not always errors — in fact, most are intentional and essential to how virtual memory works. There are three kinds:

1. Minor (soft) page fault: The page is valid but not currently mapped in this process's page table — perhaps it was never accessed yet (demand paging), or it was swapped in from a file that is already in the page cache. No disk I/O needed. The kernel just updates the PTE. Fast (~microseconds).

2. Major (hard) page fault: The page is valid but its content is on disk (it was swapped out, or it is a memory-mapped file not yet read). The kernel must issue disk I/O to bring it back. Slow (~milliseconds — a 1000× penalty over RAM access).

3. Invalid page fault: The address is genuinely bad — the program accessed an unmapped address, wrote to a read-only page, tried to execute a non-executable page, or user code tried to access a kernel-only page. The kernel delivers SIGSEGV to the process → segfault. This is the fault you see when you dereference a null pointer.

The fault handling flow on x86-64 Linux:

1. MMU raises exception #14 (Page Fault).
2. CPU saves registers, pushes error code onto the kernel stack, jumps to the kernel's page fault handler (in Linux: do_page_fault() → handle_mm_fault()).
3. CR2 register contains the faulting virtual address.
4. The kernel looks up the VMA (Virtual Memory Area) containing that address using find_vma().
5. Based on the VMA's flags and the fault type, the kernel either: maps a new anonymous page, reads from swap, reads from a file, performs copy-on-write, or sends SIGSEGV.
6. Returns from the exception; the instruction that caused the fault is restarted.

/* Deliberately trigger and catch page faults via signal handling */
#include <stdio.h>
#include <signal.h>
#include <setjmp.h>
#include <stdlib.h>
static jmp_buf jbuf;
static void segv_handler(int sig, siginfo_t *si, void *ctx) {
    printf("Page fault (SIGSEGV) at address: %p\n", si->si_addr);
    longjmp(jbuf, 1);  /* recover — normally you'd crash here */
}
int main(void) {
    struct sigaction sa = { .sa_sigaction = segv_handler,
                            .sa_flags = SA_SIGINFO };
    sigaction(SIGSEGV, &sa, NULL);
    if (setjmp(jbuf) == 0) {
        int *null_ptr = NULL;
        *null_ptr = 42;  /* triggers invalid page fault → SIGSEGV */
    }
    printf("Recovered. Continuing.\n");
    return 0;
}

The page table tells the MMU which virtual pages have valid translations. But the kernel also needs a higher-level view of the address space — it needs to know what each region is (stack, heap, code, mmap'd file, etc.) and what to do when a fault occurs in each region.

Linux represents this with VMAs — Virtual Memory Areas (struct vm_area_struct in include/linux/mm_types.h). Each VMA describes a contiguous range of virtual addresses with uniform properties: the start/end virtual address, protection flags (VM_READ, VM_WRITE, VM_EXEC), and optionally a backing file/offset for file-mapped regions.

All the VMAs of a process are stored in two data structures: a red-black tree (for fast O(log n) lookups by address) and a linked list (for sequential iteration). Together they make up the process's struct mm_struct — the memory descriptor.

You can see all VMAs for a running process at /proc/<pid>/maps (human-readable) or /proc/<pid>/smaps (with detailed statistics):

55a1b2c4d000-55a1b2c4e000 r--p 00000000 08:01 1234567  /usr/bin/cat
55a1b2c4e000-55a1b2c4f000 r-xp 00001000 08:01 1234567  /usr/bin/cat
7f3a1b2c0000-7f3a1b4c0000 r--p 00000000 08:01 9876543  /lib/x86_64-linux-gnu/libc.so.6
7ffd12340000-7ffd12360000 rw-p 00000000 00:00 0        [stack]

Each line is one VMA. The r-xp means readable, executable, private mapping.

// Read your own process's VMAs programmatically
#include <stdio.h>
#include <stdlib.h>
int main(void) {
    FILE *maps = fopen("/proc/self/maps", "r");
    if (!maps) { perror("fopen"); return 1; }
    char line[256];
    printf("%-36s %-5s %s\n", "Address Range", "Perms", "Mapping");
    printf("%s\n", "------------------------------------------------------------------");
    while (fgets(line, sizeof(line), maps))
        fputs(line, stdout);
    fclose(maps);
    return 0;
}

The default 4 KiB page size is a great general-purpose choice, but databases, JVMs, scientific computing, and hypervisors often allocate hundreds of gigabytes of memory where millions of 4K pages means millions of TLB entries — impossible to cache. The solution is huge pages.

On x86-64, the PMD entry can directly point to a 2 MiB frame instead of a PTE table (a 2 MiB huge page). The PUD entry can point to a 1 GiB frame (a 1 GiB gigantic page). A single TLB entry now covers 512× or 262,144× more memory.

Linux supports huge pages in two ways:

1. Explicit Huge Pages (hugetlbfs): The kernel pre-allocates a pool of huge pages, accessible via mmap with MAP_HUGETLB or through /sys/kernel/mm/hugepages/. Used by databases like Oracle and applications like DPDK.

2. Transparent Huge Pages (THP): The kernel automatically promotes aligned, contiguous 4K pages to a 2 MiB huge page when possible, without application changes. Enabled via /sys/kernel/mm/transparent_hugepage/enabled. THP can cause latency spikes due to compaction (defragmenting memory to find a contiguous 2 MiB region), so databases often disable THP and use explicit huge pages instead.

The trade-off is always granularity vs. TLB pressure: huge pages mean fewer TLB entries needed (good) but also more wasted physical memory if the page is only partially used (internal fragmentation — bad for sparse workloads).

/* Allocate a 2 MiB huge page explicitly on Linux */
#include <stdio.h>
#include <sys/mman.h>
#define HUGE_PAGE_SIZE (2UL * 1024 * 1024)  /* 2 MiB */
int main(void) {
    /* MAP_HUGETLB requests a huge page from the kernel pool.
       Requires: echo 4 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages */
    void *p = mmap(NULL, HUGE_PAGE_SIZE,
                   PROT_READ | PROT_WRITE,
                   MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB,
                   -1, 0);
    if (p == MAP_FAILED) {
        perror("mmap (try enabling hugepages first)");
        return 1;
    }
    printf("Huge page allocated at %p\n", p);
    /* One TLB entry now covers all 2 MiB */
    munmap(p, HUGE_PAGE_SIZE);
    return 0;
}

One of the most elegant applications of virtual memory is Copy-on-Write (CoW), which makes fork() nearly instant even for large processes.

When fork() is called, the kernel must create a new process with its own address space — but copying gigabytes of RAM would take seconds. Instead:

1. The kernel clones the parent's page tables, pointing both parent and child PTEs at the same physical frames.
2. Both parent and child PTEs are marked read-only, even for pages that were originally writable.
3. The fork() returns immediately — no data was copied.

Now, if neither process writes to a shared page, they continue sharing it forever (this is how shared libraries work — they are read-only and are always shared).

But when either process writes to a CoW page:

1. The MMU raises a write-protection fault (a page fault with the write bit set and the present bit set in the PTE).
2. The kernel allocates a new physical frame.
3. Copies the page content into the new frame.
4. Updates the writing process's PTE to point to the new (private) frame, with write permission restored.
5. The write proceeds on the new private copy.

CoW means fork() + exec() (which replaces the address space immediately) is essentially free — exec throws away the CoW pages without ever actually copying them. This is the copy-on-write optimization that makes Unix shell pipelines fast.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
int shared_data = 42;   /* In parent's page — shared with child via CoW */
int main(void) {
    printf("Before fork: shared_data=%d at vaddr=%p\n",
           shared_data, (void*)&shared_data);
    pid_t pid = fork();
    if (pid == 0) {
        /* Child: writing here triggers CoW — kernel makes a private copy */
        shared_data = 99;
        printf("Child: shared_data=%d at vaddr=%p (new physical frame!)\n",
               shared_data, (void*)&shared_data);
        exit(0);
    } else {
        wait(NULL);
        /* Parent's copy is unchanged — CoW preserved isolation */
        printf("Parent: shared_data=%d (unchanged)\n", shared_data);
    }
    return 0;
}

The best way to build intuition about virtual memory is to observe it directly with Linux's built-in tools.

/proc/<pid>/maps: Lists all VMAs — address range, permissions (r=read, w=write, x=execute, p=private/CoW, s=shared), offset, device, inode, and filename. Run cat /proc/self/maps to see your shell's layout.

/proc/<pid>/smaps: Per-VMA memory statistics including Size (virtual), Rss (resident in RAM), Pss (proportional share of shared pages), Shared_Clean/Dirty, Private_Clean/Dirty, Referenced, Anonymous, Swap.

/proc/<pid>/pagemap: A binary file (8 bytes per virtual page) that lets you translate virtual page numbers to physical frame numbers for any page of any process (requires root for other processes). This is how you can verify CoW behavior: fork() and check whether parent and child share frame numbers before and after a write.

pmap -x <pid>: A friendlier view of /proc/<pid>/maps + /proc/<pid>/smaps.

perf stat -e dTLB-loads,dTLB-load-misses: Hardware Performance Counters that directly measure TLB behavior.

/proc/<pid>/status contains VmPeak, VmSize (current virtual), VmRSS (resident), VmSwap, VmHWM (peak RSS) — the standard summary metrics for memory usage.

For kernel developers, the function show_mem() in mm/oom_kill.c and dump_page() in mm/debug.c are the kernel equivalents of these user-space inspection tools.

#!/bin/bash
# Watch demand paging live: launch a program and track its RSS growth
PID=$(
    # Start a process that slowly touches memory
    bash -c '
        python3 -c "
import time, mmap, os
size = 64 * 1024 * 1024  # 64 MB
m = mmap.mmap(-1, size)
for i in range(0, size, 4096):
    m[i] = 1          # touch one byte per page
    time.sleep(0.001) # slow enough to observe
"' &
    echo $!
)
for i in $(seq 1 10); do
    RSS=$(awk '/VmRSS/ {print $2}' /proc/$PID/status 2>/dev/null)
    VIRT=$(awk '/VmSize/ {print $2}' /proc/$PID/status 2>/dev/null)
    echo "VmSize=${VIRT}kB  VmRSS=${RSS}kB"
    sleep 0.5
done
kill $PID 2>/dev/null