The Von Neumann Architecture

Before stored-program computers, machines were hardwired for specific tasks. Changing the computation meant physically reconfiguring the machine. Von Neumann's insight — formalized in the 1945 First Draft of a Report on the EDVAC — was that instructions are just numbers, and numbers can be stored in memory alongside data.

This is profound. A program is not something carved into hardware; it is data that the CPU reads and interprets. The number 0x83C001 in x86 assembly is both a valid integer and a valid instruction (add eax, 1). The CPU decides which it is purely by context: if the program counter points to that address, it is an instruction; if the CPU is computing with it, it is data.

The implications are enormous:

1. Universality: The same hardware can run any program by loading different data.
2. Self-modifying code: A program can write new instructions into memory and then execute them (the basis of JIT compilation).
3. Compilers: A program (the compiler) can produce another program (the binary) as its output data.
4. Operating systems: The kernel is just a program loaded at boot. It can load other programs by copying instruction bytes into memory.

The analogy: think of a piano roll for a player piano. The piano (hardware) is fixed. The music changes because you swap the roll (program). Von Neumann's insight is that the roll and the sheet music you write while the piano plays can live in the same drawer.

/* Demonstration: instructions ARE bytes in memory.
   Compile with: gcc -o selfmod selfmod.c -z execstack -no-pie
   (execstack only for demo — never do this in production!) */
#include <stdio.h>
#include <stdint.h>

int add_one(int x) { return x + 1; }

int main(void) {
    /* Print the raw bytes of the add_one function */
    uint8_t *p = (uint8_t *)add_one;
    printf("First 8 bytes of add_one: ");
    for (int i = 0; i < 8; i++)
        printf("%02x ", p[i]);
    printf("\n");
    /* On x86-64 you'll see: 55 48 89 e5 ... (function prologue)
       These are the same bytes that would be 'data' if we
       passed them to memcpy. The CPU treats them as instructions
       only because the instruction pointer reaches them. */
    return 0;
}

Von Neumann described five logical units. Modern hardware has evolved dramatically, but every component maps to one of these five:

1. Input Unit — brings data from the outside world (keyboard, disk, network). In modern systems this is the I/O subsystem: PCIe devices, USB controllers, SATA controllers.

2. Output Unit — sends results to the outside world (display, disk, network). In practice, input and output devices often share hardware (a NIC both receives and transmits).

3. Memory Unit — stores both program instructions and data as a flat array of addressable bytes. Von Neumann specified a single unified memory space.

4. Arithmetic and Logic Unit (ALU) — performs arithmetic (add, subtract, multiply) and logical (AND, OR, NOT, compare) operations. This is the part of the CPU that actually computes.

5. Control Unit (CU) — orchestrates everything. It reads (fetches) the next instruction from memory, figures out what it means (decodes it), and directs the ALU and memory accordingly. The CU contains the program counter (PC) — the register that holds the address of the next instruction to execute.

The ALU and CU together constitute the CPU. In 1945 these were separate racks of vacuum tubes. Today they are transistors on a single die, but the logical separation is preserved in every architecture manual.

The bus connects all these components. Think of it as a highway: the address bus carries which memory location to access, the data bus carries the actual bytes, and the control bus carries signals like READ/WRITE and INTERRUPT. The width of the data bus (8-bit, 16-bit, 32-bit, 64-bit) is one of the things people mean when they say a computer is '64-bit'.

/* You can observe the register file (part of the CU/ALU) in action
   by compiling a simple function and reading its assembly.
   On Linux with gcc: gcc -O1 -S -o out.s example.c */

int add(int a, int b) {
    return a + b;
}
/* On x86-64 this compiles to roughly:
     add:
         lea eax, [rdi + rsi]   ; ALU: compute a + b
         ret                     ; CU: return to caller
   rdi, rsi, eax are CPU REGISTERS — tiny memory cells inside the CPU.
   The ALU reads from registers, computes, writes result back. */

The heartbeat of every von Neumann machine is the fetch-decode-execute cycle (sometimes called the instruction cycle). On every tick (or set of ticks) of the clock, the CPU repeats these steps:

1. FETCH: The control unit reads the bytes at the address stored in the program counter (PC) from memory. On x86-64, instructions are variable-length (1 to 15 bytes). On RISC architectures (RISC-V, ARM), instructions are fixed-width (typically 4 bytes). After fetching, the PC is automatically incremented to point to the next instruction.

2. DECODE: The control unit interprets what those bytes mean. The first byte (or bits) is the opcode — it tells the CU what operation to perform. Subsequent bytes specify operands: registers, immediate values, or memory addresses.

3. EXECUTE: The ALU (or another functional unit like the FPU for floating point, or the load/store unit for memory) performs the operation. The result may be written to a register or back to memory.

4. WRITEBACK (sometimes listed separately): Results are committed to registers or memory.

This cycle repeats billions of times per second on a modern CPU. At 3 GHz, you get 3 × 10^9 cycles per second. Instructions typically take 1–5 cycles each, so modern CPUs execute hundreds of millions to billions of instructions per second.

The PC is the key: it normally advances sequentially (PC += instruction_size), which is why code executes in order. Branch instructions (if, while, function calls) change the PC to a different address, creating non-sequential execution. A function call saves the return address (the old PC + offset) so execution can resume after the function returns.

# RISC-V assembly: the fetch-decode-execute cycle is most visible
# in a clean ISA like RISC-V where all instructions are 4 bytes.
# This function computes sum = 1 + 2 + 3.
#
# Assemble with: riscv64-linux-gnu-gcc -O0 -S loop.c
# Or view online: https://godbolt.org (select RISC-V rv64gc gcc)

    .text
    .globl sum_three
sum_three:
    li   a0, 0          # a0 = 0  (FETCH: PC=0x0, opcode=li)
    addi a0, a0, 1      # a0 += 1 (FETCH: PC=0x4, opcode=addi)
    addi a0, a0, 2      # a0 += 2 (FETCH: PC=0x8)
    addi a0, a0, 3      # a0 += 3 (FETCH: PC=0xC)
    ret                 # return a0 to caller (PC = link register)
# Notice: each instruction is exactly 4 bytes, so PC increments by 4
# at each step. The 'ret' instruction is a special case: it sets
# PC = ra (return address register), jumping back to the caller.

In the von Neumann model, memory is a single, flat, addressable array of bytes. Every byte has a unique integer address starting at 0. The CPU reads and writes memory using these addresses.

This is worth internalizing deeply because it has enormous consequences:

• Code and data share the same space. Your program's instructions live at some addresses. Your variables live at other addresses. The CPU's safety mechanism (in modern systems, the MMU with page permissions) is what prevents you from accidentally jumping into a data buffer and executing garbage as code. When this protection fails, you get a code injection vulnerability.

• Addresses are just integers. A pointer in C is literally an integer holding a memory address. int *p = (int *)0x7fff5fbff4c0; is a perfectly valid C expression — it says 'treat the integer 0x7fff5fbff4c0 as the address of an int'.

• The address space can be larger than physical RAM. Modern CPUs use virtual memory — the address the CPU sees is not the physical address of the DRAM chip. The OS and MMU translate virtual to physical. This is how two programs can both have a variable at 'address 0x1000' without colliding. This translation is one of the most important functions of the Linux kernel.

• Endianness matters. A 32-bit integer occupies 4 consecutive bytes. On a little-endian machine (x86), the least significant byte is at the lowest address. On a big-endian machine (some network hardware, older SPARC), it is reversed. This matters enormously when you parse network packets, read binary files, or write kernel drivers.

#include <stdio.h>
#include <stdint.h>

int global_data = 42;

int main(void) {
    int local_var = 99;

    /* Code (text segment) and data share the address space.
       Print their addresses to see where each lives. */
    printf("main() (code)  lives at: %p\n", (void *)main);
    printf("global_data    lives at: %p\n", (void *)&global_data);
    printf("local_var      lives at: %p\n", (void *)&local_var);

    /* Endianness demo */
    uint32_t val = 0x01020304;
    uint8_t *bytes = (uint8_t *)&val;
    printf("0x01020304 in memory: %02x %02x %02x %02x\n",
           bytes[0], bytes[1], bytes[2], bytes[3]);
    /* On x86 (little-endian): 04 03 02 01
       On big-endian:           01 02 03 04 */
    return 0;
}
/* Run with: gcc -o mem mem.c && ./mem
   You'll see code is at a HIGH address (text segment),
   globals in the middle (data segment), locals on the stack (high). */

Here is the central tragedy of the von Neumann architecture: the CPU and memory communicate over a single shared bus. Instructions must be fetched over this bus. Data must be read and written over this bus. Both cannot happen simultaneously on a classic design.

The bus is the bottleneck. CPUs have gotten exponentially faster (Moore's Law drove clock speeds from ~1 MHz in 1975 to ~5 GHz today, a 5000× increase). But DRAM memory access latency has improved by only about 10–20×. The gap between CPU speed and memory speed has grown into a chasm.

The term "von Neumann bottleneck" was coined by computer scientist John Backus (the inventor of FORTRAN) in his 1977 Turing Award lecture. He described it as a "narrow pipeline" between the CPU and memory through which data and instructions must flow, creating a fundamental throughput limit.

In concrete numbers on a modern x86:

• A register-to-register ADD: ~0.25 ns (sub-nanosecond)
• An L1 cache hit: ~1 ns
• An L2 cache hit: ~5 ns
• An L3 cache hit: ~20 ns
• A DRAM access: ~60–100 ns
• An NVMe SSD read: ~100,000 ns (100 µs)

This is why the memory hierarchy (registers → L1 → L2 → L3 → RAM → disk) was invented: to paper over the bottleneck with increasingly large but slower caches closer to the CPU. And it is why cache-friendly code (sequential memory access, small working sets) is so dramatically faster than cache-unfriendly code.

From a kernel perspective: the entire Linux page cache, the slab allocator, NUMA-awareness, CPU affinity, and huge pages all exist, in part, to mitigate the von Neumann bottleneck.

/* Demonstrating the bottleneck: cache-friendly vs cache-unfriendly access.
   Compile: gcc -O2 -o cache_demo cache_demo.c && ./cache_demo
   The difference can be 10-100x on real hardware. */
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#define N 4096

static int matrix[N][N];

long row_major_sum(void) {
    long sum = 0;
    /* Row-major: accesses matrix[i][0], [i][1], [i][2]...
       Sequential in memory -> cache lines loaded once -> FAST */
    for (int i = 0; i < N; i++)
        for (int j = 0; j < N; j++)
            sum += matrix[i][j];
    return sum;
}

long col_major_sum(void) {
    long sum = 0;
    /* Column-major: accesses matrix[0][j], [1][j], [2][j]...
       Jumps N*4 bytes each step -> cache miss every access -> SLOW */
    for (int j = 0; j < N; j++)
        for (int i = 0; i < N; i++)
            sum += matrix[i][j];
    return sum;
}

int main(void) {
    struct timespec t0, t1;
    clock_gettime(CLOCK_MONOTONIC, &t0);
    long r = row_major_sum();
    clock_gettime(CLOCK_MONOTONIC, &t1);
    printf("Row-major:    %ld ms (sum=%ld)\n",
           (t1.tv_nsec - t0.tv_nsec) / 1000000 +
           (t1.tv_sec - t0.tv_sec) * 1000, r);
    clock_gettime(CLOCK_MONOTONIC, &t0);
    long c = col_major_sum();
    clock_gettime(CLOCK_MONOTONIC, &t1);
    printf("Column-major: %ld ms (sum=%ld)\n",
           (t1.tv_nsec - t0.tv_nsec) / 1000000 +
           (t1.tv_sec - t0.tv_sec) * 1000, c);
    return 0;
}

If memory is the bottleneck, the most obvious fix is to bring some memory inside the CPU itself, where it can be accessed in zero bus cycles. That is exactly what registers are.

Registers are tiny, extremely fast memory cells built into the CPU core. They are not on the memory bus at all — they are accessed by wires directly inside the chip, in a single clock cycle or less. A modern x86-64 CPU has:

• 16 general-purpose 64-bit registers (rax, rbx, rcx, rdx, rsi, rdi, rsp, rbp, r8–r15)
• 16 SSE/AVX registers for SIMD operations (xmm0–xmm15, up to 512-bit zmm registers with AVX-512)
• Special registers: rip (program counter), rflags (condition codes), segment registers

Total: perhaps a few kilobytes. That is thousands of times less than even a tiny cache, but it is right there inside the execution engine.

The ABI (Application Binary Interface) specifies which registers are used for function arguments, return values, and which must be preserved across calls. On Linux x86-64 (System V AMD64 ABI):

• Function arguments: rdi, rsi, rdx, rcx, r8, r9 (first 6 integer args)
• Return value: rax (and rdx for 128-bit)
• Caller-saved (may be clobbered): rax, rcx, rdx, rsi, rdi, r8–r11
• Callee-saved (must be preserved): rbx, rbp, r12–r15

The Linux kernel uses this ABI for syscalls too, with a slight difference: rax holds the syscall number, and the kernel uses rcx and r11 for its own purposes, so the 6th argument shifts to r9.

Understanding registers is essential for reading kernel oops crash dumps, writing inline assembly in kernel drivers, and understanding calling conventions.

/* See exactly how registers work via inline asm on x86-64.
   This performs the 'write' syscall directly, bypassing libc.
   Compile: gcc -O0 -o asm_write asm_write.c && ./asm_write */
#include <unistd.h>

int main(void) {
    const char msg[] = "hello from raw syscall\n";
    long ret;
    __asm__ volatile (
        "syscall"
        : "=a" (ret)          /* output: rax = return value */
        : "0" (1),            /* rax = 1 (SYS_write) */
          "D" (1),            /* rdi = 1 (stdout fd) */
          "S" (msg),          /* rsi = pointer to message */
          "d" (sizeof(msg)-1) /* rdx = length */
        : "rcx", "r11", "memory"
    );
    /* rax now holds bytes written (or negative errno) */
    return (int)ret < 0 ? 1 : 0;
}

The von Neumann bottleneck is partly caused by the fact that the CPU must share one bus for both fetching instructions and accessing data. A different design, the Harvard architecture, solves this by having physically separate memories and buses for instructions and data.

In a pure Harvard machine:

• The instruction memory (program ROM/flash) is on its own bus. The CPU can fetch the next instruction from here while simultaneously...
• ...reading or writing data memory (RAM) on its own separate bus.

This doubles the memory bandwidth available to the CPU with no other changes. Harvard architecture is dominant in:

• Microcontrollers: AVR (Arduino), PIC, ARM Cortex-M. These have separate flash (code) and SRAM (data) with independent buses. This is why an Arduino can fetch the next instruction and read a sensor simultaneously.
• DSPs (Digital Signal Processors): Need to fetch filter coefficients (data) and instructions at the same time.
• GPUs: Separate instruction caches and data caches.

However, pure Harvard has a critical limitation: you cannot generate code at runtime (no JIT), and you cannot store program data (look-up tables) in instruction memory easily. This inflexibility makes pure Harvard unsuitable for general-purpose computing.

Modern CPUs use a Modified Harvard architecture. At the L1 cache level, there are separate L1 instruction cache (L1i) and L1 data cache (L1d), giving the performance benefit of Harvard. But at the L2 level and above, and at the DRAM level, the caches are unified — the same physical memory and bus, like von Neumann. This means JIT compilation works: you write code bytes to data memory, then the CPU's cache coherence protocol ensures those bytes are visible to the instruction fetch pipeline.

You can observe this split cache on Linux:

# Observe the Modified Harvard architecture in your CPU's cache topology
# on Linux:
bash -c '
# L1 instruction cache size
echo "L1 instruction cache:"
cat /sys/devices/system/cpu/cpu0/cache/index1/type
cat /sys/devices/system/cpu/cpu0/cache/index1/size

# L1 data cache size  
echo "L1 data cache:"
cat /sys/devices/system/cpu/cpu0/cache/index0/type
cat /sys/devices/system/cpu/cpu0/cache/index0/size

# L2 (unified)
echo "L2 cache (unified):"
cat /sys/devices/system/cpu/cpu0/cache/index2/type
cat /sys/devices/system/cpu/cpu0/cache/index2/size
'
# Expected output on a typical x86:
# L1 instruction cache: Instruction / 32K
# L1 data cache:        Data        / 32K
# L2 cache (unified):   Unified     / 256K

The fact that instructions and data share the same memory is not just an architectural curiosity — it is the root cause of entire families of security vulnerabilities.

Buffer overflow attacks: If an attacker can write past the end of a stack buffer, they can overwrite the return address saved on the stack. The CPU treats this address as data (a number to put in the PC). But once 'ret' executes, the CPU fetches instructions from that address. If the attacker has written shellcode into that buffer, they have just redirected execution into data they control. This is the textbook code injection attack, first widely exploited in the 1980s.

Defensive mechanisms invented specifically because of this von Neumann property:

• NX bit (No-Execute) / XD bit (Execute Disable) / W^X (Write XOR Execute): The MMU marks memory pages as either writable OR executable, but not both. If the CPU tries to fetch an instruction from a page not marked executable, it raises a fault. Linux uses this: stack and heap pages are writable but not executable by default. This is enforced at the hardware level by the page table entry's NX bit (bit 63 on x86-64).
• ASLR (Address Space Layout Randomization): Randomizes where code, stack, and heap are placed in the address space, making it harder for an attacker to know where to redirect the PC.
• Stack canaries: The compiler inserts a random 'canary' value before the return address. Before executing 'ret', it checks if the canary has been overwritten.

The deep irony: every one of these mitigations exists because von Neumann made instructions and data indistinguishable — the very property that makes general-purpose computers universal.

/* Demonstrate that the kernel enforces NX on the stack.
   This WILL segfault on a modern Linux system — that's the point.
   Compile: gcc -z execstack -o nx_demo nx_demo.c  (bypasses NX for demo)
   Without -z execstack, the OS enforces NX and it crashes.
*/
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>

int main(void) {
    /* These bytes are the x86-64 encoding of:
       mov eax, 42; ret  */
    unsigned char code[] = { 0xb8, 0x2a, 0x00, 0x00, 0x00, 0xc3 };

    /* Try to make the stack region executable */
    /* Without -z execstack, mprotect on stack region returns EACCES
       on systems with strict NX enforcement. */
    typedef int (*fn_t)(void);
    fn_t fn = (fn_t)code;

    /* With -z execstack: prints 42
       Without it (default): SIGSEGV — kernel's NX protection fires */
    printf("result: %d\n", fn());
    return 0;
}
/* Check NX status of a binary:
   readelf -l ./nx_demo | grep GNU_STACK
   RWE means executable stack (bad); RW means NX enforced (good) */

The Program Counter (PC) — called RIP on x86-64, PC on ARM, also called the Instruction Pointer — is the register that makes sequential execution happen automatically. After each fetch, the CPU hardware increments the PC by the size of the instruction just fetched.

Control flow changes happen by modifying the PC:

• Unconditional jump (jmp, b): Set PC to a new address.
• Conditional branch (je, bne): Set PC to a new address only if a condition flag is set.
• Call (call, bl): Push the return address (current PC + instruction size) onto the stack, then set PC to the function's address.
• Return (ret, bx lr): Pop the return address from the stack, set PC to it.
• Interrupt/Trap/Syscall: Save PC (and CPU state) to a well-known location, then set PC to the interrupt handler address in the Interrupt Descriptor Table (IDT) on x86. This is how the Linux kernel is entered on every system call.

The PC being a register is significant: on x86-64 you can read it with lea rax, [rip]. On older x86 (32-bit), RIP-relative addressing did not exist, so position-independent code had to use a trick (call + pop) to discover where it was in memory.

Understanding the PC is essential for reading kernel stack traces. When the kernel panics and prints something like RIP: 0xffffffff81234567 do_page_fault+0x123/0x456, that address is the PC — exactly where the crash occurred, expressed as function + offset from function start.

/* Observe the program counter (RIP) directly on x86-64.
   Compile: gcc -O0 -o pc_demo pc_demo.c && ./pc_demo */
#include <stdio.h>

int main(void) {
    void *rip_here;
    /* RIP-relative LEA: load the address of the *next* instruction */
    __asm__ volatile ("lea (%%rip), %0" : "=r"(rip_here));
    printf("RIP right here: %p\n", rip_here);
    printf("main() starts at: %p\n", (void *)main);
    printf("Offset into main: %td bytes\n",
           (char *)rip_here - (char *)main);
    /* The offset tells you how far into the function you are —
       exactly the same format as a kernel oops stack trace. */
    return 0;
}

Von Neumann's architecture includes I/O as a first-class component, but the mechanisms for doing I/O were not fully specified in the original model and have evolved considerably.

There are two fundamental approaches to how the CPU communicates with I/O devices:

1. Memory-Mapped I/O (MMIO): Device registers are mapped into the normal address space. Writing to address 0xFED00000 might configure an APIC timer. Reading from 0xB8000 reads the VGA framebuffer. The CPU uses ordinary MOV instructions. No special I/O instructions needed. This is the dominant model in modern systems and is used for almost all PCI/PCIe devices. The Linux kernel deals with MMIO extensively — ioremap() maps device physical addresses into kernel virtual addresses.

2. Port-Mapped I/O (PMIO): x86 has a separate 16-bit I/O address space (64K ports) accessed with special IN and OUT instructions. The 0x80 port is the classic 'POST code' debug port. 0x3F8 is COM1 serial. Modern hardware rarely uses this, but it still exists for legacy reasons, and the Linux kernel uses it for old devices (8259 PIC, PS/2 controller).

DMA (Direct Memory Access): Transferring data one byte at a time through the CPU is wasteful. DMA controllers allow devices to copy data directly to/from RAM, bypassing the CPU entirely. The CPU programs the DMA controller (source address, destination address, length), starts the transfer, and gets an interrupt when it's done. Nearly all high-throughput devices (NIC, NVMe, SATA, GPU) use DMA. In the Linux kernel, the DMA API (dma_alloc_coherent, dma_map_single) manages the complexity of physical vs virtual addresses, cache coherency, and IOMMU protection.

This matters for kernel development: writing a device driver means deciding whether to use MMIO or PMIO, setting up DMA buffers correctly, and handling interrupts — all of which are direct consequences of the von Neumann I/O model.

/* Port-mapped I/O on x86 Linux — reading the real-time clock
   via the CMOS I/O ports (0x70/0x71).
   Must run as root or with ioperm: sudo ./cmos_read
   Compile: gcc -O0 -o cmos_read cmos_read.c */
#include <stdio.h>
#include <sys/io.h>

#define CMOS_ADDR 0x70
#define CMOS_DATA 0x71

unsigned char cmos_read(unsigned char reg) {
    outb(reg, CMOS_ADDR);   /* OUT instruction: write register index */
    return inb(CMOS_DATA);  /* IN  instruction: read register value  */
}

int main(void) {
    if (ioperm(CMOS_ADDR, 2, 1)) {
        perror("ioperm (need root)");
        return 1;
    }
    unsigned char sec  = cmos_read(0x00); /* seconds (BCD) */
    unsigned char min  = cmos_read(0x02); /* minutes (BCD) */
    unsigned char hour = cmos_read(0x04); /* hours   (BCD) */
    /* BCD decode: tens digit in high nibble, units in low */
    printf("CMOS time: %02d:%02d:%02d\n",
           (hour>>4)*10+(hour&0xF),
           (min >>4)*10+(min &0xF),
           (sec >>4)*10+(sec &0xF));
    return 0;
}