CPU Anatomy: ALU, Registers, Control Unit, Datapath

A CPU core executes a loop that has never changed in 75 years of computing history:

1. Fetch an instruction from memory.
2. Decode it — figure out what operation it is.
3. Execute it — actually do the arithmetic or memory access.
4. Write back the result.
5. Repeat, forever.

This is called the fetch-decode-execute cycle (or instruction cycle). Everything inside a CPU core — every component we will study — exists to make this loop happen correctly and quickly.

Think of a CPU core as a very fast, very obedient worker at an assembly line. The worker has a small notepad (registers) to jot down intermediate results, a calculator (ALU) to do math, a supervisor (control unit) who reads instructions and tells everyone what to do, and a conveyor belt (datapath) that moves data between all the parts.

The key insight: a CPU has no creativity. It can only do exactly what an instruction says, one step at a time (or in a pipeline, several steps for different instructions simultaneously). All software complexity — your operating system, your browser — is an illusion built from billions of these tiny, dumb steps executed extremely fast.

# Watch the fetch-decode-execute cycle in slow motion:
# Compile a tiny C program and trace the instructions it generates.

# File: add.c
# int main() { return 1 + 2; }

$ gcc -O0 -S -o add.s add.c   # emit x86-64 assembly
$ cat add.s
# You will see: movl $1, %eax   (fetch: load 1 into register eax)
#               addl $2, %eax   (execute: ALU adds 2 to eax)
#               ret             (write back: eax holds return value 3)

# Each line of assembly = one trip around the fetch-decode-execute loop.

A register is a tiny, named storage location built directly inside the CPU core from flip-flop circuits. Unlike RAM, which is a separate chip accessed over a bus, registers are on-die — they are nanoseconds away from the ALU, not tens of nanoseconds like DRAM.

Registers are precious. A typical CPU has only 16–32 general-purpose integer registers, a handful of floating-point/SIMD registers, and a few special-purpose ones. Why so few? Because each register costs silicon area and power. Making 1024 registers would make the chip slow and hot.

The register file is the hardware array of all general-purpose registers plus the read/write ports that let the ALU access them in a single clock cycle. In a simple design, the register file has two read ports (so the ALU can read operand A and operand B simultaneously) and one write port (to store the result).

Special-purpose registers you must know:

• Program Counter (PC) — holds the memory address of the next instruction to fetch. Also called the Instruction Pointer (IP) on x86. It is incremented automatically after each fetch, and changed explicitly by branch/jump instructions.
• Instruction Register (IR) — holds the currently executing instruction's binary encoding, while the control unit is decoding it.
• Stack Pointer (SP) — points to the top of the call stack.
• Status/Flags Register — holds condition bits set by the ALU: Zero (Z), Carry (C), Negative (N), Overflow (V). Conditional branches read these flags.

On x86-64, the general-purpose registers are named rax, rbx, rcx, rdx, rsi, rdi, rsp, rbp, and r8–r15. On ARM64 (AArch64), they are x0–x30 plus sp. On RISC-V, they are x0–x31.

A crucial rule in most ABIs: some registers are caller-saved (you can trash them freely, but must save them yourself before calling a function), and others are callee-saved (a function you call promises to restore them). The Linux kernel's calling convention is defined in the ABI for each architecture.

// Seeing registers in C: compiler maps variables to registers
// File: regs.c
int add(int a, int b) {
    int c = a + b;  // compiler likely uses registers only
    return c;
}

// Compile and inspect the assembly:
// $ gcc -O1 -S -o regs.s regs.c && cat regs.s
//
// x86-64 result (approximately):
// add:
//     leal (%rdi,%rsi), %eax   ; eax = edi + esi  (ALU add)
//     ret                       ; return value is in eax
//
// 'a' is in %edi, 'b' is in %esi, result in %eax
// No memory touched at all — pure register arithmetic!

// ARM64 result (approximately):
// add:
//     add  w0, w0, w1   ; w0 = w0 + w1
//     ret

The ALU is a combinational logic circuit — meaning it has no memory or state of its own, just wires and logic gates. You feed it two inputs and a control signal that says which operation to perform, and it outputs a result immediately (within the propagation delay of the gates, typically a fraction of a nanosecond).

An ALU can typically perform:

• Integer arithmetic: ADD, SUB, MUL (sometimes), DIV (often not — too slow, done separately)
• Bitwise logic: AND, OR, XOR, NOT
• Shift operations: left shift (×2), right shift (÷2), arithmetic right shift (preserving sign)
• Comparison: subtract two numbers and set flags based on the result (Zero flag if equal, Carry/Overflow for unsigned/signed comparison)

The ALU does not handle floating-point numbers — that is the job of the FPU (Floating-Point Unit), a separate but related circuit. Modern CPUs also have SIMD ALUs (Single Instruction Multiple Data) that operate on 128, 256, or 512-bit vectors in a single cycle.

Internally, a 1-bit full adder is made from two XOR gates, two AND gates, and one OR gate. A 64-bit adder is a cascade of 64 such full adders (with carry lookahead optimization to make it fast). The entire ALU is built from these primitives — it is logic gates all the way down.

The flags register is set as a side effect of ALU operations. For example, SUB rax, rbx computes rax - rbx, writes the result to rax, and sets:

• ZF (Zero Flag): 1 if the result is 0 (meaning rax == rbx before the subtract)
• SF (Sign Flag): 1 if the result is negative
• CF (Carry Flag): 1 if there was an unsigned overflow
• OF (Overflow Flag): 1 if there was a signed overflow

This is how if (a == b) works in hardware: compile to CMP a, b (which is just SUB that discards the result but keeps the flags), then JE (Jump if Equal) which jumps only if ZF is set.

// Seeing the ALU flags in action: conditional branching
// File: compare.c
int max(int a, int b) {
    if (a > b) return a;
    return b;
}

// x86-64 assembly (gcc -O1):
// max:
//     cmpl  %esi, %edi     ; ALU: compute edi - esi, set flags, discard result
//     jle   .return_b      ; if SF!=OF or ZF=1 (i.e., a <= b), jump
//     movl  %edi, %eax     ; return a
//     ret
// .return_b:
//     movl  %esi, %eax     ; return b
//     ret
//
// The entire 'if' statement is just:
//   one ALU operation (CMP) that sets the flags register
//   one conditional jump that reads the flags register

// Verify on Godbolt: https://godbolt.org  — paste the C, choose x86-64 gcc

The control unit (CU) is the part of the CPU that reads each instruction's binary encoding from the Instruction Register, figures out what it means, and then generates the control signals that tell every other component what to do during that clock cycle.

Think of the control unit as the conductor of an orchestra. It does not play any instruments itself — instead, it reads the score (the instruction encoding) and signals each musician (ALU, register file, memory interface) when to play and what to play.

For example, when the control unit sees the binary encoding 0x48 0x01 0xF8 (which is ADD %rdi, %rax in x86-64), it must simultaneously assert:

• Register file read address A = 0 (register 0 = rax)
• Register file read address B = 7 (register 7 = rdi)
• ALU operation select = ADD
• Register file write address = 0 (write result back to rax)
• Register file write enable = 1
• Memory write enable = 0 (we are not writing to memory)

All these signals are asserted within the same clock cycle.

How is the control unit implemented?

Two classical approaches:

1. Hardwired control: Pure combinational logic — a big boolean function from instruction bits to control signals. Very fast, but hard to change.
2. Microprogrammed control: The CU contains a small ROM (the microcode store) where each row is a set of control signals for one micro-operation. A complex instruction like DIV is decoded into a sequence of micro-ops fetched from this ROM. Intel x86 CPUs use microcode — it is why you can fix CPU bugs with a microcode update loaded at boot.

Modern out-of-order CPUs have a much more complex control flow — the front-end decodes x86 macro-instructions into RISC-like micro-ops which are then fed into a scheduler. But the core idea — read instruction, emit control signals — is unchanged.

// You can see 'microcode updates' loaded by the Linux kernel at boot:
// (run on any x86 Linux system)
$ dmesg | grep -i microcode
// Example output:
// [    0.000000] microcode: microcode updated early to revision 0xf4
// [    0.456789] microcode: sig=0x806ec, pf=0x22, revision=0xf4

// The kernel loads /lib/firmware/intel-ucode/ or /lib/firmware/amd-ucode/
// before userspace starts. This patches the control unit's microcode ROM
// to fix hardware bugs (like Spectre variants) without replacing the chip.

// To see which microcode your CPU has loaded:
$ cat /proc/cpuinfo | grep microcode

These two registers are the engine of the fetch-decode-execute cycle.

The Program Counter (PC) — called the Instruction Pointer (rip) on x86-64, pc on ARM, pc on RISC-V — holds the memory address of the next instruction to be fetched. It is the CPU's finger on the page — it always points to where we are in the program.

At the start of the fetch phase, the memory system fetches the bytes at the address in the PC. After a successful fetch:

• For a fixed-width ISA (ARM64, RISC-V): PC is automatically incremented by the instruction width (4 bytes). Simple.
• For a variable-width ISA (x86): the CPU must decode enough of the instruction to know its length, then increment the PC by that length (1 to 15 bytes on x86). This is one reason x86 decode is expensive.

Jump/branch instructions work by writing a new value into the PC. JMP 0x400500 simply stores 0x400500 into the PC register. CALL additionally pushes the return address (current PC + instruction size) onto the stack. RET pops that address back into the PC.

The Instruction Register (IR) holds the raw binary bytes of the instruction currently being decoded and executed. Once the instruction finishes, the IR is overwritten with the next fetched instruction. You can think of the IR as the 'current work order' sitting on the control unit's desk.

Why does this matter for the kernel? The kernel saves and restores the PC (and all other registers) on every context switch — that is how it can pause one process and resume another. The saved PC is what the Linux task_struct calls thread.ip (x86). When the scheduler restores that task, it loads the saved PC back into rip and the process continues exactly where it left off — it never knew it was paused.

// Inspect the saved program counter in a Linux process
// (uses ptrace, which is how debuggers like gdb work)

#include <sys/ptrace.h>
#include <sys/user.h>   // struct user_regs_struct
#include <sys/wait.h>
#include <stdio.h>
#include <unistd.h>

int main() {
    pid_t child = fork();
    if (child == 0) {
        // Child: allow parent to trace us
        ptrace(PTRACE_TRACEME, 0, NULL, NULL);
        raise(SIGSTOP);          // stop so parent can read our registers
        return 42;
    }
    // Parent: wait for child to stop, read its registers
    int status;
    waitpid(child, &status, 0);

    struct user_regs_struct regs;
    ptrace(PTRACE_GETREGS, child, NULL, &regs);
    printf("Child RIP (program counter) = 0x%llx\n", regs.rip);
    printf("Child RSP (stack pointer)   = 0x%llx\n", regs.rsp);

    ptrace(PTRACE_CONT, child, NULL, NULL);  // resume child
    waitpid(child, &status, 0);
    return 0;
}
// Compile: gcc -o pcread pcread.c && ./pcread

The datapath is the collection of all the functional units (ALU, register file, PC, IR, memory interfaces) plus the buses and multiplexers that connect them. It is the physical stage on which the instruction cycle plays out.

A bus is just a bundle of wires that carries multi-bit data. A multiplexer (mux) is a circuit that selects one of several inputs to pass through to an output, controlled by a select signal from the control unit.

Here is how data flows through the datapath for a typical register-register ADD instruction (like RISC-V add x3, x1, x2):

1. Fetch: PC drives the address bus → instruction memory returns the 32-bit instruction word → IR is loaded.
2. Decode: Control unit reads IR, emits: read reg x1, read reg x2, ALU op = ADD, write reg x3.
3. Read registers: Register file outputs the values in x1 and x2 onto two data buses feeding the ALU inputs.
4. Execute: ALU receives x1 and x2, computes x1+x2, outputs result and flags.
5. Write back: The ALU result is driven back to the register file write port; x3 is updated.
6. Increment PC: PC = PC + 4 (RISC-V has fixed 32-bit instructions).

For a load instruction (lw x3, 0(x1)) the datapath is slightly different: after the ALU computes the effective address (x1 + 0), that address goes to the data memory instead of writing back to a register. The data memory returns the loaded value, which then goes to the register file.

The distinction between the instruction bus/path and the data bus/path is the Harvard architecture (separate instruction and data memories) vs von Neumann (unified memory). Most real CPUs are von Neumann at the DRAM level, but Harvard at the L1 cache level (split instruction cache / data cache) — a hybrid that gives speed without the complexity of separate memory chips.

// RISC-V assembly showing the datapath flow explicitly
// File: datapath_demo.s  (RISC-V 32-bit)

    .text
    .global _start
_start:
    li   x1, 10        # load immediate 10 into x1  (ALU: x1 = 0 + 10)
    li   x2, 32        # load immediate 32 into x2
    add  x3, x1, x2   # x3 = x1 + x2  (pure register-register: datapath as described above)
    # x3 now holds 42

    # Memory datapath (load/store):
    # Assume sp (x2) points to a valid stack area:
    sw   x3, 0(sp)     # store x3 to memory[sp+0]  (ALU computes address, memory write)
    lw   x4, 0(sp)     # load from memory[sp+0] into x4  (ALU computes address, memory read)
    # x4 should now equal x3 = 42

# To run this on Linux x86 host with RISC-V toolchain:
# riscv64-linux-gnu-gcc -nostdlib -o demo datapath_demo.s
# qemu-riscv64 ./demo

Every synchronous digital circuit — and all modern CPUs are synchronous — is governed by a clock signal: a square wave that alternates between 0 and 1 at a fixed frequency. The clock is generated by a crystal oscillator (typically 100 MHz for a base reference clock) and then multiplied up by a Phase-Locked Loop (PLL) inside the CPU to the final frequency (e.g., 4 GHz = 4,000,000,000 cycles per second).

Why do we need a clock? Because logic gates are not instantaneous — signals take time to propagate through wires and transistors. The clock guarantees a synchronization point: at the rising edge of each clock pulse, all flip-flops (registers, latches) capture their input values simultaneously. Between clock edges, combinational logic (like the ALU) is free to churn through its computation. As long as the computation finishes before the next clock edge, the result is valid. The maximum speed you can clock a circuit is determined by the critical path — the longest chain of logic gates that any signal must traverse between two register stages.

Clock cycle time vs. CPI vs. IPC:

• Clock period = 1 / frequency (e.g., 4 GHz → 0.25 ns per cycle)
• CPI (Cycles Per Instruction): average cycles needed for one instruction. A simple RISC pipeline might achieve CPI close to 1. A complex memory access might take 200 cycles (cache miss to DRAM).
• IPC (Instructions Per Cycle) = 1/CPI. Modern out-of-order CPUs can sustain IPC > 4 by executing multiple instructions simultaneously.

CPU performance = (Instructions / Program) × (Cycles / Instruction) × (Seconds / Cycle). This is the CPU Performance Equation and it is the foundation of all CPU optimization thinking.

What stops us from just running at 100 GHz? Two things: (1) power dissipation — power scales as frequency squared (P ∝ C·V²·f), so doubling frequency quadruples heat; (2) signal integrity — at very high frequencies, wires on-chip start to behave as transmission lines with reflections and crosstalk. This is why the industry switched from frequency scaling to multicore designs around 2004.

// Read the current CPU frequency on Linux (shows real-time scaling)
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq
// Outputs something like: 3600000  (= 3.6 GHz, in kHz)

// See all available frequencies and the current governor:
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
// Common governors: 'performance' (max always), 'powersave' (min always),
// 'schedutil' (Linux kernel scheduler sets it based on load)

// The kernel's cpufreq subsystem is the code that talks to the PLL
// and adjusts the multiplier at runtime.
// See: drivers/cpufreq/ in the kernel source tree

An instruction is ultimately just a pattern of bits. The Instruction Set Architecture (ISA) defines exactly which bit pattern means which operation. The control unit's job is to read those bits and emit the right control signals.

There are two broad families:

RISC (Reduced Instruction Set Computer): Fixed-width instructions (32 bits in ARM64, RISC-V; 16 or 32 bits in Thumb/RV-C compressed). Simple, regular encoding. Each instruction does one small thing. Examples: ARM, RISC-V, MIPS, PowerPC.

CISC (Complex Instruction Set Computer): Variable-width instructions (1–15 bytes for x86-64). Many addressing modes. Closer to high-level operations (e.g., REP MOVS copies a block of memory). The x86 family is the canonical example.

Let us decode a real RISC-V instruction to see how bit fields map to control signals:

add x3, x1, x2    (add register x1 and x2, write to x3)

Binary encoding (32 bits):
31       25 24   20 19  15 14  12 11    7 6       0
[0000000] [00010] [00001] [000] [00011] [0110011]
  funct7    rs2     rs1   funct3   rd    opcode

• opcode (bits 6:0) = 0110011 → R-type register operation
• rd (bits 11:7) = 00011 → destination is x3
• funct3 (bits 14:12) = 000 → ADD (not SUB, XOR, etc.)
• rs1 (bits 19:15) = 00001 → source 1 is x1
• rs2 (bits 24:20) = 00010 → source 2 is x2
• funct7 (bits 31:25) = 0000000 → ADD (not SUB which would be 0100000)

The control unit is essentially a lookup table that takes these fields as input and drives the register file addresses and ALU operation select accordingly. The regularity of RISC encoding makes this straightforward — the rs1, rs2, and rd fields are always in the same bit positions in every R-type instruction, so the register file can start reading them before the opcode is fully decoded. This overlapping is part of why RISC pipelines are fast.

// Inspect raw instruction encodings with objdump
// File: encode_demo.c
int add(int a, int b) { return a + b; }

// Compile for RISC-V:
// $ riscv64-linux-gnu-gcc -O1 -c encode_demo.c -o encode_demo.o
// $ riscv64-linux-gnu-objdump -d encode_demo.o
//
// Output:
// 0000000000000000 <add>:
//    0: 00b50533   add   a0, a0, a1
//
// The hex 0x00b50533 in binary is:
// 0000 0000 1011 0101 0000 0101 0011 0011
// Let's parse it:
// funct7 = 0000000
// rs2    = 01011  (= 11 decimal = a1 = x11)
// rs1    = 01010  (= 10 decimal = a0 = x10)
// funct3 = 000
// rd     = 01010  (= 10 decimal = a0 = x10)
// opcode = 0110011  (R-type arithmetic)
// Matches exactly the RISC-V spec!

The same ISA can be implemented in very different datapath organizations. Understanding the tradeoffs is crucial — pipelines are the dominant technique in every modern CPU.

Single-Cycle Datapath: The entire fetch-decode-execute cycle for one instruction completes in a single clock cycle. The clock period must be long enough for the slowest instruction (usually a load, which requires ALU + memory access). Fast instructions are wasted waiting. Simple to design, used in textbooks and educational CPUs (like the one in Patterson & Hennessy Chapter 4). Terrible performance in practice.

Multi-Cycle Datapath: Break the instruction cycle into multiple steps, each taking one clock cycle. Different instructions use different numbers of cycles. Slightly better hardware utilization, but still only one instruction in flight at a time.

Pipelined Datapath: The revolutionary idea. Like an assembly line in a car factory — while car #5 is being welded, car #4 is being painted, car #3 is having its engine installed, all simultaneously. In a classic 5-stage pipeline (RISC-V, MIPS):

In steady state, five different instructions are in five different stages simultaneously. Throughput approaches 1 instruction per cycle (IPC ≈ 1). The catch: pipeline hazards — data hazards (instruction needs a result not yet written back), control hazards (branch changes the PC before we know which way to go), structural hazards (two stages need the same hardware unit).

Solving hazards requires forwarding (bypass wires from EX/MEM outputs back to EX inputs), stalls (pause the pipeline by inserting 'bubble' NOP instructions), and branch prediction (guess the branch outcome and speculatively execute; flush and retry if wrong).

Modern CPUs like Intel's Alder Lake have 15-20 pipeline stages and execute 4-6 instructions per cycle out-of-order. But the conceptual foundation is still this 5-stage pipeline.

// Observe pipeline stalls (data hazards) in real assembly
// A load-use hazard: instruction after a load needs the loaded value

// RISC-V:
//   lw  x1, 0(x2)    // load from memory into x1  (takes multiple cycles)
//   add x3, x1, x4   // IMMEDIATELY uses x1 → pipeline stall (1 cycle)

// The assembler/compiler inserts a NOP or reorders instructions to avoid this:
// $ riscv64-linux-gnu-gcc -O1 -S hazard_demo.c
// GCC will often reorder instructions to fill the load delay slot.

// On x86, you can see pipeline effects via performance counters:
$ perf stat -e cycles,instructions,branch-misses ./your_program
// Output includes IPC = instructions/cycles
// A low IPC (e.g., 0.5) indicates many stalls or cache misses
// A high IPC (e.g., 3.8) means the CPU is executing efficiently

The kernel is not magical — it is software that runs on this exact hardware and manipulates these exact components. Here are the direct connections:

Context Switching (register file): When the kernel preempts a process, __switch_to() (in arch/x86/kernel/process_64.c for x86) saves all general-purpose registers, the PC (rip), the stack pointer (rsp), the flags register, and floating-point state into the task_struct. When the scheduled-in task runs, those values are loaded back. The CPU has no idea it was paused — from its perspective, execution just continued.

System Calls (special registers + control flow): The syscall instruction on x86-64 does something the control unit handles specially: it saves the user-space rip and rflags into MSRs (Model-Specific Registers), loads the kernel's entry-point address into rip from another MSR (LSTAR), and switches the privilege mode bit. The kernel's entry_SYSCALL_64 in arch/x86/entry/entry_64.S then saves the remaining registers onto the kernel stack.

kprobes (PC manipulation): The kprobes subsystem can insert a breakpoint instruction (INT3 on x86, which is a single byte 0xCC) at any kernel address by patching the instruction bytes at runtime. When the CPU's control unit fetches that byte, it triggers a debug trap. The trap handler reads the saved rip to know which probe fired, calls the registered probe handler, then restores the original instruction and resumes. This is live kernel tracing by direct manipulation of the instruction register flow.

CPU Feature Detection (CPUID instruction): Before the kernel enables features like AVX-512 (wider SIMD ALU), it queries the CPU using the CPUID instruction, which causes the control unit to read feature flags from internal ROM and return them in registers. See arch/x86/kernel/cpu/common.c.

// From Linux kernel arch/x86/entry/entry_64.S (simplified concept)
// This is real kernel assembly that runs on every system call:

// ENTRY(entry_SYSCALL_64):
//   swapgs                    // switch GS register to kernel's per-CPU area
//   movq %rsp, %gs:cpu_tss+TSS_sp2  // save user stack pointer
//   movq %gs:cpu_tss+TSS_sp1, %rsp  // load kernel stack pointer
//
//   pushq  $__USER_DS         // save user data segment
//   pushq  %gs:cpu_tss+TSS_sp2 // save user rsp
//   pushq  %r11               // save user rflags (saved by syscall insn)
//   pushq  $__USER_CS
//   pushq  %rcx               // save user rip  (saved by syscall insn into rcx)
//   pushq  %rax               // save syscall number
//   PUSH_AND_CLEAR_REGS       // save all other general-purpose registers
//   ...
//   call   do_syscall_64      // dispatch to C handler

// The register-saving here IS the 'register file save' from our CPU anatomy.
// To read the actual file:
// $ find linux-source -name 'entry_64.S' | head -1