What Is an OS? Kernel vs Userspace

Imagine ten people trying to write on the same sheet of paper simultaneously, with no rules. That is what happens when multiple programs run on bare hardware with no supervisor. Program A writes to memory address 0x1000; Program B also writes to 0x1000. Program A's data is destroyed. Or worse: Program B reads Program A's password out of memory.

The hardware itself offers no protection. A CPU instruction like MOV [0x1000], AX will happily overwrite anything at that address — your data, another program's code, the hardware registers that control a disk — with zero warnings. The CPU does not care. It just executes instructions.

The operating system exists to solve exactly this problem. It acts as both a resource manager (deciding who gets access to what hardware, when) and an abstraction layer (presenting each program with a clean, isolated view of a machine it appears to own entirely). These two roles are complementary: to present an abstraction you need to manage the resource, and vice versa.

The analogy that works best: the OS is like a bank. Your money (CPU time, RAM pages, disk sectors) is pooled with everyone else's in a vault (the hardware). The bank (OS) keeps careful ledgers, enforces rules, and gives you a clean interface (your account) that hides the messy reality underneath. You never touch the vault directly — you go through the bank.

The OS's authority over programs would be meaningless if programs could simply ignore it. A program could just overwrite the OS's memory, or execute privileged instructions directly. The OS would be advisory at best.

This is why CPU architects gave the hardware itself an enforcement mechanism: privilege levels, commonly called rings on x86 or exception levels on ARM.

On x86, the CPU has four rings (0–3), but in practice only two are used: Ring 0 (kernel mode, maximum privilege) and Ring 3 (user mode, restricted). The current privilege level is tracked in the CPU's CS (code segment) register — specifically in its low two bits, called the CPL (Current Privilege Level).

In Ring 0, code can do anything: read and write any physical memory address, execute privileged instructions like HLT (halt the CPU), IN/OUT (talk to hardware ports), LGDT (load the descriptor table that defines memory layout), LIDT (load the interrupt descriptor table), and so on.

In Ring 3, those instructions are illegal. If user code attempts HLT or a direct I/O port access, the CPU does not just ignore it — it raises a General Protection Fault (exception 13 on x86), which transfers control to the kernel's exception handler. The kernel then typically terminates the misbehaving process with a segmentation fault signal.

On ARM (AArch64), the privilege levels are called Exception Levels: EL0 (userspace), EL1 (OS kernel), EL2 (hypervisor), EL3 (firmware/secure monitor). On RISC-V, they are U-mode (user), S-mode (supervisor/kernel), and M-mode (machine/firmware). The naming differs, the concept is universal: hardware-enforced privilege tiers.

This is the single most important architectural fact for kernel programming: the kernel's authority is not a software convention — it is enforced by the CPU itself at the silicon level. A user program literally cannot execute privileged instructions, regardless of what code it contains.

/* Attempting a privileged instruction from user space in C.
 * Inline asm: the 'hlt' instruction is Ring-0 only.
 * Compile: gcc -O0 -o try_hlt try_hlt.c
 * Run:     ./try_hlt
 * Result:  Killed  (SIGSEGV or SIGILL from the kernel) */
#include <stdio.h>
int main(void) {
    printf("About to execute HLT from Ring 3...\n");
    __asm__ volatile("hlt");  /* General Protection Fault -> kernel kills us */
    printf("(you will never see this)\n");
    return 0;
}

Given the dual-mode hardware, a natural architecture emerges: put the OS code in Ring 0 and everything else in Ring 3. This is the kernel/userspace split.

Kernel space is where the kernel image lives — the code, data structures, and stacks that the OS uses to manage the system. In a typical 64-bit Linux system, the upper portion of every process's virtual address space (from 0xffff800000000000 upward on x86-64) is permanently mapped to kernel memory. The CPU, while in user mode, cannot read or write those addresses — the page table entries that map them have the User/Supervisor bit cleared, so any access raises a page fault that the kernel handles as a protection violation.

User space is everything else: your shell, your browser, your Python interpreter, the ls command. These all run in Ring 3 with restricted access. They have their own virtual address spaces, isolated from each other and from the kernel.

An important subtlety: the kernel is not a separate process. It is better thought of as a mode of the CPU. When a user program makes a system call, the CPU does not context-switch to a different process — it changes its privilege level to Ring 0 and jumps to a kernel entry point. The kernel then runs on behalf of the user process, using the kernel's own stack and memory mappings, before returning to Ring 3 and resuming the user program. This is the in-kernel execution context model that most monolithic kernels like Linux use.

The split serves three goals simultaneously:

1. Safety: a bug in user code cannot corrupt kernel data structures.
2. Security: a malicious program cannot read another process's memory or forge kernel operations.
3. Abstraction: user programs can be written without knowing anything about the specific hardware they run on.

/* Read /proc/self/maps to see your own address space split.
 * The kernel-mapped region (vsyscall, vvar, vdso) appears near the top.
 * The user stack, heap, and code segments appear at lower addresses. */
#include <stdio.h>
int main(void) {
    FILE *f = fopen("/proc/self/maps", "r");
    char line[256];
    while (fgets(line, sizeof(line), f))
        printf("%s", line);
    fclose(f);
    return 0;
}

If user programs cannot directly access hardware or kernel data, how do they do anything useful? How does printf() write to the terminal? How does open() open a file? The answer is the system call — the single, carefully controlled gate through the Ring 3 / Ring 0 boundary.

A system call works like this:

1. The user program places a syscall number in a designated CPU register (on x86-64: rax), and arguments in rdi, rsi, rdx, r10, r8, r9.
2. The program executes the SYSCALL instruction (x86-64) or SVC #0 (AArch64) or ECALL (RISC-V). This is the only instruction that voluntarily raises privilege level — and it does so in a completely controlled way.
3. The CPU atomically: saves the return address in rcx, saves the current flags in r11, switches to Ring 0, and jumps to the address stored in the LSTAR MSR (a kernel-configured register). That address is the kernel's syscall entry point (entry_SYSCALL_64 in Linux).
4. The kernel validates the syscall number, validates arguments (crucially, it must not blindly trust pointers from user space), performs the requested operation, and places the return value in rax.
5. The SYSRET instruction drops back to Ring 3 and resumes the user program.

The SYSCALL/SYSRET pair is a fast-path mechanism. The older INT 0x80 / IRET path (still present for 32-bit compatibility) works similarly but with more overhead.

This design is beautiful: user code can request kernel services but cannot execute them directly. The kernel is always in control of what happens at Ring 0. No matter what code a user process contains, it cannot skip the validation step — the CPU enforces the ring transition.

In C, you almost never write a raw syscall instruction. The C standard library (glibc or musl) wraps every syscall in a C function. write(fd, buf, n) in glibc expands to roughly: load rax=1 (the Linux write syscall number), load arguments, execute SYSCALL. But you can invoke syscalls directly using the syscall() libc wrapper or raw inline assembly, which is illuminating.

/* Write "hello\n" using a raw Linux syscall — no libc write().
 * x86-64 Linux: syscall 1 = write, syscall 60 = exit.
 * Compile: gcc -nostdlib -static -o raw_syscall raw_syscall.c
 * (or use nasm/gas for the asm version below) */
#include <unistd.h>   /* for the syscall() wrapper */
#include <sys/syscall.h>

void _start(void) {          /* no libc: entry is _start, not main */
    const char msg[] = "hello from raw syscall\n";
    /* SYS_write = 1 on x86-64 */
    syscall(SYS_write, 1, msg, sizeof(msg) - 1);
    syscall(SYS_exit, 0);
}

A modern OS is not just a gatekeeper — it provides a rich set of abstractions that hide the gory details of hardware management behind clean interfaces. These fall into six broad categories:

1. Process Management. The OS gives each running program the illusion of having the CPU to itself. It does this by rapidly multiplexing the CPU among processes (the scheduler), saving and restoring CPU register state (the context switch), and providing isolation so one process cannot directly interfere with another. System calls like fork(), exec(), wait(), and kill() are the user-facing interface to process management.

2. Memory Management. Every process gets its own private virtual address space — a contiguous, isolated range of addresses — even though physically RAM is shared. The OS configures page tables (hardware data structures that map virtual addresses to physical memory frames) and uses the CPU's Memory Management Unit (MMU) to enforce isolation. It also handles demand paging (only loading memory pages from disk when they are actually accessed) and copy-on-write semantics for fork().

3. File Systems. The OS abstracts storage devices (hard drives, SSDs, NVMe, USB sticks) into a unified hierarchical file system with named files and directories. open(), read(), write(), close() — these syscalls work identically whether the underlying storage is an NVMe SSD, a network file share, or a virtual filesystem in RAM (tmpfs). The Virtual File System (VFS) layer in the kernel is the abstraction that makes this work.

4. Device I/O. Hardware devices (network cards, GPUs, keyboards, USB peripherals) each speak their own protocol. The OS provides device drivers — kernel modules that translate the generic OS I/O model into device-specific commands. From user space, you read from a terminal, a file, a socket, or a pipe using the same read() call — the VFS routes it to the appropriate driver.

5. Inter-Process Communication (IPC). Processes need to cooperate: a web server spawns worker processes, a shell connects a pipe between ls and grep, a GUI toolkit communicates with a compositor. The OS provides pipes, FIFOs, shared memory, message queues, Unix domain sockets, and signals as IPC mechanisms.

6. Security and Protection. The OS enforces the concept of users and permissions. Each process runs as a user with a UID and GID. Each file has an owner and permission bits. The kernel checks permissions on every access. More sophisticated mechanisms — capabilities, namespaces, cgroups, seccomp filters, SELinux/AppArmor — build on this foundation.

All six categories share a common theme: the OS turns raw, dangerous, heterogeneous hardware into clean, safe, uniform abstractions. That translation is the fundamental value proposition of an operating system.

/* Mini tour of OS services via syscalls.
 * fork() -> process management
 * mmap() -> memory management
 * open()/read() -> file system
 * pipe() -> IPC */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <fcntl.h>

int main(void) {
    /* Memory management: allocate a page from the OS */
    void *page = mmap(NULL, 4096, PROT_READ|PROT_WRITE,
                      MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
    if (page == MAP_FAILED) { perror("mmap"); exit(1); }
    printf("mmap gave us page at %p\n", page);

    /* File system: read one byte from /etc/hostname */
    int fd = open("/etc/hostname", O_RDONLY);
    char hostname[64] = {0};
    read(fd, hostname, sizeof(hostname)-1);
    close(fd);
    printf("hostname: %s", hostname);

    /* Process management: fork a child */
    pid_t pid = fork();
    if (pid == 0) {
        printf("child pid=%d, parent pid=%d\n", getpid(), getppid());
        exit(0);
    } else {
        wait(NULL);   /* IPC-ish: synchronize with child termination */
        printf("parent: child exited\n");
    }
    munmap(page, 4096);
    return 0;
}

The word kernel (from the German Kern, meaning seed or core) reflects its role: it is the innermost, most trusted piece of software in the system. Everything else — the shell, init system, daemons, user applications — runs on top of it and depends on it being correct.

What makes the kernel trusted? Not just that it runs in Ring 0, but that it is the only code in Ring 0 (aside from firmware, which the kernel trusts as a lower layer). When the kernel says a memory mapping belongs to process A, the hardware enforces that. When the kernel says a file is owned by user root, no user-space process can change that without going through the kernel.

In Linux, the kernel is a monolithic kernel: all kernel subsystems (memory manager, scheduler, file systems, device drivers, networking stack) run in the same Ring 0 address space and can call each other directly. This is fast — a function call across subsystems is just a CALL instruction — but it means a bug in one subsystem (say, a buggy NIC driver) can corrupt data in another (say, the VFS layer).

The alternative is a microkernel: only the absolute minimum runs in Ring 0 (typically IPC, memory protection, and basic scheduling), and everything else runs as user-space servers. The classic example is Mach (which underpins macOS). Microkernels are safer (a buggy driver cannot corrupt the kernel) but slower (every service call crosses the user/kernel boundary, which has overhead).

Linux takes a middle path with loadable kernel modules (LKMs): drivers can be compiled separately and loaded/unloaded at runtime (insmod/rmmod), but once loaded they run in Ring 0 with full kernel privileges. This is pragmatic: it lets vendors ship proprietary drivers without distributing them with the kernel source, but it means a buggy or malicious module can compromise the entire system.

A key emergent property of the monolithic design: the kernel has no user-space heap allocator (malloc), no user-space file descriptor table, no libc. It has its own allocators (kmalloc, vmalloc), its own data structures, its own locking primitives. When you write kernel code, you are writing in a completely different environment from user space, even though it is the same C language.

/* Hello World as a Linux kernel module.
 * This code runs in Ring 0 when loaded with insmod.
 * Note: no printf, no malloc — kernel equivalents instead.
 *
 * Build: make -C /lib/modules/$(uname -r)/build M=$(pwd) modules
 * Load:  sudo insmod hello.ko
 * Check: dmesg | tail -3
 * Unload: sudo rmmod hello */
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>

MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Ring 0 Hello World");

static int __init hello_init(void) {
    printk(KERN_INFO "hello: loaded into Ring 0, pid=%d\n",
           current->pid);   /* 'current' = pointer to current task_struct */
    return 0;
}

static void __exit hello_exit(void) {
    printk(KERN_INFO "hello: unloaded\n");
}

module_init(hello_init);
module_exit(hello_exit);

So far the story has been one-directional: user code makes system calls, the kernel services them. But hardware needs to get the CPU's attention spontaneously — a key was pressed, a network packet arrived, a timer fired. The mechanism for this is interrupts.

An interrupt is a hardware signal that suspends whatever the CPU is currently executing, saves its state (registers, instruction pointer, flags) onto the kernel stack, and jumps to a kernel interrupt handler (also called an interrupt service routine or ISR). The handler runs in Ring 0, processes the event (reads the keypress from the keyboard controller, stores the packet in a buffer, advances the timer), and then returns via IRET (x86) or ERET (ARM), restoring the saved state and resuming the interrupted code.

Exceptions are the CPU's own interrupts — they fire when something unexpected happens inside the CPU pipeline: a divide-by-zero, an invalid instruction, a page fault (accessing an unmapped address), a General Protection Fault (privilege violation). These are also handled by jumping to a kernel handler registered in the Interrupt Descriptor Table (IDT) on x86 or the Exception Vector Table on ARM.

The IDT/vector table is set up by the kernel at boot time. Each entry is a gate that specifies the handler address and the privilege level required to invoke it. Exception 14 (page fault) always goes to the kernel's do_page_fault handler; exception 13 (GPF) always goes to do_general_protection.

The important insight: interrupts are the mechanism that prevents a runaway user process from hogging the CPU forever. A hardware timer fires an interrupt roughly every 1–10 ms (the HZ clock tick), transferring control to the kernel's timer interrupt handler, which invokes the scheduler. This is how preemptive multitasking works — the kernel does not ask user processes to yield; it takes control via the timer interrupt.

For kernel programmers, interrupts introduce a critical programming constraint: interrupt handlers run asynchronously, potentially interrupting any code at any time. This means interrupt handlers must be fast, must not sleep (cannot call any function that might block), and must use spinlocks rather than sleeping locks to protect shared data.

/* Demonstrate that the CPU is being interrupted (timer ticks) by
 * showing how often we lose the CPU involuntarily.
 * This is a user-space probe — kernel timer IRQ fires ~250 Hz on Linux. */
#include <stdio.h>
#include <time.h>
#include <stdint.h>

static inline uint64_t rdtsc(void) {
    uint32_t lo, hi;
    __asm__ volatile("rdtsc" : "=a"(lo), "=d"(hi));
    return ((uint64_t)hi << 32) | lo;
}

int main(void) {
    uint64_t prev = rdtsc(), cur, gap;
    int big_gaps = 0;
    for (int i = 0; i < 1000000; i++) {
        cur = rdtsc();
        gap = cur - prev;
        /* A gap >> 1000 cycles suggests we were interrupted/descheduled */
        if (gap > 500000) {
            printf("gap at iter %7d: %lu cycles (~%.1f us at 3GHz)\n",
                   i, gap, gap / 3000.0);
            if (++big_gaps >= 5) break;
        }
        prev = cur;
    }
    return 0;
}

One of the OS's most profound tricks is giving every process the illusion that it has the entire address space to itself. Process A thinks it owns addresses 0x400000–0x7fffffffffff. So does Process B. Yet they never see each other's data. How?

The hardware provides a Memory Management Unit (MMU), a chip (integrated into modern CPUs) that translates every memory address a program uses — a virtual address — into a physical address (the actual location in RAM) using a data structure called a page table.

The page table is a tree of arrays maintained by the kernel. Each leaf entry maps a 4 KB page of virtual address space to a 4 KB page frame in physical RAM. The leaf entry also contains permission bits: readable, writable, executable, and — critically — a U/S bit (User/Supervisor) that controls whether Ring 3 code can access this page.

The kernel sets U/S=0 for its own memory mappings. If a user program tries to read a kernel virtual address (which appears in the upper half of the address space), the MMU raises a page fault because U/S=0 for that page. The kernel's page fault handler then kills the process.

When the OS switches from one process to another (context switch), it loads a new page table root pointer into the CR3 register (x86-64). Instantly, every virtual address in the CPU refers to the new process's physical memory. The old process's memory is invisible. This is the mechanism behind process isolation.

The page table walk (virtual → physical) happens on every single memory access. To avoid doing this in software every time, CPUs have a Translation Lookaside Buffer (TLB) — a small, fast cache of recently used page table entries. A TLB miss triggers a page table walk in hardware (on x86) or a software TLB miss handler (on MIPS/some RISC-V configurations). When the OS changes CR3, it invalidates the entire TLB — this is called a TLB flush, and it is one reason context switches have overhead.

/* Demonstrate virtual vs physical address independence.
 * Two child processes each have a variable at the SAME virtual address
 * but they hold DIFFERENT values — because their page tables differ. */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>

int main(void) {
    int x = 42;
    pid_t pid = fork();
    if (pid == 0) {
        x = 99;  /* child modifies its own copy (copy-on-write) */
        printf("child:  &x=%p  x=%d\n", (void*)&x, x);
        exit(0);
    } else {
        wait(NULL);
        printf("parent: &x=%p  x=%d\n", (void*)&x, x);
        /* Same virtual address, different values! */
    }
    return 0;
}

Where does the kernel come from? How does it get into Ring 0 in the first place? Understanding the boot sequence demystifies the kernel's privileged position.

1. Power-on reset. The CPU resets to a known state and begins executing instructions at a fixed physical address (on x86: 0xFFFFFFF0, which maps to firmware ROM). At this point, the CPU is in Real Mode (a legacy 16-bit mode with no memory protection — a throwback to the original 8086).

2. Firmware (BIOS/UEFI). The firmware initializes RAM, probes hardware, and either reads the Master Boot Record (BIOS legacy boot) or loads an EFI application (UEFI). In either case, the firmware eventually hands off to a bootloader.

3. Bootloader (GRUB2, systemd-boot, etc.). The bootloader switches the CPU from Real Mode through Protected Mode (32-bit, hardware-enforced segments) to Long Mode (64-bit, paging active). It then loads the kernel image (typically compressed as vmlinuz) into RAM and executes it.

4. Kernel decompression and initialization. The kernel's head_64.S (x86-64 assembly entry point) decompresses the kernel, sets up an initial page table, and jumps to start_kernel() in C. At this point, the kernel is running in Ring 0 with paging enabled but most subsystems uninitialized.

5. start_kernel() initializes every kernel subsystem in a carefully ordered sequence: memory allocator, scheduler, interrupt handling, device drivers, file systems. Finally, it spawns the very first user-space process: init (PID 1), which in modern Linux is typically systemd.

6. From this point, all user-space processes are descendants of PID 1 (via fork()/exec()). The kernel is running in Ring 0, user space in Ring 3, and the dual-mode boundary is fully enforced.

The key insight: the kernel does not stay in Ring 0 continuously. Between system calls and interrupts, the CPU is in Ring 3 running user code. The kernel is entered when needed, not running as a background thread. This is why we say the kernel is a mode of the CPU rather than a process.

The kernel/userspace split can be drawn in different places, and where you draw it defines the OS architecture.

Monolithic kernels (Linux, FreeBSD, traditional Unix): all OS services run in a single Ring 0 address space. Device drivers, file systems, the network stack, the memory manager — they all share the same address space and can call each other directly. Pros: fast (direct function calls), simple programming model. Cons: a bug anywhere can corrupt everything; large trusted computing base (TCB).

Microkernels (L4, seL4, Mach, MINIX 3): only IPC, basic memory protection, and thread scheduling live in Ring 0. File systems, device drivers, and even parts of the network stack run as unprivileged user-space servers and communicate via message passing. Pros: excellent isolation (a buggy driver crashes as a user process, not the kernel), formally verifiable TCB (seL4 is the first formally verified OS kernel). Cons: message-passing overhead, more complex inter-server coordination.

Hybrid / multi-server kernels (Windows NT kernel, macOS XNU, QNX): the kernel is technically monolithic or near-monolithic, but subsystems are structured as separable components. Windows NT calls them "executive components" (Object Manager, I/O Manager, Memory Manager, etc.); they share an address space but have well-defined interfaces.

Unikernels (MirageOS, Unikraft, Nanos): an entire application and its OS are compiled into a single bootable image with no user/kernel split at all. The application runs in Ring 0. This trades isolation for extreme performance and tiny image size; popular for single-function cloud workloads.

For Linux kernel contribution, you are working with a monolithic kernel that uses loadable modules for extensibility. The architectural debates about monolithic vs micro are worth knowing because they inform every major design decision in the Linux kernel — from why system call overhead matters to why the kernel/userspace boundary must be crossed for every file access.