R0KernelPath
module 071 / 262

The Kernel Source Tree Layout

The Linux kernel source tree is a 30-million-line city — this module is your map, compass, and street-level tour guide.

~12h focused study12 concepts

Overview

When you first clone the Linux kernel repository and run ls, you are staring at the result of 33+ years of collaborative engineering by tens of thousands of contributors — roughly 30 million lines of C, assembly, and build-system glue organized across dozens of top-level directories. It feels overwhelming. It should not be.

The kernel is not a monolith dumped into a folder. It is a carefully partitioned city. Each district has a clear responsibility: the memory management district handles how physical and virtual memory is carved up; the filesystem district handles every file abstraction from ext4 to procfs; the architecture district houses the machine-specific glue that lets the same C code run on your x86 laptop, an ARM phone, and a RISC-V embedded board. Once you learn the neighborhood map, reading a stack trace or following a bug becomes a navigational exercise, not an archaeological dig.

This module teaches you that map. We will walk every major top-level directory, explain the reasoning behind each subdivision, and show you how to navigate the tree efficiently with tools like cscope, ctags, and the incredible Elixir cross-referencer at bootlin.com. By the end, you will be able to look at any kernel function name and immediately know which directory family it lives in, which Makefile compiled it, and how to jump to its definition in seconds.

Understanding the source layout is the prerequisite for everything else in kernel development: reading changelogs, understanding subsystem boundaries, knowing which maintainer to email, and knowing where to put your own new code. This is the foundation — and it is genuinely beautiful how much thought went into it.

why this matters for the kernel

Every kernel contribution begins with navigation. When Linus Torvalds reviews a patch, the first thing reviewers check is whether the code is in the right place — wrong directory means instant feedback to move it. Subsystem maintainers own specific directories, and patches must go through them. If you add a new filesystem, it goes under fs/; a new Ethernet driver under drivers/net/ethernet/; ARM-specific boot code under arch/arm/. The directory structure is not just organizational preference — it encodes the maintainership hierarchy, the compile boundaries, and the Kconfig selection logic. Getting this wrong wastes everyone's time, including yours.

Beyond contributions, day-to-day kernel debugging requires this knowledge viscerally. When a bug report shows a stack trace starting with __alloc_pages_nodemask, you need to know instantly that this is in mm/ (memory management), that the relevant expert is the MM maintainer, and that MAINTAINERS will tell you their email. When you see a symbol in arch/x86/kernel/traps.c, you know it is architecture-specific trap handling and likely not portable. This spatial intuition — knowing where things live — is what separates a kernel newcomer from someone who can productively read, debug, and extend the kernel within their first few weeks.

Linux Kernel Source Tree: Major Subsystems and Their Relationships

rendering diagram…

The major top-level directories of the Linux kernel source tree and their primary relationships. Boot flows from init/ which initializes kernel/, mm/, and fs/ in sequence. Drivers sit atop the block and net subsystems. All subsystems depend on mm/ for memory allocation and share headers from include/.

The concepts

01Cloning and First Impressions: What ls Shows You

The canonical source is git.kernel.org. Linus's tree is at git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git. Cloning it gives you around 1.1 GB of object store and a working tree of ~30 million lines. Your first ls at the root reveals approximately 25 top-level directories plus a handful of top-level files.

The top-level files matter too:

  • Makefile — the root of the entire build system. Every make command starts here.
  • Kconfig — the root configuration node; all other Kconfig files are sourced recursively from here.
  • MAINTAINERS — a 20,000-line database mapping every file pattern to a named human maintainer, their email, mailing list, and patch submission tree. This is your social map of the kernel.
  • COPYING — GPLv2 license.
  • README — surprisingly thin; points to Documentation/.
  • .clang-format — the kernel's canonical C formatting rules.

The directories break down by function. Commit this rough taxonomy to memory:

DirectoryRole
arch/CPU architecture-specific code
kernel/Core kernel: scheduling, signals, timers
mm/Memory management
fs/Filesystems
drivers/Device drivers (~50% of all lines)
net/Networking stack
include/Header files
lib/Generic library routines
init/Early boot and init process
ipc/Inter-process communication
security/LSMs: SELinux, AppArmor, etc.
crypto/Cryptographic algorithms
block/Block device layer
sound/Audio subsystem (ALSA)
virt/Virtualization (KVM host side)
tools/Userspace tools for kernel development
Documentation/Kernel documentation in RST format
scripts/Build scripts, checkpatch, etc.
samples/Tiny example kernel modules

The sheer number of lines is dominated by drivers/ (~15M lines), followed by arch/ (~5M lines), then sound/, net/, and fs/. The core logic you need for most kernel work — scheduling, memory, syscalls — is actually in a surprisingly compact ~200k lines spread across kernel/, mm/, and init/.

# Clone (shallow saves time during learning)
git clone --depth=1 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
cd linux

# Count lines by top-level directory (takes ~10s)
find . -maxdepth 2 -name '*.c' -o -name '*.h' | \
  awk -F/ '{print $2}' | sort | uniq -c | sort -rn | head -20

# Quick size survey
du -sh */ | sort -h | tail -20

02arch/: The Architecture Abstraction Layer

arch/ is the layer that makes Linux portable. It contains everything that must be written differently for each CPU family. Every subdirectory under arch/ is a supported CPU architecture: arch/x86/, arch/arm/, arch/arm64/, arch/riscv/, arch/mips/, and about 20 others.

Within each architecture directory, there is a conventional internal structure:

  • kernel/ — arch-specific versions of core kernel operations: context switch (switch_to), system call entry/exit, interrupt/exception handling, CPU initialization.
  • mm/ — architecture-specific memory management: page table format, TLB flush routines, copy_from_user/copy_to_user implementations.
  • boot/ (or boot/compressed/) — the very first code that runs after the bootloader hands off control. On x86, this is arch/x86/boot/ which contains the 16-bit real-mode setup code and the 32-bit/64-bit transition.
  • include/asm/ — architecture-specific headers. When generic code does #include <asm/page.h>, the build system resolves this to the correct arch/<ARCH>/include/asm/page.h.
  • lib/ — highly optimized arch-specific library routines: memcpy, strlen, checksum functions written in hand-tuned assembly.
  • configs/ — a set of known-good defconfig files for common boards.

The elegance here is the asm symlink trick. During make, the build system creates a symlink include/asm pointing to arch/<ARCH>/include/asm. This means portable code can write #include <asm/irqflags.h> and automatically get the right architecture's implementation.

The x86 architecture directory is one of the most complex (~2.5M lines) because x86 has 40 years of backward compatibility requirements: 16-bit real mode, 32-bit protected mode, 64-bit long mode, virtualization extensions (VMX), and countless CPU errata workarounds. The RISC-V directory (arch/riscv/) is a delight to read by contrast — it was written cleanly from scratch and is only ~35k lines.

Key files to read first:

  • arch/x86/entry/entry_64.S — the x86-64 syscall entry point. This is where SYSCALL instruction lands.
  • arch/arm64/kernel/entry.S — ARM64 exception vector table.
  • arch/riscv/kernel/head.S — RISC-V boot entry, a clean example of early boot.
/* arch/x86/include/asm/switch_to.h (simplified) */
/* Context switch: save old task's registers, load new task's */
#define switch_to(prev, next, last)                    \
do {                                                    \
    ((last) = __switch_to_asm((prev), (next)));         \
} while (0)

/* arch/riscv/kernel/entry.S (excerpt, RISC-V assembly) */
/* When a trap fires, the CPU jumps here */
.global handle_exception
handle_exception:
    /* Save all general-purpose registers to the task's stack */
    REG_S x1,  PT_RA(sp)   /* return address */
    REG_S x3,  PT_GP(sp)   /* global pointer */
    /* ... 29 more registers ... */
    call do_trap            /* jump to C handler */

03kernel/: The Heart of the OS

kernel/ contains the architecture-independent core of the operating system — the things that make Linux Linux, not just a C library on bare metal.

Key subsystems living here:

Scheduling (kernel/sched/) — The CPU scheduler. Linux uses the Completely Fair Scheduler (CFS) for normal tasks and deadline/realtime schedulers for SCHED_DEADLINE/SCHED_FIFO. The entry point is kernel/sched/core.c (~10k lines), one of the most studied files in the kernel.

Signals (kernel/signal.c) — Signal delivery, kill(), sigaction(), signal queues. When you press Ctrl+C, the terminal sends SIGINT; this file is where that signal gets placed into the task's pending set and delivered at the next return-to-userspace.

System calls (kernel/sys.c, kernel/fork.c, etc.) — The implementations of fork(), exec(), exit(), getpid(), getuid(), and hundreds more. Note: fork is in kernel/fork.c, not kernel/sys.c — large syscalls get their own files.

Timers (kernel/time/, kernel/hrtimer.c) — Hardware-independent timer infrastructure. jiffies (coarse, HZ-based) and high-resolution timers (nanosecond precision using hardware TSC/HPET/etc).

Locking (kernel/locking/) — Mutex, spinlock, rwlock, RCU (Read-Copy-Update), and semaphore implementations. kernel/locking/mutex.c and kernel/rcu/ are essential reading for concurrency.

Tracing (kernel/trace/) — ftrace, the kernel's built-in function tracer. When you run trace-cmd or read /sys/kernel/debug/tracing/, this code is running.

Module infrastructure (kernel/module/) — Since kernel 6.x, the module loading code lives here. insmod/rmmod ultimately call into kernel/module/main.c.

Workqueues (kernel/workqueue.c) — The deferred-work system. Interrupt handlers often can't sleep, so they schedule work onto workqueues to be done later in process context. Understanding this is essential for driver writing.

The kernel/ directory is ~500k lines total. For a new kernel developer, kernel/sched/core.c and kernel/fork.c are the two most instructive files to read first — they show you the real shape of a task, how it is created, and how the CPU decides to run it.

/* kernel/fork.c — simplified view of how fork() creates a new task */
long _do_fork(struct kernel_clone_args *args)
{
    struct task_struct *p;

    /* Allocate a new task_struct and copy the parent */
    p = copy_process(NULL, 0, NUMA_NO_NODE, args);
    if (IS_ERR(p))
        return PTR_ERR(p);

    /* Wake the new task — it will run soon */
    wake_up_new_task(p);

    return get_task_pid(p, PIDTYPE_PID);
}

/* The task_struct is defined in include/linux/sched.h
   It is ~750 fields: pid, mm, files, signal, cpu, state... */

04mm/: Memory Management

mm/ is where physical and virtual memory come to life. It is one of the most complex and most important subsystems, and at ~200k lines it is surprisingly self-contained.

Physical memory management (mm/page_alloc.c) — The buddy allocator. Physical memory is organized into a buddy system of power-of-two page blocks. Allocating 8 contiguous pages? The buddy allocator finds a free 8-page block (or splits a 16-page block). This is one of the best-studied algorithms in OS history.

Virtual memory areas (mm/mmap.c) — Every process has a mm_struct (defined in include/linux/mm_types.h) containing a tree of vm_area_struct (VMAs) — each VMA is a contiguous virtual address range with uniform permissions. When you call mmap(), a new VMA is inserted. When you access an unmapped address, a page fault fires and the kernel looks up which VMA covers that address to decide what to do.

Page fault handling (mm/memory.c) — The handle_mm_fault() function. This is where demand paging happens: accessing a page that exists in a VMA but has no physical backing causes a fault, and the kernel allocates a physical page and maps it in. This is the mechanism behind lazy allocation, copy-on-write (COW) after fork(), and memory-mapped files.

The slab allocator (mm/slab.c, mm/slub.c) — The kernel cannot call malloc() (that's for userspace). Instead, it uses SLAB/SLUB to allocate objects of specific sizes efficiently. kmalloc(128, GFP_KERNEL) routes through SLUB. SLUB is the modern default; SLAB is the classic implementation.

Swapping and reclaim (mm/vmscan.c) — When physical memory is under pressure, the kernel must reclaim pages. The LRU (Least Recently Used) lists, kswapd, and direct reclaim all live here.

OOM killer (mm/oom_kill.c) — When all else fails and there's no memory to reclaim, the Out-Of-Memory killer selects a process to terminate. The scoring heuristic (oom_badness()) is in here — a fascinating study in policy design.

Huge pages (mm/hugetlb.c, mm/transparent_hugepage/) — 2MB or 1GB pages instead of 4KB, dramatically reducing TLB pressure for memory-hungry workloads.

To understand mm/ fully, you must first understand page tables (how virtual addresses map to physical ones), which is architecture-specific — but the generic mm/ code uses an abstract 5-level page table walk (PGD→P4D→PUD→PMD→PTE) that each arch/ then maps onto its actual hardware page table format.

/* mm/page_alloc.c — requesting pages from the buddy allocator */
/* GFP flags control how the allocation may behave:
   GFP_KERNEL: may sleep, most common
   GFP_ATOMIC: cannot sleep (used in interrupt handlers)
   GFP_DMA:    must be in DMA-addressable memory */

struct page *alloc_pages(gfp_t gfp_mask, unsigned int order)
{
    /* 'order' means 2^order contiguous pages.
       order=0: 1 page (4KB), order=1: 2 pages, order=9: 512 pages */
    return __alloc_pages(gfp_mask, order, numa_node_id(), NULL);
}

/* Getting a kernel virtual address from a page */
void *addr = page_address(page);  /* only valid for lowmem pages */

/* The slab allocator for kernel objects */
void *buf = kmalloc(sizeof(struct my_object), GFP_KERNEL);
kfree(buf);

05fs/: Filesystems

fs/ contains both the Virtual Filesystem Switch (VFS) — the abstract layer that makes read() work regardless of whether you're on ext4, NFS, or procfs — and the implementations of each specific filesystem.

VFS layer (fs/open.c, fs/read_write.c, fs/namei.c, fs/inode.c, fs/dcache.c) — The VFS defines abstract objects: struct super_block (a mounted filesystem), struct inode (a file's metadata), struct dentry (a directory entry/name), and struct file (an open file descriptor). Each concrete filesystem fills in function pointers for operations like ->read_iter, ->write_iter, ->lookup. When you call read(fd, buf, n) in a C program, the kernel walks: syscall → fs/read_write.c:ksys_read() → file->f_op->read_iter() → the specific filesystem's read function.

Major filesystem implementations:

  • fs/ext4/ — The workhorse Linux filesystem. ~60k lines. Start with fs/ext4/inode.c and fs/ext4/extents.c.
  • fs/btrfs/ — The modern copy-on-write filesystem with snapshots, checksums, and RAID. ~100k lines, very complex.
  • fs/xfs/ — SGI's high-performance filesystem, excellent code quality. ~100k lines.
  • fs/nfs/ — Network filesystem client. Demonstrates how VFS abstracts over network protocols.
  • fs/proc/ — The /proc virtual filesystem. No disk backing; each read generates data dynamically. fs/proc/task_mmu.c generates /proc/<pid>/maps.
  • fs/sysfs/ — The /sys filesystem exposing kernel objects and their attributes.
  • fs/tmpfs/ and fs/ramfs/ — Memory-backed filesystems. tmpfs supports swapping out to disk; ramfs does not.
  • fs/overlayfs/ — The union filesystem used by Docker containers to layer images.
  • fs/fuse/ — Filesystem in Userspace. The kernel side of the FUSE protocol.

The dentry cache (fs/dcache.c) — Looking up a path like /usr/bin/gcc requires traversing three directory entries. The kernel caches these in the dentry cache (dcache) to avoid repeated disk I/O. The dcache is one of the busiest data structures in the kernel.

Important concept — everything is a file. Sockets, pipes, /proc entries, device nodes, and even eBPF maps are all represented as inodes in some filesystem, with a struct file when opened. The VFS unification is what makes select()/poll() work across heterogeneous I/O objects.

/* How a filesystem registers itself with VFS */
/* fs/ext4/super.c */
static struct file_system_type ext4_fs_type = {
    .owner    = THIS_MODULE,
    .name     = "ext4",
    .mount    = ext4_mount,    /* called when: mount -t ext4 ... */
    .kill_sb  = kill_block_super,
    .fs_flags = FS_REQUIRES_DEV,
};

static int __init ext4_init_fs(void)
{
    return register_filesystem(&ext4_fs_type);
}

/* The inode operations table — VFS calls these function pointers */
static const struct inode_operations ext4_dir_inode_operations = {
    .create     = ext4_create,   /* create a new file */
    .lookup     = ext4_lookup,   /* find a dentry in a directory */
    .link       = ext4_link,
    .unlink     = ext4_unlink,
    .mkdir      = ext4_mkdir,
    /* ... more ops ... */
};

06drivers/: The Largest Neighborhood

drivers/ is approximately half the kernel by line count (~15 million lines). It contains device drivers — code that teaches the kernel how to talk to specific hardware. The sheer variety is staggering: GPUs, USB controllers, Bluetooth chips, NVMe SSDs, Wi-Fi cards, I2C sensors, and thousands more.

The internal structure mirrors the hardware taxonomy:

  • drivers/net/ — Network device drivers. drivers/net/ethernet/intel/ holds Intel's e1000e, igb, i40e drivers. drivers/net/wireless/ holds Wi-Fi drivers.
  • drivers/block/ — Block device drivers. drivers/nvme/ — NVMe SSDs. drivers/ata/ — SATA/PATA.
  • drivers/usb/ — USB host controllers (xHCI, EHCI), USB gadget (device-side), and USB class drivers.
  • drivers/gpu/drm/ — Direct Rendering Manager (DRM). GPU drivers: drm/i915/ (Intel), drm/amdgpu/, drm/nouveau/ (NVIDIA reverse-engineered). Each is huge (~200k lines for i915).
  • drivers/input/ — Input devices: keyboards, mice, touchscreens. The evdev interface lives here.
  • drivers/i2c/, drivers/spi/, drivers/gpio/ — Low-speed bus protocols dominant in embedded systems.
  • drivers/pci/ — The PCI bus infrastructure. Almost every PCIe device uses this.
  • drivers/platform/ — Platform drivers: firmware-described hardware (ACPI on x86, Device Tree on ARM).
  • drivers/tty/ — Terminal drivers. Serial ports, PTY (pseudo-terminal), the line discipline layer.
  • drivers/char/ — Character device miscellanea: /dev/random, /dev/mem, /dev/null.
  • drivers/firmware/ — ACPI, EFI, Device Tree parsing.

The driver model — Every driver in drivers/ interacts with the kernel through the driver model (drivers/base/): struct device, struct device_driver, bus types. When a USB device is plugged in, the USB bus scans for a driver whose id_table matches the device's vendor/product IDs, then calls the driver's probe() function. This probe/remove lifecycle is universal across PCI, USB, I2C, platform, and other bus types.

For new kernel contributors, drivers/ is often the best entry point. Writing a driver for a simple I2C sensor or a platform device involves less global state than touching mm/ or kernel/, and the failure mode is typically just a broken device rather than a kernel panic.

/* Minimal PCI driver skeleton — the pattern is the same for USB, I2C, etc. */
#include <linux/pci.h>
#include <linux/module.h>

/* Table of hardware this driver handles */
static const struct pci_device_id my_pci_ids[] = {
    { PCI_DEVICE(0x8086, 0x1234) },  /* Intel vendor, device 0x1234 */
    { 0 }  /* sentinel */
};
MODULE_DEVICE_TABLE(pci, my_pci_ids);

/* Called when matching hardware is found */
static int my_probe(struct pci_dev *pdev,
                    const struct pci_device_id *id)
{
    int err = pci_enable_device(pdev);
    if (err) return err;
    /* map BAR0, request IRQ, init hardware... */
    dev_info(&pdev->dev, "device found!\n");
    return 0;
}

static void my_remove(struct pci_dev *pdev)
{
    /* reverse of probe: free IRQ, unmap, disable */
}

static struct pci_driver my_driver = {
    .name     = "my_driver",
    .id_table = my_pci_ids,
    .probe    = my_probe,
    .remove   = my_remove,
};
module_pci_driver(my_driver);  /* registers + module init/exit */

07include/: The Header Universe

include/ contains the kernel's header files, organized into several critical subdirectories:

include/linux/ — The main collection of kernel headers. include/linux/sched.h defines struct task_struct. include/linux/mm.h defines struct mm_struct and page-manipulation macros. include/linux/fs.h defines struct inode, struct file, struct super_block. Nearly every .c file in the kernel starts with #include <linux/kernel.h> and #include <linux/module.h>. This directory has ~1000 header files and is your constant companion while reading the kernel.

include/uapi/ — The userspace API headers. These define the ABI boundary: structures, constants, and ioctl numbers that both the kernel and userspace programs use. The rule is strict: include/uapi/ contains only things that must be stable and usable from userspace without kernel internals. For example, include/uapi/linux/if_ether.h defines Ethernet frame constants that network programs use. These headers get installed into /usr/include/linux/ on your system.

include/asm-generic/ — Generic (architecture-independent) fallback implementations of assembly-defined interfaces. If an architecture doesn't provide its own asm/atomic.h, it can use the generic version from here.

include/linux/compiler.h and compiler-gcc.h — Wrappers around GCC/Clang attributes: __must_check, __packed, __aligned, likely(), unlikely(), __cold, __init. These are used pervasively throughout the kernel. likely(x) expands to __builtin_expect(!!(x), 1) — it hints to the CPU's branch predictor and to the compiler that branch is probably taken.

include/linux/list.h — The kernel's legendary intrusive doubly-linked list. Instead of a list containing void *data, a kernel list embeds struct list_head inside the data structure itself, then uses the container_of() macro to recover the outer struct. This is perhaps the most elegant data structure in the kernel and a foundational pattern you will see everywhere.

include/linux/types.h — Kernel type aliases: u8, u16, u32, u64, __le32 (little-endian 32-bit), __be32 (big-endian). The endian-annotated types (__le32, __be32) work with the sparse static analysis tool to catch endianness bugs.

/* include/linux/list.h — the intrusive list pattern */
struct list_head {
    struct list_head *next, *prev;
};

/* In your data structure, embed list_head: */
struct my_device {
    int id;
    char name[32];
    struct list_head list;  /* embed the list linkage */
};

/* Iterate over a list: */
struct my_device *dev;
list_for_each_entry(dev, &device_list, list) {
    printk(KERN_INFO "device: %s\n", dev->name);
}

/* container_of(): given a pointer to list_head,
   recover the enclosing my_device: */
struct list_head *node = /* some pointer */;
struct my_device *d = container_of(node, struct my_device, list);

/* likely/unlikely — hint the branch predictor */
if (unlikely(error_condition)) {
    /* slow error path — compiler puts this out of line */
}

08net/: The Networking Stack

net/ implements the complete networking stack, from raw Ethernet frames up through TCP/IP to BSD sockets. It is ~1 million lines and one of the most sophisticated pieces of systems software ever written.

Socket layer (net/socket.c) — The socket(), bind(), connect(), send(), recv() syscalls. This is the userspace-facing API layer. net/socket.c:sys_socket() creates a struct socket, which wraps a struct sock (the protocol-level object).

Protocol families:

  • net/ipv4/ — IPv4 networking. net/ipv4/tcp.c, tcp_input.c, tcp_output.c implement TCP — 15,000+ lines handling connection setup, flow control, congestion control (CUBIC, BBR), retransmission. net/ipv4/udp.c — UDP. net/ipv4/ip_forward.c — IP routing/forwarding.
  • net/ipv6/ — IPv6. Mirrors the IPv4 structure.
  • net/unix/ — Unix domain sockets (AF_UNIX). The fast local IPC mechanism used by D-Bus, systemd, Docker.
  • net/netlink/ — The kernel↔userspace configuration protocol. ip addr, ip route use netlink to talk to the kernel network stack. rtnetlink lives here.

Packet flow (receive path):

  1. NIC DMA's a packet into memory → triggers an interrupt.
  2. Driver calls netif_rx() or NAPI poll in net/core/dev.c.
  3. Packet traverses net/core/netfilter.c hooks (iptables/nftables rules).
  4. Protocol demux: IP layer (net/ipv4/ip_input.c) → TCP/UDP layer.
  5. Data is placed into a socket's receive buffer (sk_buff).
  6. Sleeping recv() call is woken up.

struct sk_buff (include/linux/skbuff.h) — The universal network buffer. Every packet in the kernel is represented as an sk_buff (skb). It contains pointers to the packet data and metadata (protocol headers parsed so far, timestamps, device). Understanding sk_buff is prerequisite to any network driver work.

net/core/filter.c — The BPF/eBPF JIT. This is where eBPF programs attached to socket filters, XDP hooks, and tc classifiers get executed. One of the most actively developed parts of the modern kernel.

/* The sk_buff — the kernel's packet structure */
/* When you send data via TCP, the journey: */

/* 1. Userspace: write(fd, buf, len) */
/* 2. net/socket.c -> net/ipv4/tcp.c:tcp_sendmsg() */
/* 3. Data is copied into sk_buff(s) */
struct sk_buff *skb = alloc_skb(size, GFP_KERNEL);
skb_reserve(skb, NET_SKB_PAD);      /* headroom for headers */
skb_put(skb, data_len);             /* extend data area */
memcpy(skb->data, user_data, data_len);

/* 4. TCP adds its header: */
struct tcphdr *th = (struct tcphdr *)skb_push(skb, tcp_hdr_len);
th->source = htons(sport);
th->dest   = htons(dport);
/* htons() = host-to-network byte order (big-endian) */

/* 5. IP adds its header, then the driver transmits */

09init/, ipc/, security/, crypto/, block/

Several other top-level directories deserve attention:

init/ — The kernel's very first C code after architecture setup. init/main.c contains start_kernel() — the function called by arch/ boot code once the MMU is on and a basic stack is set up. It initializes every subsystem in dependency order: memory, RCU, scheduler, VFS, devices. It culminates in running the first userspace process (/sbin/init or whatever init= specifies on the kernel command line). init/main.c:start_kernel() is the best single function to read if you want to understand the kernel boot sequence — it reads almost like a table of contents for the entire kernel.

ipc/ — POSIX inter-process communication: System V semaphores (ipc/sem.c), message queues (ipc/msg.c), shared memory (ipc/shm.c), and POSIX shared memory. Also contains ipc/mqueue.c for POSIX message queues. These are the old-school IPC mechanisms (pre-socket era). They are less commonly used in new code but crucial for understanding how processes share data without sockets.

security/ — Linux Security Module (LSM) framework (security/security.c) plus implementations: security/selinux/ (SELinux, dominant on RHEL/Fedora/Android), security/apparmor/ (Ubuntu default), security/smack/, security/tomoyo/, security/bpf/. The LSM framework inserts hook calls throughout the kernel at security-sensitive operations (opening a file, creating a socket, executing a binary). Each LSM can allow or deny these operations based on its own policy. Understanding LSMs is essential for container security, Android hardening, and server hardening.

crypto/ — The kernel cryptographic API. Software implementations of AES, SHA-256, RSA, ECDSA, ChaCha20, and dozens more, plus glue code to use hardware acceleration (AES-NI on x86, ARMv8 crypto extensions). Used by filesystem encryption (fscrypt), network protocols (TLS, IPsec), dm-crypt (full-disk encryption), and eBPF. The API uses an asynchronous request model to support hardware offload.

block/ — The block device layer sitting between filesystems and disk drivers. Implements the I/O scheduler (mq-deadline, BFQ, kyber), bio merging, and the blk-mq (multi-queue) infrastructure. When ext4 issues a write, it creates a bio structure; the block layer queues it, possibly merges it with adjacent writes, then dispatches it to the NVMe or SATA driver.

/* init/main.c — start_kernel() (heavily abbreviated) */
asmlinkage __visible void __init __no_sanitize_address
start_kernel(void)
{
    /* Architecture already set up MMU, stack, and called us */
    set_task_stack_end_magic(&init_task); /* guard the init stack */
    setup_arch(&command_line);           /* arch-specific init */
    mm_init();                           /* memory management */
    sched_init();                        /* scheduler */
    rcu_init();                          /* RCU locking */
    init_IRQ();                          /* interrupt subsystem */
    timekeeping_init();                  /* clocks */
    vfs_caches_init();                   /* dcache, inode cache */
    /* ... 50+ more init calls ... */
    rest_init();  /* spawns kthread, then runs /sbin/init */
}

10Documentation/, scripts/, tools/, samples/

These support directories are less glamorous but critically important for productive kernel development.

Documentation/ — The official kernel documentation, written in reStructuredText (.rst) and buildable with Sphinx (make htmldocs). Key subdirectories:

  • Documentation/admin-guide/ — runtime kernel parameters, sysctl, kernel command line options.
  • Documentation/kernel-hacking/ and Documentation/process/ — The kernel development process, coding style, submitting patches. Documentation/process/submitting-patches.rst is required reading before your first patch.
  • Documentation/driver-api/ — The driver developer API reference.
  • Documentation/filesystems/ — Per-filesystem documentation.
  • Documentation/core-api/ — Memory management, locking, data structures.
  • Documentation/devicetree/bindings/ — Device Tree binding specifications (YAML), essential for embedded/ARM work.

scripts/ — Build system helpers and development tools:

  • scripts/checkpatch.pl — The Perl script that checks your patch for coding style violations. Run it on every patch before submitting: ./scripts/checkpatch.pl 0001-my-patch.patch.
  • scripts/get_maintainer.pl — Given a file or patch, outputs the correct maintainers and mailing lists: ./scripts/get_maintainer.pl 0001-my-patch.patch.
  • scripts/kernel-doc — Extracts kernel-doc comments (the /** style) into the Sphinx documentation.
  • scripts/bloat-o-meter — Compares two kernel builds to measure size changes.
  • scripts/dtc/ — Device Tree Compiler, for building .dtb files from .dts source.

tools/ — Userspace tools that complement kernel features:

  • tools/perf/ — The perf profiling tool. Uses kernel perf events. Massive (~300k lines).
  • tools/bpf/ — BPF development tools, including bpftool.
  • tools/testing/selftests/ — The kernel self-test suite. Tests for each subsystem that can be run with make -C tools/testing/selftests run_tests.
  • tools/include/ — Copies of kernel headers synchronized for use in these userspace tools.

samples/ — Tiny example kernel modules and programs, excellent for learning:

  • samples/kobject/ — How to create kobjects and sysfs attributes.
  • samples/kprobes/ — Attaching kprobes for dynamic tracing.
  • samples/bpf/ — BPF program examples.

The samples/ directory is a goldmine for beginners. Each sample is a working, buildable module with comments explaining what it demonstrates.

# Key scripts every kernel developer runs constantly

# 1. Check your patch for style issues
./scripts/checkpatch.pl --strict 0001-my-fix.patch

# 2. Find who to email the patch to
./scripts/get_maintainer.pl 0001-my-fix.patch
# Output example:
# Greg Kroah-Hartman <gregkh@linuxfoundation.org> (maintainer:USB...)
# linux-usb@vger.kernel.org (open list:USB...)

# 3. Build documentation
make htmldocs
# Output in Documentation/output/

# 4. Run a specific self-test
make -C tools/testing/selftests/net run_tests

# 5. Find kernel-doc comments for a function
./scripts/kernel-doc -man kernel/fork.c | man -l -

11Navigating 30M Lines: cscope, ctags, and Elixir

Knowing where directories are is necessary but not sufficient. You also need tools to jump to a function definition, find all callers of a function, or see every file that includes a particular header. Three tools dominate kernel development navigation:

cscope — The classic kernel navigation tool. cscope builds an index of all symbols, function definitions, and callers. The kernel Makefile has a built-in target:

make cscope

This generates cscope.out (and related files) in the tree root. Then you can use cscope -d to browse interactively, or integrate it with Vim (Ctrl-] to jump to definition, :cs find c function_name to find callers). The advantage: it works offline and understands C at a syntactic level.

ctags / etags — Faster to generate but less semantically rich:

make tags    # generates tags file for Vim/Neovim/Emacs
make TAGS    # generates TAGS file for Emacs

With Vim+ctags, Ctrl-] on any function name jumps to its definition, Ctrl-T jumps back. With Neovim + LSP (clangd), you get even better navigation using the actual compiler's understanding.

Bootlin Elixir Cross-Referencer (elixir.bootlin.com) — The most powerful tool for remote/casual kernel exploration. Elixir is a web-based cross-referencer that:

  • Indexes every released kernel version from 2.6.x to the latest.
  • Shows definition, all uses, and all includes for any identifier.
  • Links macro expansions, struct fields, and function pointers.
  • Lets you switch kernel versions to see when something was introduced or changed. Go to https://elixir.bootlin.com/linux/latest/source and search for schedule() — you'll see its definition, all call sites, and can click through to related functions. This is invaluable when you don't have the tree locally.

clangd + compile_commands.json — Modern editors (VS Code, Neovim) can use the Language Server Protocol with clangd for near-IDE-quality navigation. The kernel supports generating compile_commands.json:

bear -- make -j$(nproc)  # intercept the build
# or
make compile_commands.json  # supported since kernel 5.6

Then configure your editor to use clangd. This gives you go-to-definition, find-references, hover documentation, and diagnostics as you type.

git log and git blame — Never underestimate version control for navigation. git log --oneline fs/ext4/inode.c shows every commit that touched that file, with reasons. git blame fs/ext4/inode.c shows who last changed each line. git log --all -S 'function_name' (pickaxe search) finds every commit that added or removed that string — essential for understanding when and why something was introduced.

grep and git grep — For quick searches: git grep 'schedule_timeout' is faster than bare grep because it only searches tracked files. With --n for line numbers and -l for just filenames, this covers many quick navigation tasks.

# One-time index build (run from kernel source root)
make cscope tags

# Vim: jump to definition of 'schedule'
# Place cursor on 'schedule', press Ctrl-]
# Jump back with Ctrl-T

# cscope interactive queries (from shell)
cscope -d
# Then choose:
# 0: Find symbol
# 1: Find function definition
# 2: Find functions called BY this function
# 3: Find functions CALLING this function
# 4: Find text string

# Generate compile_commands.json for clangd
make compile_commands.json
# Now open VS Code / Neovim with clangd extension

# git pickaxe: when was wake_up_process() introduced?
git log --all -S 'wake_up_process' --oneline | head -5

# Find all files a maintainer owns
grep -A5 'SCHEDULER' MAINTAINERS

12The Kconfig and Makefile System: How the Tree Compiles

Understanding the tree layout requires understanding why some code gets compiled and some doesn't. The kernel uses a two-phase build system: configuration (Kconfig) followed by compilation (Makefile).

Kconfig — Every directory with compilable code has a Kconfig file defining configuration symbols. CONFIG_EXT4_FS=y means ext4 is compiled in. CONFIG_EXT4_FS=m means it compiles as a loadable module. CONFIG_EXT4_FS not set means fs/ext4/ is not compiled at all. The top-level Kconfig recursively sources all others. make menuconfig presents a TUI over the entire Kconfig tree; make defconfig loads a sane default for the current architecture.

Makefiles — Each directory has a Makefile (historically called Makefile, but the kernel uses Kbuild syntax). The pattern is:

obj-$(CONFIG_EXT4_FS) += ext4.o
ext4-y := super.o inode.o dir.o file.o ...
ext4-$(CONFIG_EXT4_ENCRYPTION) += crypto.o

obj-y means always compile. obj-m means compile as module. obj-$(CONFIG_FOO) expands to obj-y, obj-m, or empty depending on the config. The Kbuild system walks the entire source tree, reading each Makefile, and builds exactly what was configured.

The __init and __exit annotations — You will see these everywhere:

static int __init my_init(void) { ... }
static void __exit my_exit(void) { ... }

__init is __attribute__((section(".init.text"))) — code placed in a special section that the kernel frees after boot. This recovers memory used only during initialization. __exit is similarly placed in a section discarded if the code is compiled in (non-modular) since it can never be called. For a module, both are called normally.

Cross-compilation — The build system natively supports cross-compilation:

make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8

This tells the Makefile to use arch/arm64/ and to prefix all compiler invocations with aarch64-linux-gnu-.

Understanding the Kconfig/Makefile system means you can answer questions like: "Why isn't this driver compiling?" (check Kconfig dependency), "Where does this object file come from?" (trace the Makefile), and "How do I add a new source file to the kernel?" (add it to the directory's Makefile).

# fs/ext4/Kconfig (excerpt)
config EXT4_FS
    tristate "The Extended 4 (ext4) filesystem"  # tristate: y/m/n
    select CRC16
    select CRYPTO
    select CRYPTO_CRC32C
    help
      This is the next generation of the ext3 filesystem.

config EXT4_ENCRYPTION
    bool "Ext4 Encryption"
    depends on EXT4_FS  # only visible if EXT4_FS is selected
    select FS_ENCRYPTION

# fs/ext4/Makefile (excerpt)
obj-$(CONFIG_EXT4_FS) += ext4.o  # ext4.o = the module or built-in object

ext4-y := balloc.o bitmap.o block_validity.o dir.o extents.o \
          extents_status.o file.o fsmap.o fsync.o hash.o \
          ialloc.o indirect.o inline.o inode.o ioctl.o \
          mballoc.o migrate.o mmp.o namei.o page-io.o \
          readpage.o resize.o super.o symlink.o sysfs.o \
          truncate.o verity.o xattr.o xattr_trusted.o xattr_user.o

ext4-$(CONFIG_EXT4_ENCRYPTION) += crypto.o

Key insights

  • ✦The kernel source tree is not arbitrary — it encodes the maintainership hierarchy. The directory a file lives in determines who reviews your patch.
  • ✦drivers/ is ~50% of the kernel by line count, but the core logic (kernel/, mm/, fs/, net/) you need to understand the OS is a much more compact ~1-2M lines.
  • ✦The arch/ layer is the portability seam. All machine-specific code lives under arch/<ARCH>/; generic code must never include arch-specific headers directly.
  • ✦The VFS (fs/) is the unifying abstraction: sockets, pipes, device nodes, and regular files are all inodes. This is why select()/poll()/epoll() work across heterogeneous I/O.
  • ✦Kconfig symbols connect the source tree to the build: CONFIG_FOO=y compiles it in, =m makes a module, unset means the code doesn't exist in the binary at all.
  • ✦The MAINTAINERS file is the social map of the kernel. Before sending any patch, run scripts/get_maintainer.pl to find the right human and mailing list.
  • ✦Navigation tools (cscope, ctags, clangd, Elixir) are not optional niceties — reading 30M lines without semantic jump-to-definition is simply not practical.
  • ✦The __init annotation is a beautiful example of kernel frugality: initialization code is placed in a special ELF section that gets freed back to the memory pool after boot, saving real RAM on every running system.

Common pitfalls

  • ⚠Confusing include/linux/ (kernel-internal headers) with include/uapi/linux/ (stable ABI headers for userspace). Kernel internals change; UAPI headers must remain backward-compatible forever.
  • ⚠Forgetting that arch/<ARCH>/include/asm/ is reached via the <asm/...> include path through a build-time symlink — beginners often search the wrong place for architecture headers.
  • ⚠Running make tags or make cscope without first running make defconfig, resulting in an incomplete index that misses conditionally-compiled code.
  • ⚠Editing a .c file and wondering why the change has no effect — because the corresponding CONFIG_ symbol is not set, so the file was never compiled.
  • ⚠Submitting a patch to the wrong mailing list (e.g., lkml instead of the subsystem list) because they did not run scripts/get_maintainer.pl first.
  • ⚠Treating Documentation/ as authoritative for kernel behavior without cross-checking with the source. Documentation can be stale; the code is always ground truth.
  • ⚠Assuming that because a function is declared in include/linux/foo.h, its implementation is in kernel/foo.c. Implementations can be spread across mm/, arch/, or multiple files. Use cscope/Elixir to find the actual definition.
  • ⚠Modifying files under include/uapi/ for an internal kernel change — those headers are the ABI boundary and changing them can break userspace programs silently.

Practice

  1. 01
    easy

    Clone the kernel source (shallow clone is fine) and write a shell one-liner that outputs the top 10 subdirectories of drivers/ sorted by number of .c files, along with counts. Then find which single .c file in all of drivers/ is the longest (most lines). What does this tell you about where complexity concentrates?

  2. 02
    medium

    Build a minimal x86_64 kernel configuration with make tinyconfig, then make menuconfig and enable only ext4 (CONFIG_EXT4_FS=y) and the loop block device (CONFIG_BLK_DEV_LOOP=y). Build it. Now look at the generated .config and the build output: which .o files in fs/ext4/ were compiled, and which were NOT compiled because their CONFIG_ guards were not set?

  3. 03
    medium

    Use Elixir at elixir.bootlin.com to trace the complete call chain from the schedule() function down to the architecture-specific context switch. Start at kernel/sched/core.c:schedule(), follow __schedule(), then context_switch(), then switch_to(). Document which files/directories each function lives in and at what point the code crosses from arch-independent into arch-specific.

  4. 04
    hard

    Write a minimal character device kernel module (loadable, not built-in) that: (1) registers a /dev/hello device, (2) returns the string 'Hello, kernel!\n' on every read(), and (3) is correctly placed in a new directory samples/hello_cdev/ with its own Kconfig and Makefile entries so it integrates with the kernel build system. Verify with insmod, cat /dev/hello, and rmmod.

  5. 05
    easy

    Generate a cscope database for a kernel you have configured and built. Then answer three questions using only cscope (no grep, no web): (1) Which functions call wake_up_process()? (2) Where is struct task_struct defined? (3) Which files include <linux/sched.h>? Record how many results each query returns.

  6. 06
    medium

    Using git log and git blame on the mm/oom_kill.c file, answer: (a) Who wrote the original OOM killer? (b) When was the oom_badness() scoring function last significantly changed and why (read the commit message)? (c) How many distinct authors have committed to this file in the last 5 years? This exercise builds the historical navigation skill essential for understanding why code is the way it is.

Resources

🔧Linux Kernel Source Browser (Elixir)
toolfree

The single best tool for exploring the kernel source online; cross-references every symbol across every kernel version.

📄Linux Kernel Documentation — kernel-hacking guide
docsfree

Official guide to kernel internals concepts, written for developers entering the kernel for the first time.

📄Linux Kernel Documentation — submitting patches
docsfree

Required reading before your first patch; covers the exact workflow from git format-patch to mailing list submission.

📕Linux Device Drivers, 3rd Edition (LDD3)
bookfree

The canonical driver development book; free PDF from LWN; some APIs are outdated but conceptual coverage remains excellent.

✎KernelNewbies — Linux kernel source map
articlefree

Community wiki with practical getting-started guides, subsystem explanations, and links to good first bugs.

✎LWN.net — kernel development articles
articlefree

The best ongoing journalism and technical deep-dives on the kernel; the index is searchable by subsystem.

📕Linux Kernel Development, 3rd Edition
bookpaidRobert Love — Addison-Wesley / Pearson; also available on O'Reilly

The most accessible single-volume introduction to kernel internals; covers scheduling, MM, VFS, and the development process.

📕Understanding the Linux Kernel, 3rd Edition
bookpaidBovet & Cesati — O'Reilly

Deeper and more comprehensive than Love's book; focuses on x86 internals with extraordinary depth on MM and VFS.

🎓Bootlin Embedded Linux and Kernel Training Materials
coursefree

Free slide decks covering kernel architecture, driver development, and debugging — professionally produced and regularly updated.

✎Julia Evans — How to read the Linux kernel source
articlefree

Approachable beginner perspective on how to actually start reading and modifying kernel code.

🔧git.kernel.org — official Linux kernel git trees
toolfree

The canonical source; browse subsystem trees (mm, net, fs) to follow development before patches hit mainline.

Checkpoints — you can now…

  • ✓You can look at any kernel stack trace or file path and immediately name which top-level directory family it belongs to and what subsystem owns it.
  • ✓You can run make cscope tags from a configured kernel tree and use cscope to find the definition of any function and all its callers without leaving your terminal.
  • ✓You can use Elixir at elixir.bootlin.com to trace a call chain across multiple files and identify exactly where arch-independent code hands off to architecture-specific code.
  • ✓You can run scripts/get_maintainer.pl on any file or patch and identify the correct maintainer email and mailing list for a contribution.
  • ✓You can explain why a file in arch/riscv/kernel/ will never be compiled into an x86 kernel build, connecting the answer to Kconfig, the ARCH variable, and the Makefile obj-$(CONFIG_...) pattern.
  • ✓You can create a minimal kernel module in a new samples/ subdirectory with its own Kconfig and Makefile, and have it properly integrate with make menuconfig and the kernel build.
  • ✓You can use git log, git blame, and git log -S (pickaxe) to answer historical questions about why a piece of kernel code is the way it is.