C Toolchain & Translation Phases

Before writing a single line, it is worth asking: why does virtually every operating system kernel, hypervisor, embedded firmware, and high-performance runtime use C? The answer is not tradition — it is a deliberate engineering trade-off that has held up for fifty years.

C gives you direct hardware correspondence. Every C statement maps to a small, predictable set of machine instructions. There are no hidden allocations (contrast with Java's object boxing), no surprise garbage-collection pauses (contrast with Go or Python), no implicit virtual dispatch (contrast with C++), no runtime type information you did not explicitly write. When you write:

int x = 42;

You are making the compiler allocate four bytes (on all modern platforms where sizeof(int)==4) and store the bit-pattern 0x0000002A into them. That is it. The CPU does exactly that.

C also gives you pointer arithmetic — the ability to treat memory as a flat array of bytes and navigate it with integer arithmetic. This is indispensable for the kernel, which must manipulate page tables, device registers, and DMA buffers as raw memory addresses.

The trade-off is that C gives you nothing for free. Memory safety is your responsibility. Out-of-bounds accesses, use-after-free, and integer overflow are all legal C that produce undefined behavior — the compiler assumes they never happen and may optimize based on that assumption in ways that break your code. This danger is the reason you will hear about Rust in the kernel, which can guarantee memory safety. But C remains the primary language because of the sheer mass of existing, correct, audited code and the breadth of developer knowledge.

The key insight: C is a portable assembler. It was designed by Dennis Ritchie at Bell Labs explicitly as a language that would produce predictable, efficient machine code while being readable across CPU architectures. That is still what it is.

/* This C source and its machine-level meaning, side by side */
int add(int a, int b) {
    return a + b;  /* exactly one ADD instruction on x86 */
}

/* On x86-64, gcc -O1 produces:
   add:
       lea    eax, [rdi + rsi]   ; result = a + b
       ret
   No stack frame, no hidden work, one hardware instruction. */

Let us start with the simplest possible program and trace exactly what happens when you compile it.

The program:

#include <stdio.h>
int main(void) {
    printf("Hello, kernel world!\n");
    return 0;
}

When you type gcc hello.c -o hello, you are invoking a driver program that orchestrates four separate tools:

1. The Preprocessor (cpp) — textual transformation of the source
2. The Compiler (cc1 inside GCC, or clang) — converts C text to assembly
3. The Assembler (as) — converts assembly text to machine-code object files
4. The Linker (ld) — combines object files into an executable

The GCC (GNU Compiler Collection) and Clang/LLVM are the two dominant toolchains on Linux. For kernel work, both are now officially supported (Clang support for the kernel was completed circa 2021). They are largely interchangeable for learning purposes, but GCC is still the default for kernel builds.

A crucial mental model: the compiler never sees your whole program. It processes one .c file (called a translation unit) at a time. The linker is what stitches multiple translation units together. This is why you can have a function declared in one .c file and called from another — the compiler emits a placeholder reference, and the linker resolves it.

# See exactly what GCC invokes under the hood:
gcc -v hello.c -o hello
# You will see cc1 (the actual compiler), as (assembler), collect2 (linker wrapper)

# Or: run each stage manually
gcc -E hello.c -o hello.i      # stop after preprocessing
gcc -S hello.i -o hello.s      # stop after compilation to asm
gcc -c hello.s -o hello.o      # stop after assembly
gcc    hello.o -o hello        # link only

# Clang equivalent:
clang -E hello.c -o hello.i
clang -S hello.i -o hello.s
clang -c hello.s -o hello.o
clang    hello.o -o hello

The C standard (ISO/IEC 9899) does not say "run the preprocessor then the compiler". It defines eight abstract translation phases that a conforming implementation must behave as if it executes. Understanding them is key to understanding subtle C behavior.

Phase 1 — Character Mapping: The physical source file bytes are mapped to the source character set. On modern systems this is almost always UTF-8, but the C standard character set is a strict subset of ASCII. Trigraph sequences (??= for #, etc.) are replaced here — this is why ??/ at the end of a line creates an unexpected line continuation.

Phase 2 — Line Splicing: Any line ending with a backslash-newline (\\\n) is joined with the next line. This is how you write multi-line macros:

#define MAX(a, b) \\
    ((a) > (b) ? (a) : (b))

Phase 3 — Tokenization: The source is broken into preprocessing tokens: identifiers, numbers, string literals, punctuators, and whitespace/comments. Comments are replaced with a single space. This is where // and /* */ comments disappear.

Phase 4 — Preprocessing: Preprocessor directives (#include, #define, #if, etc.) are executed. #include causes the named file to be processed through phases 1–4 recursively and inserted. #define creates macro definitions. #ifdef/#ifndef conditionally include/exclude code. This is a pure text-substitution phase — the preprocessor knows nothing about C syntax or types.

Phase 5 — Character/String Literal Processing: Escape sequences inside string and character literals are converted to their character values. \n becomes byte 0x0A, \t becomes 0x09, \x41 becomes A.

Phase 6 — String Literal Concatenation: Adjacent string literals are concatenated. "hello " "world" becomes "hello world". This is useful for long strings split across lines and for composing strings from macros.

Phase 7 — Compilation: The preprocessing tokens are syntactically and semantically analyzed as C. This is the main compilation phase — parsing, type checking, intermediate representation generation, optimization, and code generation (to assembly). This is what people usually mean when they say "the compiler".

Phase 8 — Linking: All external references (calls to functions defined in other translation units, references to external variables) are resolved. Object files and libraries are combined into the final executable or shared library.

The beauty of this model: phases 1–6 are purely textual and handled by the preprocessor. Phase 7 is the "real" compiler. Phase 8 is the linker. The standard defines their behavior precisely, which is why a C program can be compiled by GCC on x86, Clang on ARM, or MSVC on Windows and produce the same behavior.

/* Demonstration of Phase 2 (line splicing) and Phase 6 (string concatenation) */

#define GREETING \
    "Hello, " \
    "kernel world!"

/* Phase 6: these two string literals become one at compile time */
const char *msg = "Hello, "  "world\n";

/* Phase 4: the preprocessor replaces GREETING before the compiler sees it */
const char *msg2 = GREETING;

/* After preprocessing (gcc -E), both become:
   const char *msg  = "Hello, " "world\n";  (then phase 6 merges)
   const char *msg2 = "Hello, " "kernel world!"; */

The preprocessor is a purely textual substitution engine. It operates on tokens, not on C syntax. This is both its power and its danger.

#include takes the entire contents of the named file and pastes it verbatim at that point in the source. This is why #include <stdio.h> puts thousands of lines of declarations into your translation unit. Try gcc -E hello.c | wc -l — a single-line program becomes thousands of lines after preprocessing.

#define for constants is a simple token replacement:

#define PAGE_SIZE 4096
char buf[PAGE_SIZE];   /* becomes: char buf[4096]; */

Function-like macros expand with arguments, but be careful — macro arguments are expanded textually, not evaluated first:

#define SQUARE(x)  x * x
SQUARE(1 + 2)  /* expands to: 1 + 2 * 1 + 2 = 5, NOT 9! */
#define SQUARE(x)  ((x) * (x))  /* correct: always parenthesize */

This parenthesization discipline is one of the most important C macro rules, and you will see it throughout the Linux kernel source.

#ifdef / #ifndef / #if conditionally compile code based on whether a macro is defined. This is how the kernel handles architecture differences:

#ifdef CONFIG_X86_64
    /* x86-64 specific code */
#elif defined(CONFIG_ARM64)
    /* ARM64 specific code */
#endif

Include guards prevent a header from being included twice (which would cause duplicate declaration errors):

#ifndef MY_HEADER_H
#define MY_HEADER_H
/* ... header contents ... */
#endif

Modern alternative: #pragma once (not in the C standard but supported by all major compilers, including for kernel headers).

The -E flag to GCC stops compilation after preprocessing and prints the result. Running gcc -E file.c and reading the output is one of the most powerful debugging tools for macro-related confusion.

/* kernel-style.c — demonstrates real kernel macro patterns */
#include <stdio.h>

/* Kernel-style safe macro with do-while trick:
   Wraps multiple statements safely for use in if/else */
#define SWAP(a, b)          \
    do {                    \
        typeof(a) _tmp = a; \
        a = b;              \
        b = _tmp;           \
    } while (0)

/* typeof() is a GCC extension used heavily in the kernel */
int main(void) {
    int x = 1, y = 2;
    SWAP(x, y);
    printf("x=%d y=%d\n", x, y);  /* x=2 y=1 */
    return 0;
}

The -S flag stops GCC after compilation and produces a .s assembly file. Reading this output is one of the most valuable skills in systems programming. It shows you exactly what the CPU will execute.

Assembly is an almost-1-to-1 representation of machine code. Each assembly instruction corresponds to one machine instruction (or occasionally a pseudo-instruction that the assembler expands). On x86-64, GCC produces AT&T syntax by default; you can get Intel syntax with -masm=intel (personal preference, Intel syntax is often easier to read).

Key x86-64 registers to know:

• rdi, rsi, rdx, rcx, r8, r9 — first 6 integer function arguments (System V AMD64 ABI)
• rax — return value
• rsp — stack pointer
• rbp — frame pointer (often omitted with -O2 and higher)

For ARM64 (AArch64), which is increasingly relevant for kernel work on Apple Silicon and embedded targets:

• x0–x7 — first 8 function arguments
• x0 — return value
• sp — stack pointer

Optimization levels dramatically change the assembly:

• -O0 (default): no optimization, every variable lives on the stack, easy to debug
• -O1: basic optimization, dead code eliminated, some inlining
• -O2: standard release optimization, loop unrolling, more aggressive inlining
• -O3: aggressive optimization including vectorization
• -Os: optimize for size

The kernel is typically compiled at -O2 with specific flags added or removed per subsystem.

Godbolt (compiler-explorer.com) is an invaluable tool: paste C code, choose compiler and flags, see the assembly side-by-side in real time. Every concept in this section becomes concrete when you experiment there.

/* sum.c */
long sum_array(const int *arr, int n) {
    long s = 0;
    for (int i = 0; i < n; i++)
        s += arr[i];
    return s;
}

/* gcc -O2 -S sum.c  (x86-64, Intel syntax: gcc -O2 -S -masm=intel)
   sum_array:
       xor     eax, eax          ; s = 0 (also clears upper 32 bits)
       test    esi, esi           ; if n == 0
       jle     .L1                ;   return 0
       lea     rdx, [rdi+rsi*4]  ; rdx = arr + n (end pointer)
   .L3:
       movsxd  rcx, DWORD [rdi]  ; rcx = *arr (sign-extend 32->64)
       add     rax, rcx          ; s += *arr
       add     rdi, 4            ; arr++
       cmp     rdi, rdx          ; arr != end
       jne     .L3               ; loop
   .L1:
       ret                       ; return s (in rax)

   Notice: the compiler turned the index variable 'i' into
   pointer arithmetic. No stack frame needed at -O2. */

The -c flag tells GCC to compile to an object file (.o) without linking. Object files are the currency of the build system — the linker takes multiple object files and combines them.

An object file is in ELF format (Executable and Linkable Format) on Linux. ELF is a structured binary format with:

• ELF header: magic number (\x7fELF), architecture, entry point
• Section headers: metadata about each section
• .text section: the compiled machine code
• .data section: initialized global/static variables
• .bss section: uninitialized globals (just a size, no actual bytes in the file)
• .rodata section: read-only data (string literals, const globals)
• Symbol table: names of all functions and global variables defined in or referenced from this file
• Relocation table: locations in the code that the linker must patch with final addresses

The readelf and objdump tools let you inspect these:

readelf -h hello.o         # ELF header
readelf -S hello.o         # section headers
objdump -d hello.o         # disassemble .text
nm hello.o                 # list symbols

Symbols are the connection between translation units. When your code calls printf, the compiler emits an undefined symbol reference in the object file. The linker finds the definition of printf in the C library (libc.so or libc.a) and patches the call address.

The symbol table has two kinds of symbols:

• Defined symbols (T in nm output for text, D for data): functions and globals this file provides
• Undefined symbols (U): references to things defined elsewhere that the linker must resolve

This is directly relevant to the kernel: the kernel has an EXPORT_SYMBOL macro that makes a kernel symbol available for use by loadable modules. Without it, a module trying to call that function fails to link.

/* math_utils.c — one translation unit */
int square(int x) { return x * x; }
int cube(int x)   { return x * square(x); }  /* calls square */

/* main.c — another translation unit */
extern int square(int);  /* declaration only; definition elsewhere */
int main(void) {
    return square(5);  /* compiler emits: undefined symbol 'square' */
}

/* Compile separately, then link:
   gcc -c math_utils.c -o math_utils.o
   gcc -c main.c -o main.o
   gcc main.o math_utils.o -o prog    # linker resolves 'square'

   nm main.o output:
     U square          <- undefined, needs linker
   nm math_utils.o output:
     T cube            <- defined in .text
     T square          <- defined in .text */

The linker (ld, invoked via gcc for convenience) performs the final stage of building an executable. It solves two problems: symbol resolution and relocation.

Symbol resolution is the process of matching every undefined symbol reference (a call or a global variable access) to a definition. If a symbol has multiple definitions, that is a linker error (multiple definition of 'foo'). If a symbol has no definition, that is also a linker error (undefined reference to 'bar'). These error messages are extremely common; now you know exactly what they mean.

Relocation is the process of filling in the actual memory addresses of functions and variables into the machine code. When the compiler generated call square, it did not know where square would live in memory — it could not, because main.c is compiled separately from math_utils.c. The compiler instead emits a placeholder (zero or a partial offset) and a relocation record that says "at this offset in the code, put the address of square". The linker, knowing where everything lands in the final binary, patches these placeholders.

Static vs. dynamic linking: With static linking (-static), all needed library code is copied into the executable. The result is a single self-contained binary. With dynamic linking (the default), the executable just records which shared libraries it needs (.so files). The dynamic linker (ld.so) loads those libraries and does relocation at runtime.

For kernel development, the kernel itself is always statically linked — there is no dynamic linker running before the kernel, because the kernel IS the foundation everything else runs on.

Link order matters for static libraries. gcc main.o -lm works; gcc -lm main.o may not. The linker scans left to right: it only includes archive members that satisfy undefined symbols seen so far.

# Inspect what libraries a binary is dynamically linked against:
ldd /bin/ls
# Output (typical Linux):
#   linux-vdso.so.1 (virtual DSO, kernel-injected)
#   libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1
#   libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
#   /lib64/ld-linux-x86-64.so.2 (the dynamic linker itself)

# Inspect relocations in an object file:
readelf -r hello.o
# Shows: offset, type, symbol name
# e.g., CALL64 relocation for 'puts' -> linker will patch this

# Build a statically linked binary:
gcc -static hello.c -o hello_static
ldd hello_static
# Output: not a dynamic executable
ls -lh hello_static  # Much larger: all of libc is included

Professional C development (and certainly kernel development) involves careful control of compiler flags. Here are the flags you will encounter most, organized by purpose.

Compilation stage control:

• -E: stop after preprocessing, print to stdout
• -S: stop after compilation, write .s assembly file
• -c: stop after assembly, write .o object file
• (no flag): compile and link to executable

Output:

• -o <file>: name the output file
• -v: verbose — show every subprocess invoked
• -###: like -v but don't actually run them (dry run)

Warning flags (essential for correct code):

• -Wall: enable most common warnings ("all" is a misnomer; it's not all)
• -Wextra: extra warnings beyond -Wall
• -Werror: treat warnings as errors (the kernel uses this)
• -Wshadow: warn when a local variable shadows an outer one
• -Wstrict-prototypes: warn on function declarations without parameter types

Optimization:

• -O0: no optimization (default)
• -O1, -O2, -O3: increasing optimization
• -Os: optimize for size
• -Og: optimize for debugging (better than -O0 for GDB)
• -fno-strict-aliasing: disable aliasing-based optimization (the kernel requires this — it does many type-punning tricks that strict aliasing would break)

Debugging:

• -g: include DWARF debug information in the output
• -g3: maximum debug info including macro expansions
• -ggdb: GDB-specific debug info

Architecture/ABI:

• -m32: compile for 32-bit x86
• -m64: compile for 64-bit x86-64
• -march=native: optimize for the current CPU's features
• -mno-red-zone: disable the x86-64 "red zone" (required for kernel — interrupt handlers don't observe the red zone convention)

Kernel-specific flags you will see in Makefiles:

• -ffreestanding: do not assume a hosted C environment (no libc)
• -fno-stack-protector: disable stack canaries (the kernel has its own protection)
• -fno-common: do not merge tentative definitions (catches bugs)
• -nostdlib: do not link standard libraries

# A thorough compile command mirroring kernel style:
gcc \
    -Wall -Wextra -Werror \
    -O2 \
    -g \
    -fno-strict-aliasing \
    -std=gnu11 \
    -c mydriver.c -o mydriver.o

# Generate preprocessed output for macro debugging:
gcc -E -dM myfile.c | sort   # -dM dumps all defined macros

# See assembly with source interleaved (very educational):
gcc -O2 -S -fverbose-asm -g sum.c -o sum.s
# Each asm line gets a comment showing which C line it came from

# Clang: extra useful flag — dump the AST (Abstract Syntax Tree)
clang -Xclang -ast-dump -fsyntax-only hello.c

The Application Binary Interface (ABI) is the contract between compiled code and the machine. It specifies:

• How function arguments are passed (which registers, which stack slots)
• How return values are returned
• Which registers a function must preserve (callee-saved) vs. may clobber (caller-saved)
• How the stack is aligned
• How data structures are laid out in memory (padding, alignment)

For x86-64 Linux, the ABI is the System V AMD64 ABI. Its calling convention:

• Integer/pointer args 1–6: rdi, rsi, rdx, rcx, r8, r9
• Floating-point args 1–8: xmm0–xmm7
• Return value: rax (or rdx:rax for 128-bit)
• Callee-saved: rbx, rbp, r12–r15
• Stack must be 16-byte aligned at a call instruction

Data alignment is critical. The C compiler inserts padding between struct members to ensure each member is naturally aligned (a 4-byte int must be at an address divisible by 4, an 8-byte long at a multiple of 8). This is why:

struct { char a; int b; } has sizeof == 8, not 5

The compiler inserts 3 bytes of padding between a and b so that b is 4-byte aligned.

In the kernel, you will often see __packed (a GCC attribute) on structs that are mapped directly to hardware registers or network packets, where there is no room for padding. The trade-off is that accessing misaligned members requires multiple instructions and may be slower.

Understanding the ABI is essential for:

• Writing inline assembly that calls C functions
• Writing C code that calls assembly
• Debugging with GDB at the assembly level
• Understanding kernel entry points and system call conventions (the syscall ABI is different from the function call ABI)

/* struct_layout.c — understand alignment and padding */
#include <stdio.h>
#include <stddef.h>

struct Padded {
    char   a;   /* 1 byte  @ offset 0 */
                /* 3 bytes padding    */
    int    b;   /* 4 bytes @ offset 4 */
    char   c;   /* 1 byte  @ offset 8 */
                /* 7 bytes padding    */
    long   d;   /* 8 bytes @ offset 16 */
};  /* total: 24 bytes */

struct Compact {
    long   d;   /* 8 bytes @ offset 0  */
    int    b;   /* 4 bytes @ offset 8  */
    char   a;   /* 1 byte  @ offset 12 */
    char   c;   /* 1 byte  @ offset 13 */
                /* 2 bytes padding     */
};  /* total: 16 bytes — reordering saves 8 bytes! */

int main(void) {
    printf("Padded:  %zu bytes\n", sizeof(struct Padded));  /* 24 */
    printf("Compact: %zu bytes\n", sizeof(struct Compact)); /* 16 */
    printf("offset of b in Padded: %zu\n",
           offsetof(struct Padded, b));  /* 4 */
    return 0;
}

This concept is not optional. Undefined behavior (UB) in C is one of the most important things to understand before writing any serious systems code, and the leading cause of security vulnerabilities in C programs.

The C standard defines some operations as having undefined behavior — meaning the standard makes no guarantee about what happens. The compiler is free to assume UB never occurs, and to optimize based on that assumption. This leads to code that compiles without warning, runs, and yet does something completely unexpected.

Common sources of UB:

• Signed integer overflow: INT_MAX + 1 is UB (not guaranteed to wrap). The compiler may assume it never happens and eliminate seemingly-necessary bounds checks.
• Out-of-bounds array access: arr[10] on a 10-element array
• Use-after-free: accessing memory after free()
• Null pointer dereference: *NULL
• Strict aliasing violations: accessing an object through a pointer of incompatible type
• Reading uninitialized variables
• Signed left shift into the sign bit

The reason UB is especially dangerous in optimizing compilers: when you write:

if (ptr != NULL) {
    use(ptr);
    /* ... later dereference ptr assuming non-null ... */
}

If the compiler can prove that dereferencing ptr occurs on the taken branch, and null dereference is UB, then the compiler may conclude ptr != NULL is always true and eliminate the check entirely.

Tools for catching UB:

• AddressSanitizer (-fsanitize=address): detects memory errors at runtime
• UndefinedBehaviorSanitizer (-fsanitize=undefined): detects signed overflow, null deref, etc.
• Valgrind: dynamic analysis for memory errors (no recompile needed)

The Linux kernel uses -fno-strict-aliasing precisely because the kernel performs type-punning that would be UB under strict aliasing rules — and the kernel developers want predictable behavior even if it costs some optimization.

/* ub_demo.c — signed overflow UB: the compiler may eliminate the check */
#include <stdio.h>
#include <limits.h>

/* With -O2, some compilers optimize away the overflow check because
   signed overflow is UB, so the compiler assumes x+1 > x is always true */
int might_overflow(int x) {
    if (x + 1 > x)   /* UB if x == INT_MAX */
        return 1;
    return 0;
}

/* Correct: use unsigned arithmetic or explicit check */
int safe_check(int x) {
    if (x == INT_MAX)
        return 0;    /* handle overflow case explicitly */
    return (x + 1 > x) ? 1 : 0;
}

/* Compile with sanitizers to catch UB:
   gcc -fsanitize=undefined -fsanitize=address -g ub_demo.c -o ub_demo
   ./ub_demo */
int main(void) {
    printf("%d\n", might_overflow(INT_MAX));  /* may print 1 with -O2 */
    return 0;
}

Once you have more than one .c file, you need a build system to manage the compilation. The standard Unix tool is make.

A Makefile contains rules of the form:

target: dependencies
\trecipe

make checks if any dependency is newer than the target (by modification timestamp). If so, it runs the recipe. This gives you incremental compilation — change one .c file, and only that file is recompiled, then the linker relinks.

The Linux kernel uses a custom build system called Kbuild. When you build the kernel, you run make in the kernel source tree, and Kbuild orchestrates the compilation of thousands of .c files across hundreds of directories. Understanding basic make is a prerequisite for understanding why things are where they are in a kernel build.

A minimal Makefile for a multi-file C project:

CC = gcc
CFLAGS = -Wall -Wextra -O2 -g

prog: main.o utils.o
\t$(CC) $(CFLAGS) -o prog main.o utils.o

main.o: main.c utils.h
\t$(CC) $(CFLAGS) -c main.c

utils.o: utils.c utils.h
\t$(CC) $(CFLAGS) -c utils.c

clean:
\trm -f *.o prog

For kernel modules, Kbuild provides an even simpler interface — a module Makefile is often just two lines:

obj-m += mymodule.o
KDIR ?= /lib/modules/$(shell uname -r)/build
all:
\t$(MAKE) -C $(KDIR) M=$(PWD) modules

This tells Kbuild to compile mymodule.c as a loadable kernel module. The rest of the flags (-ffreestanding, -mno-red-zone, the right include paths) are all handled by Kbuild automatically. This is your first glimpse of the kernel's build infrastructure.

# Minimal working kernel module Makefile:

# --- hello_module.c ---
#include <linux/module.h>
#include <linux/init.h>

static int __init hello_init(void) {
    pr_info("Hello, kernel!\n");
    return 0;
}

static void __exit hello_exit(void) {
    pr_info("Goodbye, kernel!\n");
}

module_init(hello_init);
module_exit(hello_exit);
MODULE_LICENSE("GPL");

# --- Makefile ---
# obj-m += hello_module.o
# KDIR ?= /lib/modules/$(shell uname -r)/build
# all:
# 	$(MAKE) -C $(KDIR) M=$(PWD) modules
# clean:
# 	$(MAKE) -C $(KDIR) M=$(PWD) clean

# Build and load:
# make
# sudo insmod hello_module.ko
# dmesg | tail -1   # Hello, kernel!
# sudo rmmod hello_module