The Link Layer & Ethernet

The physical layer delivers a continuous stream of bits — there is no inherent boundary between one "message" and the next. The link layer's first job is framing: imposing structure on this stream so that the receiver knows where one unit of data begins and ends.

The fundamental challenge is the framing problem: how does a receiver, seeing an endless sequence of 0s and 1s, know where frame boundaries are? There are three classic solutions:

1. Fixed-length frames — every frame is exactly N bits. Simple, but wasteful (short messages waste space; errors are harder to contain).
2. Length fields — include a length field in a header. The receiver reads the header to know how many bytes to expect. Ethernet uses this: the EtherType/Length field tells the receiver the payload size.
3. Sentinel/flag bytes with byte stuffing — designate a special byte (like 0x7E in PPP/HDLC) as a frame delimiter. If that byte appears in the data itself, stuff an escape byte before it. The receiver un-stuffs on receipt. This is elegant but requires per-byte processing.

Ethernet uses a fourth trick at the physical layer: the preamble. Each Ethernet frame begins with 7 bytes of alternating 1s and 0s (10101010...) followed by a Start Frame Delimiter (10101011). This preamble lets receivers synchronize their clock to the sender's bit rate and unambiguously detect the start of a frame — no length field needed for framing itself.

At the end of each Ethernet frame, the transmitter simply stops transmitting (an idle line signals end-of-frame). The minimum frame size (64 bytes) and maximum frame size (1518 bytes, the MTU — Maximum Transmission Unit) are enforced by the standard. The MTU of 1500 bytes for Ethernet payload is one of the most important constants in networking: IP fragmentation, TCP MSS negotiation, and jumbo frames (up to 9000 bytes in modern Ethernet) all exist because of it.

# Conceptual byte-stuffing (as used in PPP/HDLC, not Ethernet itself)
FLAG = 0x7E
ESCAPE = 0x7D

def stuff(data: bytes) -> bytes:
    out = bytearray([FLAG])
    for byte in data:
        if byte in (FLAG, ESCAPE):
            out.append(ESCAPE)
            out.append(byte ^ 0x20)  # XOR to avoid the reserved byte
        else:
            out.append(byte)
    out.append(FLAG)
    return bytes(out)

def unstuff(frame: bytes) -> bytes:
    out = bytearray()
    escaping = False
    for byte in frame[1:-1]:  # strip leading and trailing FLAG
        if byte == ESCAPE:
            escaping = True
        elif escaping:
            out.append(byte ^ 0x20)
            escaping = False
        else:
            out.append(byte)
    return bytes(out)

Ethernet's wire format is a masterpiece of pragmatic design. Every Ethernet frame (IEEE 802.3) has the following structure:

| Preamble (7B) | SFD (1B) | Dst MAC (6B) | Src MAC (6B) | EtherType/Len (2B) | Payload (46–1500B) | FCS (4B) |

• Preamble + SFD (8 bytes total): Clock synchronization. The 7-byte preamble (0xAA repeated = 10101010...) trains the receiver's clock. The SFD (0xAB = 10101011) marks the exact start of the frame header. These are not considered part of the frame proper.
• Destination MAC (6 bytes): Who this frame is for. If the first bit is 1, it's a multicast address; the special address FF:FF:FF:FF:FF:FF is a broadcast (all machines on the segment receive it).
• Source MAC (6 bytes): Who sent it.
• EtherType / Length (2 bytes): If the value is ≥ 1536 (0x0600), it's an EtherType identifying the Layer 3 protocol (e.g., 0x0800 = IPv4, 0x0806 = ARP, 0x86DD = IPv6). If it's ≤ 1500, it's a length field (IEEE 802.3 original). Modern Ethernet almost always uses EtherType.
• Payload (46–1500 bytes): The actual data. If the payload is shorter than 46 bytes, it is padded to reach the minimum frame size of 64 bytes (necessary for collision detection to work — more on this shortly).
• FCS — Frame Check Sequence (4 bytes): A CRC-32 checksum computed over the header and payload. The receiver recomputes the CRC and discards the frame if it doesn't match. This is error detection (not correction).

The minimum frame size of 64 bytes (excluding preamble/SFD) is not arbitrary — it is a direct consequence of collision detection geometry, as we'll explain in the CSMA/CD section.

A critical design choice: Ethernet has no TTL, no sequence number, no acknowledgment field. It is a best-effort, connectionless link. Reliability, if needed, is handled by higher layers (TCP). This simplicity is what makes hardware Ethernet NICs cheap and fast.

import struct

def parse_ethernet_frame(raw: bytes):
    # raw does NOT include preamble/SFD (stripped by NIC hardware)
    dst = raw[0:6]
    src = raw[6:12]
    ethertype = struct.unpack('!H', raw[12:14])[0]
    payload = raw[14:-4]   # strip 4-byte FCS
    fcs = raw[-4:]
    return {
        'dst': ':'.join(f'{b:02x}' for b in dst),
        'src': ':'.join(f'{b:02x}' for b in src),
        'ethertype': hex(ethertype),
        'payload_len': len(payload),
        'fcs': fcs.hex()
    }

# Example raw frame snippet (hex):
# ff ff ff ff ff ff  <- broadcast dst
# aa bb cc dd ee ff  <- src
# 08 06              <- EtherType 0x0806 = ARP

A MAC address (Media Access Control address) is a 48-bit (6-byte) identifier assigned to a Network Interface Controller (NIC). It is the link layer's answer to "who are you on this network?"

MAC addresses are written in colon-separated hex, e.g., a4:c3:f0:85:ac:2d. The structure is:

• Bits 47–24 (first 3 bytes): OUI — Organizationally Unique Identifier, assigned by IEEE to the manufacturer. For example, a4:c3:f0 belongs to Apple.
• Bits 23–0 (last 3 bytes): NIC-specific — chosen by the manufacturer to be unique within their OUI.

Two special bits in the most-significant byte:

• Bit 0 (LSB of byte 0): I/G bit — 0 = unicast (individual), 1 = multicast/broadcast.
• Bit 1: U/L bit — 0 = globally unique (burned-in from manufacturer), 1 = locally administered (you set it, as happens with virtual interfaces and VMs).

How MAC addresses differ from IP addresses: IP addresses are hierarchical and location-dependent (your IP changes when you move networks). MAC addresses are flat and ideally location-independent — they identify the hardware itself, not its place in the topology. This is exactly why routing between networks uses IP (routers look at IP), while forwarding within a network uses MAC (switches look at MAC).

MAC address lookup in the OS:

$ ip link show eth0          # Linux
$ ifconfig en0               # macOS
$ ipconfig /all             # Windows

A subtlety: MAC addresses can be changed in software (MAC spoofing) — the NIC firmware has a burned-in address, but the OS can override it. Modern privacy-preserving OSes (iOS 14+, Android 10+, macOS 12+) use randomized MAC addresses when probing for Wi-Fi networks, preventing tracking. This does not affect wired Ethernet in practice.

# Parse a MAC address and identify its properties
def analyze_mac(mac_str: str):
    bytes_ = [int(x, 16) for x in mac_str.split(':')]
    first_byte = bytes_[0]
    is_multicast = bool(first_byte & 0x01)  # I/G bit
    is_local = bool(first_byte & 0x02)      # U/L bit
    is_broadcast = all(b == 0xFF for b in bytes_)
    oui = ':'.join(f'{b:02x}' for b in bytes_[:3])
    return {
        'oui': oui,
        'is_broadcast': is_broadcast,
        'is_multicast': is_multicast,
        'is_locally_administered': is_local,
    }

print(analyze_mac('ff:ff:ff:ff:ff:ff'))  # broadcast
print(analyze_mac('01:00:5e:00:00:01'))  # IPv4 multicast
print(analyze_mac('a4:c3:f0:85:ac:2d'))  # Apple unicast

The original Ethernet was a shared medium: all machines connected to the same coaxial cable, and any transmission propagated to every machine simultaneously. This raises an immediate question: what happens when two machines transmit at the same time?

CSMA/CD — Carrier Sense Multiple Access with Collision Detection — is Ethernet's elegant answer. It has three components:

1. Carrier Sense (CS): Before transmitting, listen to the channel. If someone else is already transmitting, wait. This is like checking if someone is already talking before you speak.

2. Multiple Access (MA): Multiple stations share the channel. There is no central arbiter.

3. Collision Detection (CD): While transmitting, simultaneously listen to the channel. If what you hear differs from what you're sending, a collision has occurred. You immediately stop transmitting and send a jam signal (32 bits of a specific pattern) to ensure all stations detect the collision. Then both wait a random backoff time before retrying.

The Exponential Backoff Algorithm: After the k-th collision attempt (k = 1, 2, ..., 10), choose a random wait of r × 512 bit times where r is chosen uniformly from {0, 1, ..., 2^min(k,10) - 1}. After 16 failed attempts, report an error to the upper layer.

This is binary exponential backoff — the window doubles with each collision, spreading retransmissions out over time. It is a key example of distributed randomized load regulation with no central coordinator.

Why 64 bytes is the minimum frame size: For collision detection to work, a transmitter must still be transmitting when it hears the echo of its own collision. The worst case: two machines at the farthest ends of the cable transmit simultaneously. The signal propagates at roughly 2/3 the speed of light; on a 10BASE5 segment (500m max), propagation delay is about 2.5 μs each way, so round-trip time is ~5 μs. At 10 Mbps, a 64-byte (512-bit) frame takes 51.2 μs to transmit — comfortably longer than the 5 μs round-trip. The minimum frame size is directly derived from the physical cable length and propagation speed. This is why 100 Mbps Ethernet (Fast Ethernet) had to reduce the maximum cable length, and why gigabit Ethernet adds carrier extension bits to maintain the 64-byte collision detection guarantee.

CSMA/CD is now mostly historical — modern Ethernet uses full-duplex links with switches, eliminating shared media and collisions entirely. But the conceptual framework of distributed medium access control remains central to wireless LANs (Wi-Fi uses CSMA/CA — Collision Avoidance, since collision detection is impossible in radio).

# Binary exponential backoff simulation
import random

SLOT_TIME_MICROSECONDS = 51.2  # 512 bits at 10 Mbps
MAX_ATTEMPTS = 16

def transmit_with_backoff(frame_id: int) -> int:
    """Returns number of attempts needed, or -1 on failure."""
    for attempt in range(1, MAX_ATTEMPTS + 1):
        # Try to transmit; simulate collision with 30% probability
        if random.random() > 0.30:  # success
            return attempt
        # Collision: jam, then back off
        k = min(attempt, 10)
        r = random.randint(0, 2**k - 1)
        wait = r * SLOT_TIME_MICROSECONDS
        # (in a real NIC, this wait is in hardware)
        print(f'  Frame {frame_id}: collision #{attempt}, '
              f'backing off {wait:.1f} µs (r={r}, window=0..{2**k-1})')
    return -1  # give up after 16 attempts

result = transmit_with_backoff(frame_id=42)
print(f'Succeeded after {result} attempt(s)' if result > 0 else 'FAILED')

Every Ethernet frame ends with a 4-byte Frame Check Sequence (FCS) — a CRC-32 checksum. This is how the link layer detects garbled bits caused by noise, crosstalk, or hardware faults.

How CRC works (intuition): Treat the frame's bits as the coefficients of a polynomial over GF(2) (a binary field where addition is XOR). Divide this polynomial by a fixed generator polynomial G(x) — a standard like CRC-32 uses G(x) = x^32 + x^26 + x^23 + ... + 1. The remainder (32 bits) is the FCS.

The receiver recalculates the CRC over the received frame (including FCS). If the result is a known constant (not zero — a design choice to detect all-zero corruptions), the frame is accepted. If not, it is silently dropped. The link layer does not request retransmission — that is the job of higher layers.

Why CRC is better than a simple checksum: A CRC detects:

• All single-bit errors
• All burst errors of length ≤ 32 bits
• All odd numbers of bit errors
• About 99.9999998% of random error patterns

A simple modular sum would miss many of these, particularly pairs of bit flips that cancel each other out.

CRC is also used in: disk storage (hard drives, SSDs), USB, Bluetooth, ZIP/gzip files, and virtually every digital communication standard. It is one of the most widely deployed algorithms in existence.

Important nuance: CRC detects errors but does not correct them. Forward Error Correction (FEC) codes (e.g., Reed-Solomon, LDPC) are used when retransmission is impossible (deep-space communication, optical discs). Ethernet relies on higher-layer retransmission instead.

# CRC-32 conceptually (using Python's built-in for illustration)
import binascii

def compute_fcs(data: bytes) -> bytes:
    """Returns the 4-byte little-endian CRC-32 FCS for an Ethernet frame."""
    crc = binascii.crc32(data) & 0xFFFFFFFF
    return crc.to_bytes(4, byteorder='little')

def verify_frame(frame_with_fcs: bytes) -> bool:
    data = frame_with_fcs[:-4]
    received_fcs = frame_with_fcs[-4:]
    expected_fcs = compute_fcs(data)
    return received_fcs == expected_fcs

# Real NICs compute CRC in dedicated hardware (LFSR circuits)
# that process one bit per clock cycle at multi-GHz rates.
# The generator polynomial for CRC-32 in hex is: 0x04C11DB7

Early Ethernet used hubs — dumb repeaters that broadcast every frame to every port. Every machine heard every frame, and the shared-medium collision problem was universal. Switches replaced hubs by making forwarding intelligent: a switch only sends a frame out the specific port that leads to the destination.

How a switch works:

A switch is essentially a specialized computer with a forwarding table (also called the MAC address table or Content-Addressable Memory / CAM table) mapping MAC addresses to ports.

Self-learning algorithm (brilliantly simple):

1. When a frame arrives on port P with source MAC S: record S → P in the table (with a timestamp).
2. Look up destination MAC D in the table:
   • If found: forward only to the recorded port.
   • If not found (unknown unicast): flood — send to all ports except the incoming port.
   • If D is a broadcast or multicast: flood.
3. Entries expire after a timeout (typically 300 seconds) to handle machines that move.

This algorithm requires no configuration and no packets to a central controller. The switch builds a complete picture of the local topology purely from observing normal traffic. This is a beautiful example of emergent structure from local observations.

Why flooding unknown unicast works in practice: When machine A sends to B for the first time, B has not appeared in the table yet. The switch floods. B receives the frame and (if B was listening) B may respond — and that response teaches the switch where B is. On the very next packet from A to B, the switch knows exactly where to send it.

Collision domains vs. broadcast domains: Each switch port is its own collision domain (with full-duplex links, there are no collisions at all). But all ports on the same switch share one broadcast domain — broadcasts reach every machine. VLANs (Virtual LANs) partition a switch into multiple broadcast domains, an essential feature of real-world network design.

The CAM table overflow attack: If an attacker floods a switch with frames having random source MAC addresses, the CAM table fills up (it has finite hardware memory — typically 8K–64K entries). Once full, the switch falls back to flooding every frame, effectively becoming a hub. The attacker can now sniff all traffic. This is why enterprise switches implement port security (limit MAC addresses per port) and 802.1X authentication.

from collections import defaultdict
import time

TABLE_TIMEOUT = 300  # seconds

class EthernetSwitch:
    def __init__(self, num_ports: int):
        self.num_ports = num_ports
        # table: MAC -> (port, timestamp)
        self.cam_table: dict[str, tuple[int, float]] = {}

    def _learn(self, src_mac: str, port: int):
        self.cam_table[src_mac] = (port, time.monotonic())

    def _lookup(self, dst_mac: str) -> int | None:
        entry = self.cam_table.get(dst_mac)
        if entry is None:
            return None
        port, ts = entry
        if time.monotonic() - ts > TABLE_TIMEOUT:
            del self.cam_table[dst_mac]
            return None
        return port

    def receive_frame(self, src_mac: str, dst_mac: str,
                      in_port: int) -> list[int]:
        """Returns list of output ports to forward the frame to."""
        self._learn(src_mac, in_port)
        if dst_mac == 'ff:ff:ff:ff:ff:ff':  # broadcast
            return [p for p in range(self.num_ports) if p != in_port]
        out_port = self._lookup(dst_mac)
        if out_port is None:  # unknown unicast: flood
            return [p for p in range(self.num_ports) if p != in_port]
        return [out_port]  # known: forward only

Here is the central puzzle of the link layer: IP routing delivers packets to the right network, but the link layer needs a MAC address to construct the frame that carries the IP packet across the final hop. How does your machine find the MAC address of the machine with IP address 192.168.1.5?

ARP — Address Resolution Protocol (RFC 826, 1982) solves exactly this problem. It is elegant, stateless, and slightly insecure (which matters for understanding network attacks).

ARP Request/Reply cycle:

1. Machine A (IP: 192.168.1.10, MAC: aa:aa:aa:aa:aa:aa) wants to send a packet to B (IP: 192.168.1.5, MAC: unknown).
2. A broadcasts an ARP Request in an Ethernet frame to ff:ff:ff:ff:ff:ff: "Who has 192.168.1.5? Tell 192.168.1.10." (The ARP packet includes A's own IP and MAC so B can update its table immediately.)
3. Every machine on the segment receives the broadcast. Only B (192.168.1.5) replies with an ARP Reply (unicast, direct to A): "192.168.1.5 is at bb:bb:bb:bb:bb:bb."
4. A stores the mapping 192.168.1.5 → bb:bb:bb:bb:bb:bb in its ARP cache (typically 20 minutes expiry on Linux).
5. A constructs the Ethernet frame with dst MAC = bb:bb:bb:bb:bb:bb and transmits.

Viewing the ARP cache:

$ arp -a           # macOS / Linux
$ ip neigh show    # Linux (modern)

Gratuitous ARP: A machine can broadcast an ARP Reply about itself — announcing its IP-to-MAC mapping without being asked. Used when a machine boots or its IP changes, to update everyone's ARP cache proactively. Also used by high-availability systems when a virtual IP address moves to a new machine.

ARP Spoofing / ARP Poisoning: ARP has no authentication. Any machine can send an ARP Reply claiming any IP-to-MAC mapping. An attacker can tell machine A that "the router's IP is at my MAC" and tell the router that "A's IP is at my MAC." Now all traffic between A and the router flows through the attacker — a man-in-the-middle attack. ARP poisoning is the reason why network-level encryption (TLS, HTTPS) is non-negotiable even on local networks.

ARP only works within a single broadcast domain. If A and B are on different IP subnets, A sends the packet to its default gateway (the router), so it ARPs for the router's MAC address, not B's. The router then handles forwarding to B's network.

# Simulated ARP cache with timeout and lookup
import time

class ARPCache:
    TIMEOUT = 20 * 60  # 20 minutes

    def __init__(self):
        self._cache: dict[str, tuple[str, float]] = {}

    def learn(self, ip: str, mac: str):
        self._cache[ip] = (mac, time.monotonic())
        print(f'ARP learned: {ip} -> {mac}')

    def lookup(self, ip: str) -> str | None:
        entry = self._cache.get(ip)
        if entry is None:
            return None
        mac, ts = entry
        if time.monotonic() - ts > self.TIMEOUT:
            del self._cache[ip]
            print(f'ARP entry expired for {ip}')
            return None
        return mac

    def send_arp_request(self, my_ip: str, target_ip: str):
        # Broadcast: dst MAC = ff:ff:ff:ff:ff:ff
        print(f'ARP Request (broadcast): "Who has {target_ip}? '
              f'Tell {my_ip}"')

# Workflow:
cache = ARPCache()
cache.send_arp_request('192.168.1.10', '192.168.1.5')
# ... machine 192.168.1.5 replies ...
cache.learn('192.168.1.5', 'bb:bb:bb:bb:bb:bb')
print(cache.lookup('192.168.1.5'))  # bb:bb:bb:bb:bb:bb

The shift from hubs to switches was one of the most consequential changes in LAN history — not just for performance, but because it fundamentally changed the physics of Ethernet.

Hub (multipoint half-duplex): Every device connects to a hub. The hub repeats every signal to every port. All devices share one collision domain. If two transmit simultaneously, collision. CSMA/CD is mandatory. Bandwidth is shared: a 100 Mbps hub with 10 active senders gives each ~10 Mbps effective throughput (ideally).

Switch (point-to-point full-duplex): Each device connects to a dedicated switch port via its own cable. The switch creates a dedicated micro-segment between the device and the port. With full-duplex operation:

• The transmit and receive paths are separate (TX and RX wire pairs in the cable).
• There is no shared medium, no carrier sensing, no collision possible.
• CSMA/CD is disabled — the NIC can transmit and receive simultaneously.
• Each device gets the full link bandwidth in each direction.

Full-duplex 100BASE-TX Ethernet: Each device gets 100 Mbps outbound AND 100 Mbps inbound simultaneously = 200 Mbps aggregate per port. The switch's backplane bandwidth (internal switching capacity) is typically 2× the sum of all port rates to handle worst-case bidirectional traffic.

Flow Control (IEEE 802.3x PAUSE frames): In full-duplex Ethernet, a receiver can send a special PAUSE frame to ask the sender to stop transmitting for a specified number of slot times. This is link-layer flow control — the switch uses it to prevent buffer overflow when traffic bursts arrive faster than they can be forwarded.

Auto-negotiation (IEEE 802.3u): When a cable is plugged in, the two endpoints negotiate speed (10/100/1000/10000 Mbps) and duplex (half/full) using Fast Link Pulses (FLP). A classic misconfiguration: one end is forced to 100/full, the other auto-negotiates and falls back to 100/half — the full-duplex end floods the link while the half-duplex end detects "collisions" on every frame. This duplex mismatch causes terrible performance and is a real-world debugging trap.

Once you connect multiple switches together, you face a new and serious problem: broadcast storms.

Imagine two switches S1 and S2 connected by two cables (for redundancy — a very reasonable design). When a broadcast frame arrives at S1, S1 floods it out all ports including both links to S2. S2 receives two copies and floods them both back to S1. S1 receives them and floods again. The frame circulates forever, growing exponentially as each switch amplifies it. Within seconds, the network is saturated and all real traffic stops. This is a broadcast storm, and it is catastrophic.

Spanning Tree Protocol (STP, IEEE 802.1D) solves this by having switches collectively agree on a loop-free logical topology — a spanning tree of the physical network graph — and blocking all redundant links.

STP algorithm overview:

1. Elect a Root Bridge: All switches exchange BPDU (Bridge Protocol Data Units) messages. The switch with the lowest Bridge ID (priority + MAC) becomes the root.
2. Find root ports: Each non-root switch selects the port that leads to the root via the lowest-cost path as its Root Port.
3. Find designated ports: For each network segment, the switch with the lowest-cost path to the root becomes the Designated Bridge for that segment; its port is the Designated Port.
4. Block everything else: All remaining ports are put into Blocking state — they receive BPDUs but do not forward frames.

The result: a tree rooted at the Root Bridge, with no loops. Blocked links are kept as standby; if an active link fails, STP reconverges (50–60 seconds for classic 802.1D, 1–6 seconds for RSTP — Rapid Spanning Tree, 802.1w).

Real-world implication: STP means that in a redundantly-wired network, roughly half the physical links are always blocked and unused. This is a major inefficiency — modern data-center designs use protocols like TRILL, SPB, or ECMP with LACP (Link Aggregation) to use all links simultaneously.

A flat Ethernet network where all machines share one broadcast domain has two problems: security (every machine sees every broadcast, ARP, DHCP) and scale (with 10,000 machines, broadcast traffic becomes significant). VLANs (Virtual Local Area Networks, IEEE 802.1Q) solve both.

A VLAN is a logical partition of a switch (or set of switches) into independent broadcast domains. Ports are assigned to VLANs in configuration. A frame on VLAN 10 cannot reach a port on VLAN 20 without going through a router (Layer 3). From the network's perspective, VLANs are completely separate networks sharing the same physical hardware.

802.1Q tagging: To carry multiple VLANs over a single trunk link between switches, Ethernet frames are tagged with a 4-byte 802.1Q header inserted after the source MAC:

| ... Src MAC | 0x8100 (TPID) | TCI (3b PCP + 1b DEI + 12b VLAN ID) | EtherType | ... |

The 12-bit VLAN ID supports 4094 distinct VLANs (0 and 4095 are reserved). Access ports (connecting to end hosts) strip the tag; trunk ports (between switches) preserve it.

Why VLANs are essential in enterprise and cloud networks:

• Separate guest Wi-Fi from corporate LAN on the same physical access switch
• Isolate IoT devices from servers
• In cloud providers (AWS, GCP): each tenant's virtual machines appear to be on their own private Ethernet, even though they physically share switches with thousands of other tenants. This is implemented via VXLAN (VLAN encapsulated in UDP/IP), which provides 24-bit VLAN IDs (16 million virtual networks).