The Application Layer: DNS & HTTP

The client-server model is the dominant architectural pattern of the Application Layer. A server is a process that listens on a well-known port for incoming requests and provides a service; a client is a process that initiates contact to consume that service. This asymmetry is fundamental: clients initiate, servers respond.

In the OSI and TCP/IP stacks, the Application Layer is where user-facing protocols live. Below it, the Transport Layer (TCP/UDP) provides communication channels — the Application Layer decides what to say over those channels. The client-server interaction happens over a socket, which is an abstraction of one endpoint of a transport-layer connection. A server bind()s to a port (e.g., 443 for HTTPS), listen()s for connections, and accept()s each incoming one. A client connect()s to the server's IP and port.

The model creates a natural separation of concerns: servers hold and manage resources (HTML files, database records, video streams); clients request and render them. This separation enabled the web to scale to billions of devices — a server doesn't need to know anything about the client's display, operating system, or locale to serve it a correct HTTP response.

A critical subtlety: servers are concurrent. A web server may be handling 10,000 simultaneous client connections. This is achieved via threads, processes, or (predominantly today) event-driven I/O (e.g., epoll on Linux, kqueue on macOS), where a single thread multiplexes many connections by reacting to readiness events rather than blocking on each one.

import socket

# Minimal TCP echo server (Python)
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as srv:
    srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    srv.bind(('', 8080))   # listen on all interfaces, port 8080
    srv.listen(5)           # backlog of 5 pending connections
    while True:
        conn, addr = srv.accept()   # blocks until a client connects
        with conn:
            data = conn.recv(1024)
            conn.sendall(data)      # echo it back

IP addresses like 172.217.14.206 are machine-friendly but human-hostile. DNS is the distributed database that maps human-readable names (www.google.com) to IP addresses — and much more. It is perhaps the most critical infrastructure on the internet: if DNS fails, the web effectively goes dark.

DNS is organized as an inverted tree of domains. The root (.) is at the top, operated by 13 logical root server clusters (e.g., a.root-servers.net). Below the root are top-level domains (TLDs): .com, .org, .edu, .uk, etc. Below those are second-level domains registered by organizations: google.com, mit.edu. Below those are subdomains: www.mit.edu, mail.google.com.

Authority is delegated down this hierarchy. ICANN manages the root. Verisign manages .com. MIT manages mit.edu. Each level publishes Name Server (NS) records that say "for questions about this subdomain, ask these servers." This delegation means no single authority needs to know all names — a query for courses.csail.mit.edu is resolved by following the chain: root → .edu servers → mit.edu servers → csail.mit.edu servers.

Record types are the fundamental data units:

• A record: name → IPv4 address
• AAAA record: name → IPv6 address
• CNAME record: name → canonical name (aliasing)
• MX record: name → mail server
• NS record: zone → authoritative name servers
• TXT record: arbitrary text (used for SPF, DKIM, domain verification)
• SOA record: Start of Authority — metadata about a zone

Each record has a TTL (Time to Live) in seconds, which controls how long resolvers may cache it. A TTL of 300 means resolvers recheck every 5 minutes; a TTL of 86400 means they cache for a day. Low TTLs cost more queries but enable faster failover; high TTLs reduce load but slow propagation.

# Using Python's dnspython to inspect DNS records
import dns.resolver

for rdata in dns.resolver.resolve('www.mit.edu', 'A'):
    print('A record:', rdata.address)

for rdata in dns.resolver.resolve('mit.edu', 'MX'):
    print('MX preference:', rdata.preference, 'server:', rdata.exchange)

for rdata in dns.resolver.resolve('mit.edu', 'NS'):
    print('NS:', rdata.target)

How does your laptop turn www.github.com into 140.82.114.4? The answer involves a beautifully choreographed conversation between multiple DNS servers.

Your machine has a stub resolver — typically just a library call like getaddrinfo(). It contacts a recursive resolver (also called a full-service resolver or recursive nameserver), which is usually provided by your ISP, your router, or a public service like 8.8.8.8 (Google) or 1.1.1.1 (Cloudflare). The recursive resolver does the hard work.

If the recursive resolver has no cache entry for www.github.com, it performs an iterative query walk:

1. Query a root server: "Who knows about .com?" → Returns NS records for .com TLD servers.
2. Query a .com TLD server: "Who knows about github.com?" → Returns NS records for GitHub's authoritative servers.
3. Query GitHub's authoritative server: "What is the A record for www.github.com?" → Returns 140.82.114.4.

The recursive resolver caches each response according to its TTL, then returns the final answer to your stub resolver. Your OS kernel also has its own DNS cache, and your browser has yet another layer. This multi-level caching is why DNS can scale to handle trillions of queries per day across the global internet.

Negative caching (RFC 2308): If a name does not exist, the resolver caches the negative result (NXDOMAIN) for the TTL in the SOA record's minimum field. This prevents thundering-herd effects when mistyped names are queried repeatedly.

The protocol itself uses UDP port 53 by default (fast, no connection overhead for small queries) and falls back to TCP port 53 for responses larger than 512 bytes or for zone transfers. DNS over HTTPS (DoH) and DNS over TLS (DoT) encrypt queries to prevent surveillance and tampering.

# Pseudocode: iterative recursive resolution
def resolve(name, qtype, resolver):
    if cache.has(name, qtype):
        return cache.get(name, qtype)
    
    # Start at root
    ns_addrs = ROOT_SERVERS
    
    while True:
        response = query(ns_addrs, name, qtype)
        
        if response.is_answer():
            cache.store(name, qtype, response, ttl=response.ttl)
            return response.answer
        
        elif response.is_referral():
            # Response has NS records pointing to next-level servers
            ns_addrs = resolve(response.ns_names, 'A', resolver)
        
        elif response.is_nxdomain():
            cache.store_negative(name, ttl=response.soa_min_ttl)
            raise NXDomainError(name)

HTTP (HyperText Transfer Protocol) is a stateless, text-based, request-response protocol. The client sends a request; the server sends a response; neither party remembers the other afterward. This statelessness is a deliberate architectural choice that makes servers massively scalable: any server in a farm can handle any request because there is no session state to route around.

An HTTP/1.1 request has three parts:

1. Request line: METHOD /path HTTP/1.1
2. Headers: key-value metadata (Host, User-Agent, Accept, Cookie, etc.)
3. Body: optional payload (for POST/PUT)

The nine HTTP methods each have precise semantics defined in RFC 7231:

• GET: Retrieve a resource. Safe (no side effects) and idempotent (repeating gives same result).
• POST: Create or submit data. Neither safe nor idempotent.
• PUT: Replace a resource at a URI. Idempotent but not safe.
• PATCH: Partial update. Neither safe nor idempotent (unless designed to be).
• DELETE: Remove a resource. Idempotent.
• HEAD: Like GET but response body is omitted (useful for checking existence/metadata).
• OPTIONS: Discover what methods the server supports (used in CORS preflight).

HTTP status codes are grouped by first digit:

• 1xx: Informational (100 Continue)
• 2xx: Success (200 OK, 201 Created, 204 No Content)
• 3xx: Redirection (301 Moved Permanently, 302 Found, 304 Not Modified)
• 4xx: Client Error (400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 429 Too Many Requests)
• 5xx: Server Error (500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable)

Persistent connections (HTTP/1.1 default with Connection: keep-alive) allow multiple requests over a single TCP connection, avoiding the 3-way handshake overhead. Pipelining (send multiple requests without waiting for responses) was specified but widely disabled due to head-of-line blocking: if request #1 gets a slow response, requests #2 and #3 are stuck waiting even if their responses are ready.

# Raw HTTP/1.1 request/response over a socket
import socket

def http_get(host, path='/'):
    with socket.create_connection((host, 80)) as s:
        request = (
            f'GET {path} HTTP/1.1\r\n'
            f'Host: {host}\r\n'
            'Connection: close\r\n'
            '\r\n'
        )
        s.sendall(request.encode())
        response = b''
        while chunk := s.recv(4096):
            response += chunk
    return response.decode(errors='replace')

# The response looks like:
# HTTP/1.1 200 OK
# Content-Type: text/html; charset=utf-8
# Content-Length: 12345
# 
# <!DOCTYPE html>...

HTTP is stateless at the protocol level, but the web needs state — shopping carts, login sessions, user preferences. Two mechanisms bridge this gap.

Cookies (RFC 6265) are small key-value strings the server sends in a Set-Cookie response header. The browser stores them and sends them back in a Cookie request header on subsequent requests to the same origin. Cookies carry sessions, A/B test assignments, and tracking identifiers. Security-critical attributes:

• HttpOnly: JavaScript cannot access the cookie (mitigates XSS theft).
• Secure: Only sent over HTTPS.
• SameSite=Strict/Lax/None: Controls cross-site sending (mitigates CSRF).

HTTP Caching is one of the most powerful and frequently misunderstood features of HTTP. The goal is to avoid retransmitting unchanged content. Key headers:

Cache-Control: max-age=3600 — the response is fresh for 3600 seconds; the browser serves it from cache without hitting the server.

Cache-Control: no-cache — confusingly named: the browser must revalidate with the server before using a cached copy (but if the server says "not modified", no body is retransmitted).

Cache-Control: no-store — never cache at all (use for sensitive data).

Conditional requests enable efficient revalidation. The server sends ETag: "abc123" (a fingerprint of the content) or Last-Modified: Wed, 21 Oct 2023 07:28:00 GMT. On revalidation, the browser sends If-None-Match: "abc123" or If-Modified-Since: .... If the content is unchanged, the server responds with 304 Not Modified and an empty body — saving bandwidth while keeping the cache fresh.

Caching is hierarchical: browser cache → CDN edge cache → origin server. A CDN (Content Delivery Network) like Cloudflare or Fastly caches responses at dozens of geographically distributed nodes, serving users from the nearest one. The Vary header tells caches to store separate copies for different request variants (e.g., Vary: Accept-Encoding means cache separate gzipped and uncompressed copies).

# Server response headers for a cacheable, versioned asset
HTTP/1.1 200 OK
Content-Type: application/javascript
Content-Length: 45231
Cache-Control: public, max-age=31536000, immutable
ETag: "v3.2.1-a8f2c9"
Content-Encoding: gzip

# For a dynamic page that must revalidate:
HTTP/1.1 200 OK
Content-Type: text/html
Cache-Control: no-cache
ETag: "user-42-dashboard-1718000000"

# Revalidation request from browser:
GET /dashboard HTTP/1.1
Host: example.com
If-None-Match: "user-42-dashboard-1718000000"

# Server response if unchanged:
HTTP/1.1 304 Not Modified
ETag: "user-42-dashboard-1718000000"

HTTP/1.1 has a fundamental performance bottleneck: head-of-line (HOL) blocking at the application layer. A browser opening a typical webpage needs 50–100 resources (HTML, CSS, JS, images, fonts). With HTTP/1.1, each TCP connection carries requests serially — while request #1's large response is in flight, requests #2 through #100 wait. Browsers work around this with 6 parallel TCP connections per domain, but that's wasteful (6 TCP handshakes, 6 congestion-control windows) and still inadequate.

HTTP/2 (RFC 7540, 2015) reimagines HTTP at the framing level:

Binary framing: HTTP/2 replaces the text-based line format with a binary frame format. Every interaction is split into frames (minimum unit: a header block or a data chunk), each tagged with a stream ID. Frames from different streams are interleaved on a single TCP connection.

Multiplexing: Multiple concurrent request/response streams share one TCP connection. Stream 1 carries request A; stream 3 carries request B; stream 5 carries request C — all interleaved on the wire. This eliminates application-layer HOL blocking. The browser opens exactly one TCP connection per origin.

Header compression (HPACK): HTTP headers are verbose and repetitive — User-Agent, Accept-Encoding, and Cookie headers repeat on every request. HPACK uses static and dynamic Huffman-coded tables to compress headers dramatically, sometimes by 85–90%.

Server push: The server can proactively send resources the client hasn't asked for yet (e.g., push CSS and JS when the HTML is requested). In practice this feature proved hard to use correctly and was deprecated in HTTP/3.

Stream prioritization: Streams can declare dependencies and weights, letting the browser signal "send the CSS before the images." This is also being redesigned in newer work (RFC 9218).

HTTP/2 is negotiated via ALPN (Application-Layer Protocol Negotiation) during the TLS handshake — effectively free, no extra round trip. Nearly all HTTP/2 in practice runs over TLS.

# HTTP/2 is binary; you don't write frames manually.
# Here's how to use it in Python with httpx:
import httpx
import asyncio

async def fetch_many(urls):
    # httpx uses HTTP/2 multiplexing automatically
    async with httpx.AsyncClient(http2=True) as client:
        tasks = [client.get(url) for url in urls]
        responses = await asyncio.gather(*tasks)
        for r in responses:
            print(r.http_version, r.status_code, r.url)

# All requests fly over a single TLS+TCP connection,
# multiplexed as separate HTTP/2 streams.
urls = [
    'https://www.example.com/style.css',
    'https://www.example.com/app.js',
    'https://www.example.com/logo.png',
]
asyncio.run(fetch_many(urls))

HTTP/2 solves application-layer HOL blocking but introduces a subtler problem: transport-layer HOL blocking. HTTP/2 streams share a single TCP connection. TCP guarantees in-order delivery of bytes — if one packet is lost, TCP halts delivery of all subsequent packets, including those for completely independent HTTP/2 streams, until the lost packet is retransmitted. This TCP retransmission penalty hits HTTP/2 harder than HTTP/1.1 (which used multiple connections) on lossy networks like WiFi and cellular.

HTTP/3 (RFC 9114, 2022) solves this by replacing TCP with QUIC (RFC 9000), a new transport protocol built on top of UDP. QUIC rebuilds all of TCP's reliability, congestion control, and ordering guarantees — but does so at the stream level, not the connection level. A lost packet only blocks the stream it belongs to; other streams proceed unimpeded.

QUIC's major advantages over TCP+TLS:

1. 0-RTT connection establishment: On a repeat connection, QUIC can send application data in the first packet (0 round trips), versus TCP's 1 RTT for the handshake plus 1 RTT for TLS 1.3 — a saving of 1–2 RTTs that matters enormously on mobile.

2. Stream-level loss recovery: No transport-layer HOL blocking between streams.

3. Connection migration: QUIC connections are identified by a Connection ID, not a (source IP, source port, dest IP, dest port) 4-tuple. When you switch from WiFi to cellular, the connection continues without interruption — the server recognizes the same Connection ID.

4. Built-in encryption: QUIC mandates TLS 1.3 internally. There is no unencrypted QUIC.

5. Faster iteration: QUIC runs in userspace, so new versions deploy via software updates, not OS kernel updates.

As of 2025, roughly 25–30% of all HTTP traffic globally uses HTTP/3, including all of Google, Meta, and Cloudflare's properties.

# QUIC connection establishment (simplified pseudocode)
# comparing TCP+TLS1.3 vs QUIC

# TCP + TLS 1.3 (new connection): 2 RTTs before data
# RTT 1: TCP SYN / SYN-ACK / ACK
# RTT 2: TLS ClientHello / ServerHello+Cert / Finished
# RTT 3: First application data

# QUIC (new connection): 1 RTT before data
# RTT 1: QUIC Initial (contains TLS ClientHello)
#         Server: QUIC Handshake (TLS ServerHello+Cert+Finished)
# RTT 2: Client sends Finished + FIRST APPLICATION DATA

# QUIC (repeat connection with 0-RTT):
# RTT 0: Client sends ClientHello + application data immediately
#         (uses session ticket from previous connection)
# NOTE: 0-RTT data is not replay-safe; only use for idempotent GETs

REST (Representational State Transfer) is not a protocol — it is an architectural style defined by Roy Fielding in his 2000 PhD dissertation. Fielding was describing the properties of the web itself and asking: what architectural constraints make a distributed hypermedia system scalable, evolvable, and visible?

REST defines six constraints:

1. Client-Server: Separation of UI concerns from data storage concerns. The client and server evolve independently.

2. Stateless: Every request from client to server must contain all information necessary to understand the request. Session state is kept on the client (or passed in every request via tokens). This enables horizontal scaling — any server instance can handle any request.

3. Cache: Responses must label themselves as cacheable or non-cacheable (via Cache-Control). Cacheability improves efficiency and scalability.

4. Uniform Interface: The central REST constraint. It has four sub-constraints:

   • Resource identification in requests: URIs identify resources (/users/42, /orders/99).
   • Resource manipulation through representations: Clients manipulate resources via representations (JSON, XML, HTML) — not via stored procedures or commands.
   • Self-descriptive messages: Each message includes enough information to describe how to process it (Content-Type: application/json).
   • HATEOAS (Hypermedia as the Engine of Application State): Responses contain links to valid next actions. Clients discover capabilities dynamically. (This constraint is widely ignored in practice.)

5. Layered System: The client cannot tell whether it's connected directly to the origin server or to a CDN, load balancer, or cache. Enables composition.

6. Code on Demand (optional): Servers can extend client functionality by transferring executable code (JavaScript). The only optional REST constraint.

In practice, RESTful APIs design resources as nouns in URIs and use HTTP methods as verbs. GET /users/42 retrieves user 42. PUT /users/42 replaces it. PATCH /users/42 partially updates it. DELETE /users/42 removes it. POST /users creates a new user. A key insight: the URI identifies the resource, not the action. /deleteUser?id=42 is not RESTful; DELETE /users/42 is.

# A RESTful API in Python (Flask)
from flask import Flask, request, jsonify, abort

app = Flask(__name__)
users = {1: {'name': 'Alice', 'email': 'alice@example.com'}}
next_id = 2

@app.get('/users/<int:uid>')
def get_user(uid):
    user = users.get(uid)
    if not user:
        abort(404)
    # Self-descriptive: status code 200, JSON body
    return jsonify(user), 200

@app.post('/users')
def create_user():
    global next_id
    data = request.get_json()
    users[next_id] = data
    # 201 Created + Location header pointing to new resource
    resp = jsonify(data)
    resp.headers['Location'] = f'/users/{next_id}'
    next_id += 1
    return resp, 201

@app.delete('/users/<int:uid>')
def delete_user(uid):
    if uid not in users:
        abort(404)
    del users[uid]
    return '', 204   # 204 No Content: success, no body

HTTP by itself is transmitted in plaintext. Any router, ISP, or attacker on-path can read your passwords, cookies, and credit card numbers. HTTPS is HTTP running over TLS (Transport Layer Security), which provides three guarantees:

1. Confidentiality: Symmetric encryption (AES-GCM) prevents eavesdropping.
2. Integrity: HMAC/AEAD prevents tampering with the data in transit.
3. Authentication: The server presents a certificate signed by a trusted Certificate Authority (CA), proving it is who it claims to be (prevents impersonation).

The TLS 1.3 handshake (RFC 8446, 2018) is elegantly minimal:

• Client → Server: ClientHello with supported cipher suites and a Diffie-Hellman key share.
• Server → Client: ServerHello with chosen cipher, its DH key share, certificate, and Finished — all in one flight.
• Client → Server: Verifies certificate, sends Finished, then immediately sends application data.

Total cost: 1 RTT for a new connection. (TLS 1.2 required 2 RTTs.)

Certificate chains: A server certificate is signed by an intermediate CA, which is signed by a root CA. Root CAs are pre-installed in your OS/browser trust store (about 150 root CAs globally). The server sends the full chain; the client verifies it bottom-up.

HSTS (HTTP Strict Transport Security): Strict-Transport-Security: max-age=31536000; includeSubDomains instructs the browser to always use HTTPS for this domain for the next year, even if the user types http://. This prevents SSL stripping attacks.

Certificate Transparency (CT): All publicly trusted certificates must be logged to append-only public logs (RFC 9162). Browsers check for SCTs (Signed Certificate Timestamps) to detect misissued certificates. This is how researchers discover when CAs are compromised.

# TLS 1.3 handshake (pseudocode)

# --- Client sends ---
ClientHello:
  random: 32 bytes
  supported_versions: [TLS 1.3, TLS 1.2]
  cipher_suites: [TLS_AES_128_GCM_SHA256, TLS_CHACHA20_POLY1305_SHA256]
  key_share: (x25519, client_public_key)   # DH share

# --- Server sends (all in one flight) ---
ServerHello:
  random: 32 bytes
  chosen_cipher: TLS_AES_128_GCM_SHA256
  key_share: (x25519, server_public_key)

# Derive handshake secret = HKDF(DH_shared_secret, ...)
EncryptedExtensions: { }           # encrypted with handshake key
Certificate: { server_cert_chain } # encrypted
CertificateVerify: { sig }         # proves server has private key
Finished: { HMAC of transcript }   # authenticates the handshake

# --- Client sends ---
Finished: { HMAC of transcript }
[Application data begins immediately]

Now we can trace the complete lifecycle of a web request. When you type https://www.example.com/index.html and press Enter:

Step 1 — Browser cache check: The browser checks its cache for www.example.com's IP address. If the DNS TTL hasn't expired, it skips DNS entirely.

Step 2 — DNS resolution: The stub resolver calls the OS DNS API. The OS checks /etc/hosts, then its cache, then queries the configured recursive resolver (e.g., 1.1.1.1). The resolver performs the iterative walk (root → TLD → authoritative), returns A record 93.184.216.34, and caches it.

Step 3 — TCP connection: The browser opens a TCP connection to 93.184.216.34:443. SYN → SYN-ACK → ACK (1 RTT).

Step 4 — TLS handshake: TLS 1.3 handshake (1 RTT). The browser verifies the server's certificate chain, derives session keys.

Step 5 — HTTP request: If HTTP/2 was negotiated via ALPN, the browser sends an HTTP/2 HEADERS frame for GET /index.html. The server responds with a HEADERS frame (status 200) followed by DATA frames with the HTML body.

Step 6 — HTML parsing and subresource discovery: The browser's HTML parser encounters <link rel='stylesheet' href='/style.css'> and <script src='/app.js'>. It immediately sends HTTP/2 requests for these on new streams of the same connection — true multiplexing.

Step 7 — Caching: If the response included Cache-Control: max-age=86400, the browser stores it. Next visit, it sends If-None-Match: "abc" and gets a 304 Not Modified — saving the entire response body.

Step 8 — Rendering: CSS is parsed into a CSSOM, HTML into a DOM, they are combined into a Render Tree, laid out, and painted to the screen.

Total time for repeat visit on fast connection: ~50ms DNS (if cached: 0ms), ~30ms TCP, ~30ms TLS (0ms with QUIC 0-RTT), ~50ms for first HTTP response. The network round-trip time dominates.