Mitigating Bufferbloat with CAKE Qdisc

A gigabit fibre connection with a ping time that spikes to 800ms the moment a large upload starts is not a bandwidth problem — it is a queueing problem. Home users routinely diagnose this as “slow WiFi” or “ISP throttling” when the actual root cause is unmanaged buffer depth in the router’s egress queue. Bufferbloat mitigation is the specific engineering discipline of controlling how packets queue at a bottleneck link so that latency-sensitive traffic (VoIP, gaming, DNS lookups) is not starved by bulk transfers (backups, torrents, video uploads). This article breaks down the architecture behind Active Queue Management (AQM), specifically the CAKE qdisc, and how to deploy it correctly on a Linux-based router or OpenWrt device.

#The Problem: Latency Under Load, Not Bandwidth Starvation

Every router interface maintains an output buffer. When the ingress rate exceeds the egress capacity — which happens constantly at the WAN interface of a home router — packets queue in that buffer rather than being dropped immediately. Modern consumer hardware, and most ISP-provided CPE, ships with oversized default buffers on the theory that dropping packets is always worse than delaying them. This is incorrect for interactive traffic.

A standard FIFO (First-In-First-Out) queue with a deep buffer will absorb a saturating TCP flow, filling to capacity and holding every subsequent packet — including your DNS query or SSH keystroke — behind megabytes of queued bulk data. This is the textbook definition of bufferbloat, first formally characterised by Jim Gettys and Kathleen Nichols. Without active bufferbloat mitigation, TCP’s own congestion control loop cannot function correctly, because the sender never sees a timely signal that the queue is full — it only sees delay, and standard loss-based TCP variants like CUBIC will keep pushing until the buffer overflows.

#Architectural Breakdown: Why FIFO Queues Fail

The core failure mode is the mismatch between buffer size and the Bandwidth-Delay Product (BDP) of the actual path. A buffer sized for a 1Gbps datacentre NIC, deployed on a 100Mbps consumer WAN link, is catastrophically oversized relative to the link’s true BDP. The fix is not simply shrinking the buffer (which reintroduces underutilisation and loss-driven throughput collapse) — it requires an intelligent queueing discipline that can distinguish flows and manage queue depth dynamically.

#The Role of AQM and CAKE Qdisc

Active Queue Management algorithms — RED, PIE, fq_codel, and finally CAKE — solve this by tracking queue sojourn time (how long a packet actually sits in the buffer) rather than raw byte count, and dropping or marking packets before the buffer saturates. CAKE (Common Applications Kept Enhanced), standardised in RFC 8290, is the current reference implementation for effective bufferbloat mitigation on Linux. It combines four architectural functions in a single qdisc:

• Shaping — enforces a hard ceiling below the physical link rate, forcing the bottleneck queue to exist inside the router (where CAKE can manage it) rather than upstream at the ISP’s uncontrolled hardware queue.
• Flow isolation — hashes packets into sub-queues by source/destination tuple (via triple-isolate, dual-srchost, or dual-dsthost keywords), ensuring one bulk flow cannot monopolise the link against dozens of smaller flows.
• Priority tiering — via Diffserv-aware tin separation, giving DSCP-marked voice/video traffic priority scheduling without a separate classifier.
• AQM (COBALT) — a Codel/BLUE hybrid that keeps the managed queue’s sojourn time near a 5ms target, dropping or ECN-marking packets early enough that TCP backs off before latency becomes visible to the user.

This is conceptually identical to the same architectural patterns used in distributed message queues — back-pressure applied early, fairness enforced per-tenant, and priority lanes for latency-sensitive payloads — just implemented at the packet scheduler layer instead of the application layer.

#Implementation Logic: Deploying SQM at the Router

Effective bufferbloat mitigation depends entirely on accurate link characterisation before any qdisc is applied. Shaping below the wrong rate either wastes throughput or fails to move the bottleneck into CAKE’s control.

#Step 1: Identify True Link Capacity

Run a saturating bidirectional test against a nearby, well-provisioned server while a ping runs concurrently. Flent (the FLExible Network Tester) is the standard tool for this because it captures latency-under-load, not just throughput.

1sudo apt install flent netperf
2flent rrul -p all_scaled -l 60 -H netperf-eu.bufferbloat.net -t "baseline-no-shaping"

Record the sustained download/upload throughput and, critically, the ping RTT increase under load. An unmanaged connection will typically show RTT inflation from a baseline of ~15ms to 300–900ms during the saturating phase — this delta is the quantifiable cost of unmitigated bufferbloat.

#Step 2: Install and Configure tc + CAKE

On a Linux router (Debian, OpenWrt, or a custom firewall build), CAKE is applied directly via tc, the traffic control subsystem. The rate must be set to roughly 92–96% of measured throughput, not the ISP’s advertised sync rate, to account for encapsulation overhead (PPPoE, DOCSIS, VLAN tagging).

1# Clear any existing qdisc on the WAN interface
2tc qdisc del dev eth1 root 2>/dev/null
3
4# Egress shaping: 45Mbit measured upload -> shape to 43Mbit
5tc qdisc add dev eth1 root cake bandwidth 43mbit 
6    diffserv4 dual-srchost nat ack-filter
7
8# Ingress shaping requires an IFB (Intermediate Functional Block) device
9ip link add ifb0 type ifb
10ip link set ifb0 up
11tc qdisc add dev eth1 handle ffff: ingress
12tc filter add dev eth1 parent ffff: matchall action mirred egress redirect dev ifb0
13tc qdisc add dev ifb0 root cake bandwidth 380mbit dual-dsthost nat

The diffserv4 keyword enables four-tier priority scheduling based on DSCP markings, nat tells CAKE to look through NAT translation when hashing flows on the router itself, and ack-filter thins redundant TCP ACKs on asymmetric links to reclaim upstream headroom.

#Code & Configurations

For OpenWrt deployments, the sqm-scripts package abstracts the raw tc invocation into a UCI configuration block, which is the recommended production path for consumer-grade hardware:

1config queue 'eth1'
2    option enabled '1'
3    option interface 'eth1'
4    option download '38000'
5    option upload '9500'
6    option qdisc 'cake'
7    option script 'piece_of_cake.qos'
8    option qdisc_advanced '1'
9    option squash_dscp '1'
10    option squash_ingress '1'
11    option ingress_ecn 'ECN'
12    option egress_ecn 'ECN'

Post-deployment, re-run the identical Flent test to validate the outcome. A correctly tuned CAKE deployment should hold RTT inflation under 30ms even at full saturation — a reduction of over 90% compared to the unmanaged FIFO baseline. Validating the change is not optional; shaping at the wrong rate is a common source of silent throughput loss.

1flent rrul -p all_scaled -l 60 -H netperf-eu.bufferbloat.net -t "cake-enabled-shaped-43mbit"
2# Compare RTT columns between the two output plots directly

#Failure Modes & Edge Cases

Bufferbloat mitigation deployments fail in predictable ways when the underlying assumptions about the link are wrong:

• Variable-rate links (DOCSIS, LTE/5G, satellite): Cable modems and cellular backhauls fluctuate their true capacity by 20–40% depending on network load and PHY-layer retransmission. A static CAKE rate set for peak throughput will under-shape during congestion windows, reintroducing bloat exactly when it matters most. Autorate variants (cake-autorate) that poll RTT continuously and adjust the shaped rate dynamically are required here.
• CGNAT and double NAT: When the ISP performs Carrier-Grade NAT upstream, the actual bottleneck queue may sit inside the ISP’s infrastructure, entirely outside the reach of CAKE running on customer premises equipment. No amount of local bufferbloat mitigation fixes a bottleneck the router doesn’t control — the only mitigation is over-provisioning the local shaped rate conservatively and accepting the residual ISP-side latency as a fixed cost.
• VPN and tunnel overhead: WireGuard or IPsec tunnels add 40–80 bytes of overhead per packet. If CAKE is applied to the physical WAN interface rather than the tunnel interface, its rate calculations ignore encapsulation overhead, causing the shaper to permit more raw bytes onto the wire than the physical link can actually clear cleanly.
• WiFi as the true bottleneck: CAKE on the WAN interface does nothing for contention occurring on the local wireless hop. The cake qdisc has a dedicated wash and per-station queueing mode intended for AP deployment (via OpenWrt’s mac80211 integration), but this is a separate configuration surface from WAN shaping and is frequently overlooked, leading to reports that “CAKE didn’t fix my gaming latency” when the actual congestion point was the 2.4GHz radio.

#Scaling & Security Trade-offs

Deploying CAKE-based bufferbloat mitigation across a fleet of routers, or at higher link speeds, introduces trade-offs that a Staff Engineer needs to model explicitly before rollout:

• CPU overhead vs. link speed: CAKE’s per-packet hashing and COBALT AQM logic are single-threaded per qdisc instance in most kernel builds. On sub-gigabit links (typical home use), overhead is negligible (<2% CPU on modern SoCs). Above ~500Mbit/s on low-power ARM routers (e.g. MT7621-based hardware), CAKE processing becomes the throughput ceiling itself, not the WAN link — multi-queue offload or a more powerful x86 router becomes necessary.
• Flow isolation vs. fairness at scale: triple-isolate gives the strongest per-flow fairness but is more exploitable by a client opening hundreds of parallel connections to claim a disproportionate share of bandwidth. dual-srchost mode enforces fairness per internal IP address instead, which is generally the correct default for a household router serving multiple devices.
• DPI vs. DSCP trust: CAKE’s diffserv4 priority tiers trust DSCP markings set by upstream applications or switches. On an untrusted network segment, a malicious or misconfigured host can mark all its traffic as highest priority (DSCP EF/CS7), defeating the fairness model. Production deployments at the network edge should re-mark or strip DSCP fields from untrusted ingress traffic before they reach the CAKE qdisc, rather than trusting client-set values outright.
• Static shaping vs. autorate complexity: Static rate configuration is simpler to audit and reason about but degrades on variable-capacity links, as noted above. Autorate scripts add a control loop with its own tuning parameters (polling interval, hysteresis thresholds) and represent additional operational surface area that must be monitored and version-controlled like any other piece of infrastructure automation.