Latency Cut to 12 ms on a Sky Broadband Line with QoS and a Cake Queue Discipline

On this Sky Fibre Pro connection, the idle ping to the first hop sat at 14 ms with the line syncing at 67 Mbit down and 18 Mbit up. A large upload, a cloud backup, or a video call with screen sharing pushed the same ping to 160 to 190 ms within two seconds. The loaded-line figure worth watching was the gap between those two states, because that gap made the connection feel slow even when raw throughput looked normal.

CAKE, the Common Applications Kept Enhanced queue discipline, supplied the fix. It has shipped in the Linux kernel since 4.19 and is exposed cleanly in OpenWrt through the SQM package. With the shaper set below the upload sync rate and fairness enabled across hosts and flows, one backup stream stopped crowding out voice traffic. Under full upload load, working latency moved from the 180 ms region to a steady 12 to 15 ms.

The upload queue on the Sky Hub

The Sky Hub was using a single large FIFO transmit queue with tail-drop behaviour, the common arrangement on ISP-supplied routers. When the upload filled, packets accumulated inside that buffer. TCP saw no packet loss, treated the path as if capacity still existed, and continued sending. A DNS lookup or game packet then sat behind roughly two hundred milliseconds of backup traffic before reaching the DSLAM.

The measurement came from the Waveform bufferbloat test, with the older DSLReports speed test giving the same kind of loaded-latency split for upload and download. On the untouched Sky Hub, download scored A while upload scored F, with a 170 ms increase in working latency. Download rarely bloated on this asymmetric line because the congested buffer sat upstream at the provider, outside the home router. Upload bloated inside the house, where the router owned the queue.

There was no useful gain from running CAKE against the 67 Mbit download direction on this line. The 18 Mbit upload was the drain rate the local router could control, so that was where the shaper had to sit.

The OpenWrt settings that held the line at 12 ms

The OpenWrt device sat behind the Sky Hub in modem-passthrough, with the Hub demoted to a bridge. In LuCI under Network, then SQM QoS, download was left at 0 so shaping stayed disabled in that direction. Upload was set to 17100 kbit, 95 percent of the 18000 kbit sync.

CAKE needs room below the physical rate so its own queue becomes the bottleneck before the driver ring buffer does. At 17900 kbit, the hardware buffer refilled and the latency improvement fell away. At 15000 kbit, around 3 Mbit of upload disappeared without a better latency result than the 95 percent setting. Sequential tests at 16000, 17100, and 17800 kbit showed 17100 kbit holding 12 ms, while 17800 kbit drifted toward 40 ms under a sustained upload.

The queue discipline was set to cake and the queue setup script to piece_of_cake.qos. That script applies the diffserv3 keyword and per-flow isolation, while the fairness mode used per-host plus per-flow handling so a single machine and a single transfer could not dominate the shaped upload.

Link-layer adaptation was set to ethernet with 44 bytes of overhead. That matched the fibre-to-the-cabinet PPPoE session, where encapsulation consumes part of each frame before payload is counted. An overhead miss of 20 bytes shifted the effective ceiling enough to bring back a few milliseconds of jitter during a run of full-size frames.

After applying the SQM settings, Waveform returned A+ in both directions. Upload working latency rose only 8 ms above the 12 ms idle baseline, and the screen-share call that had been dropping frames stayed stable while a 4 GB upload ran alongside it.

What the mesh added inside the house

The house used a three-node mesh: one wired gateway node and two satellites on a dedicated 5 GHz backhaul. WAN-side CAKE could clean up the ISP upload queue, yet a laptop in a back room still had to cross wireless hops before its packets reached that queue. The satellite bridge on a busy channel added 8 to 30 ms before WAN scheduling even began.

Two mesh changes lowered the internal delay. First, 802.11r fast transition was enabled across all three nodes with a shared mobility domain. A phone moving between rooms then reassociated in under 50 ms, instead of taking the 800 ms to 2 second full reauthentication path that dropped calls. Fast transition required the same SSID, the same encryption, and matching FT keys on every node. A mismatched mobility domain ID made roaming fall back silently to the slower method.

Second, the backhaul was pinned to a 5 GHz channel away from the client radios. A scan with the WiFiman app showed channels 36 and 40 crowded by two neighbouring networks, while channels 100 through 112 in the DFS range were empty. Moving the backhaul to channel 108 at 80 MHz width cut inter-node latency from 22 ms to 4 ms. DFS can force a channel change after radar detection, but three weeks of logging at this address showed no radar event.

Virgin Media, WireGuard, and DNS on the shaped path

A second comparison line ran on Virgin Media at 350 Mbit down and 35 Mbit up with the Hub 5 left in its default router mode. Its bufferbloat result was better before any tuning, with upload working latency at 30 ms, because the Hub 5 ships with a lightweight AQM. Adding an OpenWrt router downstream immediately created double NAT, which broke inbound WireGuard.

The Hub 5 does not provide true bridge mode. The workaround was modem mode: briefly hold the pinhole reset to return it to factory state, then enable modem mode in the local admin page at 192.168.100.1. That turns off routing and NAT and passes the public address through. On the Sky side, the matching arrangement was to demote the Hub to bridge duty and let the OpenWrt box hold the PPPoE session and the single public IP.

WireGuard terminated on the OpenWrt gateway and listened on UDP 51820, keeping remote access off cloud relays. The PPPoE Sky WAN MTU was 1492, and WireGuard added 60 bytes of overhead, so the tunnel interface MTU was set to 1412 to avoid fragmentation. A fragmented tunnel packet passing through the 17100 kbit shaper was counted as two packets in the queue, doubling the effective jitter for that flow.

CAKE treated the encrypted UDP tunnel as one flow in its per-flow hash and shared the shaped upload fairly with other traffic. The tunnel added 14 ms of latency on top of the underlying path, mostly from the shaper target queue delay rather than encryption cost. Throughput inside the tunnel held at 16 Mbit, inside the shaped ceiling.

A Raspberry Pi 4 ran Pi-hole as the sole DNS resolver, distributed by DHCP from the OpenWrt gateway. DNS-level blocking of tracker and ad domains removed a measurable share of outbound connections, which slightly lightened the upload queue, though the latency change was small compared with the shaper.

The case that still pulled latency upward

The 12 ms working figure held with a single upload stream and with the mixed voice-plus-backup load. With sixty simultaneous small upload flows, using a torrent swarm as the clean example, CAKE’s per-flow fairness divided the 17100 kbit ceiling so widely that each flow’s target delay accumulated and loaded latency climbed toward 25 ms.

One configuration was left untouched during the runs: raising CAKE’s target from its 5 ms default to 8 ms. Would that setting help the torrent swarm without giving back the latency gains that made the screen-share call stable?