Bypass cannot cover forwarded traffic: container VPNs are silently double-tunnelled #116

Closed
opened 2026-08-21 13:15:17 -04:00 by mysticalsoap · 2 comments
Owner

The 0.0.0.0/1 + 128.0.0.0/1 override captures forwarded packets as readily as host-originated ones, but bypass is expressed through cgroup tagging, which only applies to local sockets. Forwarded traffic has no socket to tag, so there is no way to exclude it — it goes through the tunnel with no opt-out.

Observed

media/gluetun runs its own ProtonVPN WireGuard tunnel (kernel implementation, peer 163.5.171.2:51820) and shares its netns with qbittorrent. With aqomui connected, a 100MB download inside that netns:

Interface Delta
gluetun eth0 111 MB — the transfer
host tun_aqomui 118 MB — the same bytes again

Already-encrypted WireGuard, re-encrypted inside aqomui's tunnel. Routing confirms it:

# ip route get 163.5.171.2 from 172.18.7.4 iif br-8de0ab92e4e0
163.5.171.2 from 172.18.7.4 via 10.96.0.1 dev tun_aqomui

aqomui pins its own endpoint out the physical link (205.142.240.210 via 192.168.0.1 dev enp5s0) so its tunnel doesn't loop into itself. Nothing does the equivalent for a container's VPN endpoint.

Why this is wrong rather than merely wasteful

gluetun exists so qbittorrent always has a tunnel even when aqomui is disconnected. It is a floor, not a layer. When aqomui is up that guarantee is already met, so the second wrapper buys nothing and costs double encryption plus MTU stacking (gluetun's tun0 is 1320, its outer WireGuard adds ~80, tun_aqomui wraps again).

The real damage is coupling. gluetun holds a NAT-PMP port forward that private trackers require, and its tunnel transport now runs through aqomui — so connecting or dropping the host VPN disturbs the forward. Two tunnels that exist to be independent are serialised, and the dependency runs the wrong way.

Validated workaround

Table 11 already does exactly the right thing; the traffic just never reaches it:

# ip route get 163.5.171.2 from 172.18.7.4 iif br-8de0ab92e4e0 mark 0xb
163.5.171.2 from 172.18.7.4 via 192.168.0.1 dev enp5s0 table 11 mark 11

So a mangle PREROUTING rule (before the routing decision — FORWARD is too late for policy routing) restores the intent:

iptables -t mangle -A PREROUTING -s 172.18.7.0/24 -p udp --dport 51820 -j MARK --set-mark 0xb

Matching on the WireGuard port rather than the peer address keeps it stable across ProtonVPN server rotation, and keeps the scope to tunnel transport only — the other containers on that subnet still egress through aqomui as before, so indexer traffic doesn't quietly lose its cover.

Degrades safely: with aqomui disconnected the mark finds no rule, falls through to main, and exits via the physical default regardless.

Proposed fix

Bypass needs a way to express non-socket traffic. Options worth weighing:

  • Bypass entries by source subnet or interface, alongside the existing app list — the general fix, and it covers LAN-forwarded traffic too, not just containers
  • Auto-pin detected container VPN endpoints the way aqomui already pins its own
  • At minimum, document that forwarded traffic is captured and cannot currently be excluded

Blocks QoS classification in mysticalsoap/docker#156: while everything is sealed inside one outer tunnel on enp5s0, a shaper cannot distinguish torrent traffic from anything else, whatever DSCP marks are applied inside.

The `0.0.0.0/1` + `128.0.0.0/1` override captures **forwarded** packets as readily as host-originated ones, but bypass is expressed through cgroup tagging, which only applies to local sockets. Forwarded traffic has no socket to tag, so there is no way to exclude it — it goes through the tunnel with no opt-out. ## Observed `media/gluetun` runs its own ProtonVPN WireGuard tunnel (kernel implementation, peer `163.5.171.2:51820`) and shares its netns with qbittorrent. With aqomui connected, a 100MB download inside that netns: | Interface | Delta | |---|---| | gluetun `eth0` | 111 MB — the transfer | | host `tun_aqomui` | **118 MB** — the same bytes again | Already-encrypted WireGuard, re-encrypted inside aqomui's tunnel. Routing confirms it: ``` # ip route get 163.5.171.2 from 172.18.7.4 iif br-8de0ab92e4e0 163.5.171.2 from 172.18.7.4 via 10.96.0.1 dev tun_aqomui ``` aqomui pins its own endpoint out the physical link (`205.142.240.210 via 192.168.0.1 dev enp5s0`) so its tunnel doesn't loop into itself. Nothing does the equivalent for a *container's* VPN endpoint. ## Why this is wrong rather than merely wasteful gluetun exists so qbittorrent always has a tunnel even when aqomui is disconnected. It is a floor, not a layer. When aqomui is up that guarantee is already met, so the second wrapper buys nothing and costs double encryption plus MTU stacking (gluetun's `tun0` is 1320, its outer WireGuard adds ~80, `tun_aqomui` wraps again). The real damage is coupling. gluetun holds a NAT-PMP port forward that private trackers require, and its tunnel transport now runs through aqomui — so connecting or dropping the host VPN disturbs the forward. Two tunnels that exist to be independent are serialised, and the dependency runs the wrong way. ## Validated workaround Table 11 already does exactly the right thing; the traffic just never reaches it: ``` # ip route get 163.5.171.2 from 172.18.7.4 iif br-8de0ab92e4e0 mark 0xb 163.5.171.2 from 172.18.7.4 via 192.168.0.1 dev enp5s0 table 11 mark 11 ``` So a mangle **PREROUTING** rule (before the routing decision — FORWARD is too late for policy routing) restores the intent: ``` iptables -t mangle -A PREROUTING -s 172.18.7.0/24 -p udp --dport 51820 -j MARK --set-mark 0xb ``` Matching on the WireGuard port rather than the peer address keeps it stable across ProtonVPN server rotation, and keeps the scope to tunnel transport only — the other containers on that subnet still egress through aqomui as before, so indexer traffic doesn't quietly lose its cover. Degrades safely: with aqomui disconnected the mark finds no rule, falls through to `main`, and exits via the physical default regardless. ## Proposed fix Bypass needs a way to express non-socket traffic. Options worth weighing: - Bypass entries by **source subnet or interface**, alongside the existing app list — the general fix, and it covers LAN-forwarded traffic too, not just containers - Auto-pin detected **container VPN endpoints** the way aqomui already pins its own - At minimum, document that forwarded traffic is captured and cannot currently be excluded Blocks QoS classification in `mysticalsoap/docker#156`: while everything is sealed inside one outer tunnel on `enp5s0`, a shaper cannot distinguish torrent traffic from anything else, whatever DSCP marks are applied inside.
Author
Owner

Direction settled: option 1 (bypass entries by source subnet), plus documenting the limitation. Option 2 is rejected.

Why option 1: it reuses the machinery that already exists. Entries become

iptables -t mangle -A PREROUTING -s <cidr> [-p <proto> --dport <port>] -j MARK --set-mark 0xb

steering into table 11 through the same fwmark rule the cgroup bypass uses, with the same teardown and the same safe degradation (aqomui down → mark matches no rule → falls through to main). The mark must be set in PREROUTING — the routing decision sits between PREROUTING and FORWARD, so FORWARD is too late for policy routing.

Entries need optional protocol/port qualifiers, not just a CIDR: a bare subnet entry exempts everything from that subnet, while the container-VPN case only wants the tunnel transport exempted (udp --dport 51820 here) so other containers on the same bridge keep their cover. Matching the port rather than the peer address also survives VPN server rotation.

Why not option 2 (auto-detect container VPN endpoints): detection means watching traffic or conntrack for VPN-shaped flows — a watcher, the failure class this fork exists to avoid. An entry the user states once fits the mark-once preference model from #67; magic detection does not.

Remaining design question before implementation: the config and GUI surface — where network entries live in config.json and how the bypass tab presents them alongside app entries.

Direction settled: option 1 (bypass entries by source subnet), plus documenting the limitation. Option 2 is rejected. **Why option 1:** it reuses the machinery that already exists. Entries become ``` iptables -t mangle -A PREROUTING -s <cidr> [-p <proto> --dport <port>] -j MARK --set-mark 0xb ``` steering into table 11 through the same fwmark rule the cgroup bypass uses, with the same teardown and the same safe degradation (aqomui down → mark matches no rule → falls through to main). The mark must be set in PREROUTING — the routing decision sits between PREROUTING and FORWARD, so FORWARD is too late for policy routing. Entries need optional protocol/port qualifiers, not just a CIDR: a bare subnet entry exempts everything from that subnet, while the container-VPN case only wants the tunnel transport exempted (`udp --dport 51820` here) so other containers on the same bridge keep their cover. Matching the port rather than the peer address also survives VPN server rotation. **Why not option 2 (auto-detect container VPN endpoints):** detection means watching traffic or conntrack for VPN-shaped flows — a watcher, the failure class this fork exists to avoid. An entry the user states once fits the mark-once preference model from #67; magic detection does not. Remaining design question before implementation: the config and GUI surface — where network entries live in config.json and how the bypass tab presents them alongside app entries.
Author
Owner

Measured the speed effect (2026-08-22, Cloudflare 50MB single-stream, 3+ runs per leg, ~1 Gbps line):

Path Speed
Host raw (bypass cgroup) ~85 MB/s
Host via aqomui (OpenVPN) ~35-40 MB/s
Container VPN double-tunnelled ~29 MB/s
Container VPN direct (mark rule active) ~55-65 MB/s

So the double tunnel roughly halves the container VPN's throughput, and most of that cost is the outer OpenVPN tunnel's own ceiling (~37 MB/s) rather than the second encryption layer per se — the inner WireGuard inherits the outer tunnel's throughput cap.

Also confirms the workaround does not persist: the manually added mangle rule from the original report was gone within a day (service firewall teardown or reboot), and the double tunnel silently returned — verified by interface counters (113 MiB on tun_aqomui RX during a 100 MB in-container download) before re-applying the rule (0 MiB after). A user-visible, service-owned rule that survives restarts is part of the feature, not a nicety.

Measured the speed effect (2026-08-22, Cloudflare 50MB single-stream, 3+ runs per leg, ~1 Gbps line): | Path | Speed | |---|---| | Host raw (bypass cgroup) | ~85 MB/s | | Host via aqomui (OpenVPN) | ~35-40 MB/s | | Container VPN double-tunnelled | ~29 MB/s | | Container VPN direct (mark rule active) | ~55-65 MB/s | So the double tunnel roughly halves the container VPN's throughput, and most of that cost is the outer OpenVPN tunnel's own ceiling (~37 MB/s) rather than the second encryption layer per se — the inner WireGuard inherits the outer tunnel's throughput cap. Also confirms the workaround does not persist: the manually added mangle rule from the original report was gone within a day (service firewall teardown or reboot), and the double tunnel silently returned — verified by interface counters (113 MiB on tun_aqomui RX during a 100 MB in-container download) before re-applying the rule (0 MiB after). A user-visible, service-owned rule that survives restarts is part of the feature, not a nicety.
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
mysticalsoap/aqomui#116
No description provided.