Say what dejima's notifications cost, before someone pays it #12

Merged
branderson merged 3 commits from ntfy-runbook into main 2026-08-17 18:37:58 -07:00
Collaborator

Summary generated by LLM agent

dejima's notification feature needs the devbox to reach an ntfy server, and an ntfy server on the operator's own network is one of the things rules 7 to 10 drop. Nothing in either runbook said so, and the failure is silent: the post goes nowhere, the only trace is a line in a user unit's journal, and the feature reads as switched off rather than as firewalled.

The rule that fixes it is one line, and it is the widest exception on the page. On most home networks ntfy is not a host, it is a name on the reverse proxy that fronts everything else, so its address is the proxy's address and its port is 443. These rules match an address and a port; they cannot see the Host header or the TLS SNI, which are the only things that distinguish one vhost from another. A rule entered as "allow ntfy" therefore grants every service that proxy routes, and the four DROPs stop meaning what the rest of the page says they mean. Nothing catches it afterwards: dejima-devbox-audit probes the gateway, the gateway is still dropped, and PASS keeps being reported on a devbox that can now reach the whole proxy.

So the new section in proxmox/README.md §3 states the requirement briefly and spends most of its length on that trap. It gives the two shapes that avoid it, both of which work by making ntfy a destination the firewall can actually name: a dedicated port on the proxy (simpler, and existing vhosts on 443 do not inherit it), or a dedicated address (same URL for every client, but listen 443 ssl binds the wildcard, so every other vhost answers on the new address until each one is pinned off it). There is an nginx block for the first shape, three probes to run from the VM where the third one is what actually says whether it worked, and an ntfy token scoped to publish on its own topics, which is the bound the firewall has no way to express.

The section also says where the notify service runs, because it decides what else applies: it is a systemd user service on the VM rather than something inside the container, so dejima's own egress allowlist never sees the post. There is no second list to update, and equally nothing one layer in that narrows this pinhole if it is cut too wide. That correction came back from following the section on a real box, and is the second commit here.

The rule joins the "Values that must agree" table, the exception-creep hazard points at it as the worked example, and the "Rules 3-6 are the entire remaining LAN exposure" claim now accounts for it. docs/building-a-devbox.md gets a short guest-side pointer after installing dejima, since that is where someone turning notifications on will be. proxmox/firewall.fw.template carries the rule as an optional placeholder line above the DROPs with the same warning, and its "two hosts, however many ports" line now says when it is three.

Tested with bats test/: 98 passing, including the address-leak guard (everything added is a placeholder or an example.com name) and the link and anchor checks over the new cross-references.

**Summary generated by LLM agent** dejima's notification feature needs the devbox to reach an ntfy server, and an ntfy server on the operator's own network is one of the things rules 7 to 10 drop. Nothing in either runbook said so, and the failure is silent: the post goes nowhere, the only trace is a line in a user unit's journal, and the feature reads as switched off rather than as firewalled. The rule that fixes it is one line, and it is the widest exception on the page. On most home networks ntfy is not a host, it is a name on the reverse proxy that fronts everything else, so its address is the proxy's address and its port is 443. These rules match an address and a port; they cannot see the Host header or the TLS SNI, which are the only things that distinguish one vhost from another. A rule entered as "allow ntfy" therefore grants every service that proxy routes, and the four DROPs stop meaning what the rest of the page says they mean. Nothing catches it afterwards: `dejima-devbox-audit` probes the gateway, the gateway is still dropped, and PASS keeps being reported on a devbox that can now reach the whole proxy. So the new section in `proxmox/README.md` §3 states the requirement briefly and spends most of its length on that trap. It gives the two shapes that avoid it, both of which work by making ntfy a destination the firewall can actually name: a dedicated port on the proxy (simpler, and existing vhosts on 443 do not inherit it), or a dedicated address (same URL for every client, but `listen 443 ssl` binds the wildcard, so every other vhost answers on the new address until each one is pinned off it). There is an nginx block for the first shape, three probes to run from the VM where the third one is what actually says whether it worked, and an ntfy token scoped to publish on its own topics, which is the bound the firewall has no way to express. The section also says where the notify service runs, because it decides what else applies: it is a systemd user service on the VM rather than something inside the container, so dejima's own egress allowlist never sees the post. There is no second list to update, and equally nothing one layer in that narrows this pinhole if it is cut too wide. That correction came back from following the section on a real box, and is the second commit here. The rule joins the "Values that must agree" table, the exception-creep hazard points at it as the worked example, and the "Rules 3-6 are the entire remaining LAN exposure" claim now accounts for it. `docs/building-a-devbox.md` gets a short guest-side pointer after installing dejima, since that is where someone turning notifications on will be. `proxmox/firewall.fw.template` carries the rule as an optional placeholder line above the DROPs with the same warning, and its "two hosts, however many ports" line now says when it is three. Tested with `bats test/`: 98 passing, including the address-leak guard (everything added is a placeholder or an `example.com` name) and the link and anchor checks over the new cross-references.
dejima-agent added 1 commit 2026-08-17 14:57:18 -07:00
Say what dejima's notifications cost, before someone pays it
tests / runner-env (pull_request_target) Successful in 36s
tests / tests (pull_request_target) Successful in 54s
ce445fc8f4
Notifications post to an ntfy server on the LAN, which rules 7 to 10 drop, and
they drop it silently: no error in the session, no delivery, and a feature that
reads as switched off rather than as firewalled.

The rule that fixes it is one line and it is the widest exception on the page,
because on most networks ntfy is not a host. It is a name on the reverse proxy
that fronts everything else, so a rule naming that proxy on 443 grants every
service behind it. These rules match an address and a port and cannot see the
Host header or the SNI, which are the only things that tell one vhost from
another. Nothing catches it afterwards either: the audit probes the gateway, the
gateway is still dropped, and PASS keeps being reported on a box that can now
reach the whole proxy.

So the section states the requirement and then spends most of its length on the
version of it that gives the boundary away, with the two shapes that do not (a
dedicated port, or a dedicated address with every other vhost pinned off it),
the nginx for the first, and three probes from the VM where the third is the one
that says whether it worked. Two things the firewall cannot express are named
alongside it: a publish-scoped ntfy token, and dejima's own container allowlist,
which is a separate list that also has to have the name.

The template carries the rule as an optional placeholder line above the DROPs
with the same warning, and the "two hosts" claim it makes about the remaining
LAN exposure now says when it is three.
dejima-agent added 1 commit 2026-08-17 15:12:19 -07:00
Put the notify service where it actually runs, and drop http2
tests / runner-env (pull_request_target) Successful in 34s
tests / tests (pull_request_target) Successful in 53s
1d1c09addd
Two corrections from following this section on a real box.

The notify service is a systemd user service on the VM, not something inside the
container, so the paragraph telling operators to add the name to dejima's
allowlist as well sent them to a list that never sees the post. What is worth
saying instead is the consequence: there is no second list, and nothing one
layer in narrows this pinhole the way that allowlist narrows what a session
reaches, so rule 6b's width is the whole answer. A failed post also surfaces in
a user unit's journal rather than in a session, which is a different place to
not be looking.

And the nginx block carried `http2 on;`, which is a startup error before nginx
1.25.1. The directive was spelled `listen ... ssl http2;` until then, so either
spelling breaks half the installs it is copied into and ntfy wants neither.
Dropped, with a line saying why and where to copy a working `listen` from.
dejima-agent added 1 commit 2026-08-17 15:26:13 -07:00
Drop the http2 aside
tests / tests (pull_request_target) Successful in 53s
tests / runner-env (pull_request_target) Successful in 34s
4cf688b3c2
The block does not use the directive and the section is about a firewall
pinhole. Explaining which nginx releases spell it which way asks the reader to
decide something unrelated to the rule they came here for, in the middle of the
part that matters.
branderson approved these changes 2026-08-17 18:37:52 -07:00
branderson merged commit f0b4a50dbc into main 2026-08-17 18:37:58 -07:00
Sign in to join this conversation.