Record the devbox host key, and name both machines in alerts #9

2026-08-28T20:17:25-07:00

dejima-agent commented

2026-08-28 20:17:25 -07:00

Summary generated by LLM agent

Fixes the "could not connect to the devbox" alert that arrives even when the key, the forced command and the alert path all work.

Nothing ever wrote known_hosts

StrictHostKeyChecking is left on deliberately, and BatchMode turns its prompt into a failure. So a guard whose known_hosts is empty cannot connect at all, and reports the devbox unreachable at max priority on every cycle, forever. That is the false alarm that gets a topic muted.

A fresh container is exactly that state. The code said so in a comment and pointed the setting at a volume where it would persist, but nothing in this repository ever wrote to it. dejima-guard-install-key connects as the operator, so it populates the operator's own ~/.ssh/known_hosts, on a different account and usually a different filesystem.

trust-host writes it. It stays a separate command rather than something check does on failure: learning a host key automatically, on the one connection whose purpose is noticing that machine changed, would defeat the check it exists to support. It prints the fingerprint, states plainly that trust on first use verified nothing, accepts --fingerprint SHA256:... to check before writing, and refuses to replace a key that has changed rather than deciding for you whether that was a rebuild or an intrusion.

The reason was being thrown away

check sent ssh's stderr to /dev/null and the alert offered three guesses instead, on the one failure that stops the program working. It is kept now, logged on the guard and deliberately never put in the push, since it is the one string near an alert body that the far end has any say in. A host key failure additionally names the command that fixes it, because it is not "the box is down" and it never recovers on its own.

Naming

"The watcher could not collect from the devbox" names neither machine, and an operator watching two boxes cannot tell the pushes apart. The guard is dejima-guard; the subject is the configured host. Titles use the short name, which is what survives truncation on a lock screen, and bodies use the full one. Both come from this machine's config rather than from the watched host, which is what keeps them safe to interpolate.

Found while testing this

Fingerprinting sat at the head of a pipeline, so a key ssh-keygen could not read took the whole command down under pipefail: exit 255, no message, after the header and before the write. And an existing known_hosts entry whose key material would not parse produced an empty set, which compared equal to everything and reported the host already trusted.

Tested: 58 passing. trust-host was also run against a real SSH host for all four paths (first use, idempotent re-run, matching and non-matching --fingerprint, and a changed key).

**Summary generated by LLM agent** Fixes the "could not connect to the devbox" alert that arrives even when the key, the forced command and the alert path all work. ## Nothing ever wrote known_hosts `StrictHostKeyChecking` is left on deliberately, and `BatchMode` turns its prompt into a failure. So a guard whose `known_hosts` is empty cannot connect at all, and reports the devbox unreachable at max priority on every cycle, forever. That is the false alarm that gets a topic muted. A fresh container is exactly that state. The code said so in a comment and pointed the setting at a volume where it would persist, but nothing in this repository ever wrote to it. `dejima-guard-install-key` connects as the operator, so it populates the operator's own `~/.ssh/known_hosts`, on a different account and usually a different filesystem. `trust-host` writes it. It stays a separate command rather than something `check` does on failure: learning a host key automatically, on the one connection whose purpose is noticing that machine changed, would defeat the check it exists to support. It prints the fingerprint, states plainly that trust on first use verified nothing, accepts `--fingerprint SHA256:...` to check before writing, and refuses to replace a key that has changed rather than deciding for you whether that was a rebuild or an intrusion. ## The reason was being thrown away `check` sent ssh's stderr to `/dev/null` and the alert offered three guesses instead, on the one failure that stops the program working. It is kept now, logged on the guard and deliberately never put in the push, since it is the one string near an alert body that the far end has any say in. A host key failure additionally names the command that fixes it, because it is not "the box is down" and it never recovers on its own. ## Naming "The watcher could not collect from the devbox" names neither machine, and an operator watching two boxes cannot tell the pushes apart. The guard is `dejima-guard`; the subject is the configured host. Titles use the short name, which is what survives truncation on a lock screen, and bodies use the full one. Both come from this machine's config rather than from the watched host, which is what keeps them safe to interpolate. ## Found while testing this Fingerprinting sat at the head of a pipeline, so a key `ssh-keygen` could not read took the whole command down under `pipefail`: exit 255, no message, after the header and before the write. And an existing `known_hosts` entry whose key material would not parse produced an empty set, which compared equal to everything and reported the host already trusted. Tested: 58 passing. `trust-host` was also run against a real SSH host for all four paths (first use, idempotent re-run, matching and non-matching `--fingerprint`, and a changed key).

dejima-agent added 1 commit 2026-08-28 20:17:26 -07:00

Record the devbox host key, and name both machines in alerts 01be9885ef

Nothing ever wrote the guard's known_hosts. StrictHostKeyChecking is left on by
design and BatchMode turns its prompt into a failure, so an empty file means
every cycle reports the devbox unreachable at max priority - permanently, which
is the false alarm that gets a topic muted. A fresh container is exactly that
state. The code described where the file should live and why it must persist;
nothing put anything in it. dejima-guard-install-key connects as the operator,
so it uses the operator's own known_hosts on a different account.

trust-host writes it, and stays a separate command: learning a key
automatically on the connection whose purpose is noticing that machine changed
would defeat the check it supports. It prints the fingerprint, says plainly
that trust on first use verified nothing, takes --fingerprint to check before
writing, and refuses to replace a key that has changed rather than choosing
between a rebuild and an intrusion.

check no longer sends ssh's stderr to /dev/null. The one failure that stops
this program working arrived with its reason already discarded and three
guesses in its place. The reason is logged here and never put in the push,
since it is the one string near an alert the far end has any say in. A host key
failure names the command that fixes it.

Alerts name both machines. "The watcher could not collect from the devbox"
identifies neither, and an operator watching two boxes cannot tell the pushes
apart. Titles use the short hostname, which is what survives truncation on a
lock screen; bodies use the configured name in full. Both come from this
machine's config, not from the watched host, which is what makes them safe to
interpolate.

Two bugs found while testing this: fingerprinting at the head of a pipeline
took the whole command down under pipefail when it could not read a key, and an
unparseable existing known_hosts entry compared equal to everything and
reported the host already trusted.

branderson approved these changes 2026-08-28 20:21:29 -07:00

branderson merged commit 4ea3827188 into main

2026-08-28 20:21:34 -07:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dejima/dejima-guard#9