Run in a container, for a host that has a fleet of them and no timer #1

2026-08-28T01:12:18-07:00

dejima-agent commented

2026-08-28 01:12:18 -07:00

Summary generated by LLM agent

Optional and additive. The program stays single-shot, a systemd host still uses the timer, and nothing about the design changes — what changes is that the scheduler can be a container runtime.

Two things had to change for it to work at all

known_hosts had nowhere to live, and this would have failed on the very first run.

StrictHostKeyChecking is left at the ssh default deliberately — this is the one connection whose entire point is noticing that something about that machine changed. But a container filesystem is new every run, so there is no known_hosts, and BatchMode=yes turns the prompt into a failure.

Every collection would have reported "could not reach the devbox" at max priority, forever. A false alarm that never stops is worse than no alarm, because it is the one that gets muted. There is now an ssh_known_hosts setting, pointed at the state volume.

Configuration was file-only, which is friction in a compose file where everything else about a deployment already lives. The environment is read first, the file second. Secrets belong in the file: an environment variable is visible in docker inspect and to anything that can read /proc.

The loop is its own program

bin/dejima-guard-loop, not a --every flag. Single-shot is what a timer wants, what a test can drive, and what an operator can run by hand without wondering whether they have started something. This wraps the guard; it does not become it.

A failed cycle does not stop it. The failure this whole design exists to catch is a devbox that stops answering, and the guard reports that by exiting non-zero. A loop that gave up there would go quiet at the exact moment it had something to say — and the heartbeat, which only fires on a clean cycle, would stop too, so the dead-man's switch would report the guard as dead rather than the devbox as unreachable. Both true, one useful.

It refuses an interval under 60s and a non-numeric one, and exits promptly on TERM.

The container posture

No port is published and none ever should be. What makes a container an acceptable home for a program holding an SSH key to the devbox is that nothing can connect to it. Every capability dropped, root filesystem read-only, no-new-privileges, and a uid of its own — it opens two outbound connections and writes one directory.

The volume is not a cache. It holds the baseline, the last report and known_hosts. Losing it means a fresh baseline that compares clean against everything for the rest of its life.

Debian rather than Alpine, and it is correctness rather than habit: the diff producing the change list uses --unchanged-line-format, which is GNU diffutils and not busybox. Alpine works once diffutils and coreutils are added back, at which point the smaller base has been patched towards this one a package at a time.

Not built, not run

There is no docker in the environment this was written in. The Dockerfile and compose file are reviewed, not tested — so treat the first docker compose up as the real test.

One error was caught by that review rather than by running it: a secret with target: naming an absolute path is a swarm feature and does nothing under plain docker compose. The config secret is now read from /run/secrets with DEJIMA_GUARD_CONF pointing at it.

The two things I would expect to bite first, if anything does: a bind-mounted state directory owned by root rather than uid 10001 (the README says to chown it, the named volume inherits ownership correctly), and the authorized_keys command= restriction, which is written from the shape of SSH_ORIGINAL_COMMAND rather than from testing it.

Tested

bats test/ — 20 passing, 4 new: the environment overriding the file, a file working with no environment set, known_hosts being passed when configured, and ssh left to its own default when it is not.

**Summary generated by LLM agent** Optional and additive. The program stays single-shot, a systemd host still uses the timer, and nothing about the design changes — what changes is that the scheduler can be a container runtime. ## Two things had to change for it to work at all **`known_hosts` had nowhere to live**, and this would have failed on the very first run. `StrictHostKeyChecking` is left at the ssh default deliberately — this is the one connection whose entire point is noticing that something about that machine changed. But a container filesystem is new every run, so there is no `known_hosts`, and `BatchMode=yes` turns the prompt into a failure. Every collection would have reported **"could not reach the devbox" at max priority**, forever. A false alarm that never stops is worse than no alarm, because it is the one that gets muted. There is now an `ssh_known_hosts` setting, pointed at the state volume. **Configuration was file-only**, which is friction in a compose file where everything else about a deployment already lives. The environment is read first, the file second. Secrets belong in the file: an environment variable is visible in `docker inspect` and to anything that can read `/proc`. ## The loop is its own program `bin/dejima-guard-loop`, not a `--every` flag. Single-shot is what a timer wants, what a test can drive, and what an operator can run by hand without wondering whether they have started something. This wraps the guard; it does not become it. **A failed cycle does not stop it.** The failure this whole design exists to catch is a devbox that stops answering, and the guard reports that by exiting non-zero. A loop that gave up there would go quiet at the exact moment it had something to say — and the heartbeat, which only fires on a clean cycle, would stop too, so the dead-man's switch would report the guard as dead rather than the devbox as unreachable. Both true, one useful. It refuses an interval under 60s and a non-numeric one, and exits promptly on `TERM`. ## The container posture **No port is published and none ever should be.** What makes a container an acceptable home for a program holding an SSH key to the devbox is that nothing can connect to it. Every capability dropped, root filesystem read-only, `no-new-privileges`, and a uid of its own — it opens two outbound connections and writes one directory. **The volume is not a cache.** It holds the baseline, the last report and `known_hosts`. Losing it means a fresh baseline that compares clean against everything for the rest of its life. Debian rather than Alpine, and it is correctness rather than habit: the diff producing the change list uses `--unchanged-line-format`, which is GNU diffutils and not busybox. Alpine works once diffutils and coreutils are added back, at which point the smaller base has been patched towards this one a package at a time. ## Not built, not run **There is no docker in the environment this was written in.** The Dockerfile and compose file are reviewed, not tested — so treat the first `docker compose up` as the real test. One error was caught by that review rather than by running it: a secret with `target:` naming an absolute path is a swarm feature and does nothing under plain `docker compose`. The config secret is now read from `/run/secrets` with `DEJIMA_GUARD_CONF` pointing at it. The two things I would expect to bite first, if anything does: a bind-mounted state directory owned by root rather than uid 10001 (the README says to chown it, the named volume inherits ownership correctly), and the `authorized_keys` `command=` restriction, which is written from the shape of `SSH_ORIGINAL_COMMAND` rather than from testing it. ## Tested `bats test/` — 20 passing, 4 new: the environment overriding the file, a file working with no environment set, `known_hosts` being passed when configured, and ssh left to its own default when it is not.

dejima-agent added 1 commit 2026-08-28 01:12:19 -07:00

Run in a container, for a host that has a fleet of them and no timer 58a81b2856

Optional and additive: the program stays single-shot, a systemd host still
uses the timer, and nothing about the design changes. What changes is that
the scheduler can be a container runtime.

TWO THINGS HAD TO CHANGE FOR IT TO WORK AT ALL, and the first would have
failed on the very first run.

known_hosts had nowhere to live. StrictHostKeyChecking is left at the ssh
default on purpose - this is the one connection whose whole point is noticing
that something about that machine changed - but a container filesystem is new
every run, so there is no known_hosts, and BatchMode turns the prompt into a
failure. Every collection would have reported "could not reach the devbox" at
max priority. A false alarm that never stops is worse than no alarm, because
it is the one that gets muted. It now points at the state volume.

And configuration was file-only, which is friction in a compose file where
everything else about a deployment already lives. The environment is read
first, the file second. Secrets belong in the file: an environment variable
is visible in `docker inspect` and to anything that can read /proc.

The loop is its own program rather than a flag, because single-shot is what a
timer wants, what a test can drive, and what an operator can run by hand
without wondering whether they have started something. A failed cycle does
not stop it - the failure this design exists to catch is a devbox that stops
answering, and a loop that gave up there would go quiet exactly when it had
something to say.

Debian rather than Alpine, and it is correctness rather than habit: the diff
producing the change list uses --unchanged-line-format, which is GNU
diffutils and not busybox.

No port is published and none ever should be. What makes a container an
acceptable home for a program holding an SSH key to the devbox is that
nothing can connect to it.

NOT BUILT OR RUN. There is no docker in the environment this was written in,
so the Dockerfile and compose file are reviewed rather than tested. One error
was caught that way: a secret with `target:` naming an absolute path is a
swarm feature and does nothing under plain `docker compose`, so the config
secret is read from /run/secrets and DEJIMA_GUARD_CONF points at it.

`bats test/` 20 passing, 4 new.

branderson approved these changes 2026-08-28 16:10:59 -07:00

branderson merged commit 2236681139 into main

2026-08-28 16:11:04 -07:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dejima/dejima-guard#1