Report findings, not reasoning, and fix the kernel currency check

Three things, all about what the tool says when it runs.

The audit's output explained itself at every step: why a positive control
makes a timeout mean something, why an INDETERMINATE is not a pass, why
keeping the two kernel versions equal is "not a detail". The reasoning is
worth having and it belongs in the source, where it now lives. Output is
for findings and for the next action.

The kernel check was wrong on ordinary Debian hosts. Debian ships the
signed image as linux-image-<version> and the unsigned one as
linux-image-<version>-unsigned; uname prints neither suffix, so comparing
the package name against uname -r reported a pending restart on every host
running an unsigned image, whatever it had booted. dpkg-query was also
counting packages removed but not purged, which could announce a restart
into a kernel that is not on the disk. Both are fixed in newest_kernel,
with tests, and the line now reads yes/no rather than asking the reader to
compare two version strings themselves.

Bootstrap ended with a fixed paragraph saying the audit reports FAIL until
the node rules are applied. On a host where they already were, that
contradicted the PASS three lines above it. It now branches on the audit's
result: rules in force, or the exact command to re-run after applying
them. Exit status stays 0 either way - what this program installs is in
the guest, and a FAIL from a hypervisor that has not been configured yet
is the expected reading at that point in the runbook, not a failed install.

Tested: 83 tests pass. New coverage for the unsigned-image comparison, for
removed-but-not-purged packages, for both readings of the kernel check,
and for a host where no packaged kernel is found.

bin/dejima-devbox-audit

@@ -76,9 +76,10 @@ probe_tcp() {
 gateway_verdict() {
     local gw port result
     if ! gw=$(default_gateway); then
+        # A failure rather than a skip: this check carries the exit status on
+        # its own, and a skip would leave a green run that verified nothing.
         say "$PROG: no default route, so there is nothing to probe."
-        say "  This check carries the exit status on its own, so a missing route"
-        say "  is a failure rather than a skip. Fix the route and run again."
+        say "  Fix the route and run again."
         return 2
     fi
     for port in "${PROBE_PORTS[@]}"; do
@@ -181,32 +182,23 @@ cmd_reach() {
     verdict=$(gateway_verdict) || return $?
     case $verdict in
         PASS)
-            say "PASS: the gateway is unreachable, and the network is working."
-            say "  Both halves matter. The second is what makes the first mean"
-            say "  something rather than being indistinguishable from a dead link."
+            say "PASS: the gateway is unreachable and the forge is reachable."
             ;;
         FAIL)
             say "FAIL: the gateway answered."
-            say "  Either the hypervisor rules are not applied to this VM, or an"
-            say "  exception above the DROPs is wider than it looks. Check the"
-            say "  Network device's Firewall checkbox first, then rule order in the"
-            say "  security group - the console inserts new rules at the top, so"
-            say "  the documented order has to be built by dragging or entered"
-            say "  backwards, and a rule left where it landed does nothing."
-            say ""
-            say "  One false alarm is possible: if a rule REJECTs rather than DROPs,"
-            say "  the block is working but sends an answer back, and this probe"
-            say "  cannot tell that from a live host refusing. Use DROP."
+            say "  Check, in this order:"
+            say "    1. the Firewall checkbox on the VM's Network device"
+            say "    2. rule order in the security group: every exception must sit"
+            say "       above the DROPs. The console inserts new rules at the top,"
+            say "       so a rule left where it landed is below nothing."
+            say "    3. that the rules DROP rather than REJECT. A REJECT is working"
+            say "       and still answers, which this probe reads as reachable."
             rc=1
             ;;
         INDETERMINATE)
-            say "INDETERMINATE: nothing answered, including things that should."
-            say "  That is consistent with the firewall working and equally"
-            say "  consistent with the network being broken, and this check cannot"
-            say "  tell them apart. Fix connectivity, then run again."
-            say ""
-            say "  It is not a pass. A check that reports success when it does not"
-            say "  know is worse than no check, because it gets believed."
+            say "INDETERMINATE: nothing answered, including the forge."
+            say "  A working firewall and a broken network look the same from here."
+            say "  Fix connectivity and run again. This is not a pass."
             rc=1
             ;;
     esac
@@ -228,7 +220,7 @@ cmd_reach() {
 lease_report() {
     local f now renew expire
     if ! f=$(lease_file); then
-        say "  No lease file found, so renewal cannot be checked here."
+        say "  no lease file found, so renewal is not checked"
         return 0
     fi
     now=$(date -u +%s)
@@ -236,25 +228,44 @@ lease_report() {
     expire=$(lease_time "$f" expire) || expire=""
 
     if [ -n "$expire" ]; then
-        say "  Lease expires in $(( (expire - now) / 3600 ))h."
+        say "  lease expires in $(( (expire - now) / 3600 ))h"
     fi
     if [ -z "$renew" ]; then
-        say "  No renewal time in $f, so renewal cannot be checked."
+        say "  no renewal time in $f, so renewal is not checked"
         return 0
     fi
 
     if [ "$now" -gt "$renew" ] && [ "$(stat -c %Y "$f" 2>/dev/null || echo 0)" -lt "$renew" ]; then
-        say "  THIS LEASE HAS NOT BEEN RENEWED since its renewal time passed."
-        say "  dhclient asked and got nothing back, which is what a dropped"
-        say "  reply looks like. Check that BOTH DHCP rules are on the"
-        say "  hypervisor: the outbound one alone gets the request out and the"
-        say "  lease still never completes."
-        say "  This is visible now and costs you the network when the lease"
-        say "  expires, which is the whole reason it is worth reporting early."
+        say "  NOT RENEWING. The renewal time has passed and dhclient has not"
+        say "  rewritten $f, which is what a dropped reply looks like."
+        say "  Check that BOTH DHCP rules are on the hypervisor: the outbound one"
+        say "  alone gets the request out and the lease still never completes."
     fi
     return 0
 }
 
+# The newest kernel dpkg holds installed, in the form uname -r prints. Empty
+# if the question cannot be answered here.
+#
+# Two transformations, and leaving either out makes the answer wrong in the
+# alarming direction. Debian ships the signed image as linux-image-<version>
+# and the unsigned one as linux-image-<version>-unsigned, while uname prints
+# neither suffix, so a bare package name compares unequal on every host running
+# an unsigned image and reports a reboot that is not pending. And dpkg-query
+# reports packages it merely remembers, so a kernel removed but not purged
+# would otherwise present itself as installed.
+newest_kernel() {
+    command -v dpkg-query >/dev/null 2>&1 || return 0
+    # The || true is load-bearing. dpkg-query exits non-zero when nothing
+    # matches, and under set -e with pipefail that killed cmd_radius halfway
+    # through its own report - which looked like the report simply ending, with
+    # no error anywhere. Found by running it in a container.
+    dpkg-query -W -f='${db:Status-Abbrev} ${Package}\n' 'linux-image-[0-9]*' 2>/dev/null \
+        | sed -n 's/^ii  *linux-image-//p' \
+        | sed 's/-unsigned$//' \
+        | sort -V | tail -1 || true
+}
+
 # What root on this VM would obtain.
 #
 # This looks like the target list the audit refuses to keep, and it is not. The
@@ -265,13 +276,13 @@ lease_report() {
 #
 # Paths and modes. Never contents.
 cmd_radius() {
-    local f found=0 newest mode
+    local f found=0 newest running mode
 
     say ""
     say "== What root on this VM would get =="
 
     say ""
-    say "Credentials present:"
+    say "Credentials:"
     for f in "$HOME"/.ssh/id_* "$HOME"/.aws/credentials "$HOME"/.config/gh/hosts.yml \
              "$HOME"/.docker/config.json "$CONF_DIR"/*token*; do
         case "$f" in *.pub) continue ;; esac
@@ -279,54 +290,44 @@ cmd_radius() {
         found=1
         say "  $(stat -c '%A %U:%G' "$f" 2>/dev/null)  $f"
     done
-    [ "$found" = 1 ] || say "  none found"
-    say "  Anything listed here is reusable somewhere else. A scoped deploy"
-    say "  token is revoked and reissued; a personal key is every host that"
-    say "  trusts it."
+    if [ "$found" = 1 ]; then
+        say "  Each of these is reusable off this box. Revoke and reissue them if"
+        say "  this host is ever suspect."
+    else
+        say "  none found"
+    fi
 
     # Listed separately because the advice differs. Everything above can be
     # taken off the box; this cannot, so knowing it is here is the whole of
     # what an operator can do about it.
     say ""
-    say "The credential that cannot be moved off:"
+    say "Model API token:"
     if [ -f "$HOME/.claude/.credentials.json" ]; then
         say "  $(stat -c '%A %U:%G' "$HOME/.claude/.credentials.json" 2>/dev/null)  $HOME/.claude/.credentials.json"
-        say "  The model API token. dejima keeps it out of the container, so a"
-        say "  session can use the API while it runs and cannot carry the token"
-        say "  away. The proxy that holds it runs here, so root here has it."
-        say "  That is the step which turns temporary use into a token someone"
-        say "  keeps. If this box is ever suspect, rotate this first."
-        say ""
-        say "  Not because it is the only thing here worth taking - source and"
-        say "  uncommitted work leave just as easily - but because it is a"
-        say "  standing capability rather than a copy, and it sits at a known"
-        say "  path under a known name, which makes it among the easiest things"
-        say "  on this box to find."
+        say "  dejima keeps this out of the container. The proxy holding it runs"
+        say "  here, so root here has it, and it cannot be moved off this host."
+        say "  Rotate it first if this host is ever suspect."
     else
-        say "  no ~/.claude/.credentials.json here. Either this host does not"
-        say "  launch sessions, or authentication is by API key held elsewhere."
+        say "  not present. This host either launches no sessions, or"
+        say "  authenticates with an API key held elsewhere."
     fi
 
     say ""
     say "Group membership:"
     if id 2>/dev/null | grep -q '(docker)'; then
         say "  in the docker group, which is root-equivalent: 'docker run -v /:/host"
-        say "  --privileged' is one command and needs no exploit."
-        say "  Reported, not corrected. dejima needs it, and separating escape from"
-        say "  root was weighed and rejected - see docs/threat-model.md."
+        say "  --privileged' needs no exploit."
+        say "  Required by dejima. Reported, not corrected - docs/threat-model.md."
     else
         say "  not in the docker group"
     fi
 
     say ""
-    say "Agent memory on this host:"
+    say "Agent memory:"
     if [ -d "$HOME/.claude/projects" ]; then
-        say "  $(find "$HOME/.claude/projects" -maxdepth 1 -mindepth 1 -type d 2>/dev/null | wc -l) project directories, all of them readable by root here."
-        say "  That is what root on this VM reaches, which is this section's"
-        say "  subject. What a session inside a container reaches is a separate"
-        say "  question, decided by dejima's mounts rather than by anything here."
+        say "  $(find "$HOME/.claude/projects" -maxdepth 1 -mindepth 1 -type d 2>/dev/null | wc -l) project directories under $HOME/.claude/projects, readable by root here"
     else
-        say "  no ~/.claude/projects on this host"
+        say "  no $HOME/.claude/projects on this host"
     fi
 
     say ""
@@ -334,49 +335,40 @@ cmd_radius() {
     mode=$(addressing_mode)
     case $mode in
         static)
-            say "  static in the guest. Needs no DHCP rules on the hypervisor,"
-            say "  which is the tightest of the three modes at the boundary."
+            say "  static. Needs no DHCP rules on the hypervisor."
             ;;
         dhcp)
-            say "  DHCP. The hypervisor needs BOTH halves of the DHCP pair - out"
-            say "  udp/67 and in udp/68 - including when the address is reserved"
-            say "  for you on the router. A reservation does not stop the guest"
+            say "  DHCP. The hypervisor needs BOTH halves of the DHCP pair, out"
+            say "  udp/67 and in udp/68, including when the address is reserved for"
+            say "  you on the router: a reservation does not stop the guest"
             say "  speaking DHCP."
             lease_report
             ;;
         *)
-            say "  could not tell. Neither a static nor a dhcp stanza was found"
-            say "  in $INTERFACES_FILE and there is no lease file, so whether"
-            say "  this host needs the DHCP rules is an open question."
+            say "  could not tell: no static or dhcp stanza in $INTERFACES_FILE and"
+            say "  no lease file, so whether the DHCP rules are needed is unresolved"
             ;;
     esac
 
     say ""
-    say "Patch currency:"
-    say "  running kernel:   $(uname -r)"
-    newest=""
-    if command -v dpkg-query >/dev/null 2>&1; then
-        # The || true is load-bearing. dpkg-query exits non-zero when nothing
-        # matches, and under set -e with pipefail that killed this function
-        # halfway through its own report - which looked like the report simply
-        # ending, with no error anywhere. Found by running it in a container.
-        newest=$(dpkg-query -W -f='${Package}\n' 'linux-image-[0-9]*' 2>/dev/null \
-                 | sort -V | tail -1 | sed 's/^linux-image-//' || true)
-    fi
+    say "Kernel:"
+    running=$(uname -r)
+    newest=$(newest_kernel)
+    say "  running:          $running"
     if [ -n "$newest" ]; then
         say "  newest installed: $newest"
-        [ "$newest" = "$(uname -r)" ] \
-            || say "  THESE DIFFER. A reboot is pending, and the fix is not running."
+        if [ "$newest" = "$running" ]; then
+            say "  reboot pending:   no"
+        else
+            say "  reboot pending:   YES, so the newest kernel is not the one running"
+        fi
     else
         # Blank would read as a bug. Say which question went unanswered.
-        say "  newest installed: could not tell - no packaged kernel found."
-        say "  Expected on a container or an image-based system, and worth a"
-        say "  second look on a VM, where it usually means the check is wrong"
-        say "  rather than that no kernel is installed."
+        say "  newest installed: could not tell, no packaged kernel found."
+        say "  Expected in a container or on an image-based system; on a VM it"
+        say "  usually means this check is wrong rather than that no kernel is"
+        say "  installed."
     fi
-    say "  A pending reboot is the usual reason a patched box is not patched."
-    say "  Keeping these two equal is the whole of the response to container"
-    say "  escape here, so it is not a detail."
     return 0
 }
 
@@ -404,12 +396,8 @@ in_container() {
 container_warning() {
     in_container || return 0
     say "$PROG: this looks like a container, not a devbox."
-    say ""
-    say "  What follows describes this container. The gateway probed is the"
-    say "  container's own, any credential found is what is mounted here, and"
-    say "  the verdict is about the container's egress rather than the VM's."
-    say ""
-    say "  For a reading about the devbox, install this with"
+    say "  What follows describes the container: its gateway, its mounts, its"
+    say "  group memberships. For a reading about the devbox, install this with"
     say "  dejima-devbox-bootstrap and run it there. Continuing anyway."
     say ""
 }

bin/dejima-devbox-bootstrap

@@ -144,17 +144,35 @@ systemctl enable --now dejima-devbox-audit.timer
 say ""
 say "Running the audit once."
 say ""
-"$BIN_DIR/dejima-devbox-audit" || true
+audit_rc=0
+"$BIN_DIR/dejima-devbox-audit" || audit_rc=$?
 
 # ---------------------------------------------------------------------------
-# 6. What a guest cannot do for itself.
+# 6. Where this leaves the host.
+#
+# Exits 0 either way. What this program installs is in the guest, and it either
+# installed or died earlier; the audit's subject is the hypervisor, which is
+# configured after this runs on the documented order, so a FAIL here is the
+# expected reading rather than an install that went wrong.
+#
+# What would be wrong is saying the same thing in both cases. This printed a
+# fixed closing paragraph explaining that the audit reports FAIL until the node
+# rules are applied, which on a host where they already were contradicted the
+# PASS three lines above it.
 
 say ""
-say "Still manual, and it cannot be otherwise:"
-say ""
-say "  The hypervisor firewall rules, in proxmox/README.md section 3. They are"
-say "  applied to this VM's tap device, on the node, which is what makes them"
-say "  worth having - a compromised guest cannot reach them, and neither can"
-say "  this program."
-say ""
-say "  Until they are applied, the audit above reports FAIL. That is correct."
+if [ "$audit_rc" -eq 0 ]; then
+    say "Installed, and the hypervisor rules are in force."
+    say "The timer re-checks them hourly."
+else
+    say "Installed. The audit did not pass."
+    say ""
+    say "  If the hypervisor firewall is not applied yet, this is the expected"
+    say "  reading at this point in the runbook. Apply proxmox/README.md"
+    say "  section 3, then run:"
+    say ""
+    say "    $BIN_DIR/dejima-devbox-audit reach"
+    say ""
+    say "  Those rules are applied to this VM's tap device, on the node. Nothing"
+    say "  in the guest can apply them, and this program cannot either."
+fi

docs/building-a-devbox.md

@@ -73,7 +73,7 @@ One command. There is no separate install step.
 
 It checks its prerequisites, offers to pin the addressing, asks for your forge's
 hostname, installs the audit with its timer, runs the audit once, and then tells
-you what it could not do for you.
+you where that result leaves the host.
 
 **Expect it to report FAIL the first time**, unless you have already applied the
 node firewall. That is correct: nothing is contained yet.

test/audit.bats

@@ -189,10 +189,13 @@ setup() {
     HOME="$BATS_TEST_TMPDIR"
     run cmd_radius
     [ "$status" -eq 0 ]
-    [[ "$output" == *"Credentials present"* ]]
+    [[ "$output" == *"Credentials"* ]]
     [[ "$output" == *"Group membership"* ]]
-    [[ "$output" == *"Patch currency"* ]]
-    [[ "$output" == *"not a detail"* ]]
+    [[ "$output" == *"Kernel"* ]]
+    # The last line the report prints after the command that once killed it.
+    # Both branches of the kernel check print it, so it is not a claim about
+    # what dpkg holds - only that the function got past dpkg-query.
+    [[ "$output" == *"newest installed"* ]]
 }
 
 # --- addressing mode and lease health ---------------------------------------
@@ -249,7 +252,7 @@ setup() {
     printf 'lease {\n  renew 0 %s;\n  expire 0 %s;\n}\n' "$past" "$future" > "$fixture_lease"
     touch -d '3 hours ago' "$fixture_lease"
     run cmd_radius
-    [[ "$output" == *"NOT BEEN RENEWED"* ]]
+    [[ "$output" == *"NOT RENEWING"* ]]
 }
 
 @test "a lease still inside its renewal window is not reported as failing" {
@@ -265,7 +268,7 @@ setup() {
     # report that says nothing about leases at all, which is how this test
     # went green before the feature existed.
     [[ "$output" == *"Addressing"* ]]
-    [[ "$output" != *"NOT BEEN RENEWED"* ]]
+    [[ "$output" != *"NOT RENEWING"* ]]
 }
 
 @test "an unparseable lease file does not take the report down with it" {
@@ -276,7 +279,7 @@ setup() {
     printf 'this is not a lease\n' > "$fixture_lease"
     run cmd_radius
     [ "$status" -eq 0 ]
-    [[ "$output" == *"not a detail"* ]]
+    [[ "$output" == *"newest installed"* ]]
 }
 
 @test "the credential that cannot be moved off the box is inventoried" {
@@ -294,6 +297,63 @@ setup() {
     [[ "$output" == *"cannot be moved off"* ]]
 }
 
+# --- kernel currency --------------------------------------------------------
+#
+# A false "reboot pending" is worse than it sounds. It is the one line in the
+# report an operator is expected to act on, so a version of it that is wrong on
+# ordinary hosts teaches them to scroll past the whole section.
+
+@test "the unsigned image package is not read as a different kernel" {
+    # Debian's signed image is linux-image-<version> and the unsigned one is
+    # linux-image-<version>-unsigned. uname prints neither suffix, so comparing
+    # the bare package name reported a pending reboot on every host running an
+    # unsigned image, permanently, whatever it booted.
+    dpkg-query() { printf 'ii  linux-image-6.12.101+deb13-amd64-unsigned\n'; }
+    run newest_kernel
+    [ "$output" = "6.12.101+deb13-amd64" ]
+}
+
+@test "a kernel removed but not purged does not count as installed" {
+    # dpkg-query lists rc packages too. Counting one would announce a reboot
+    # into a kernel that is not on the disk.
+    dpkg-query() {
+        printf 'ii  linux-image-6.1.0-13-amd64\nrc  linux-image-6.12.101+deb13-amd64\n'
+    }
+    run newest_kernel
+    [ "$output" = "6.1.0-13-amd64" ]
+}
+
+@test "the newest of several installed kernels wins" {
+    dpkg-query() {
+        printf 'ii  linux-image-6.1.0-13-amd64\nii  linux-image-6.12.101+deb13-amd64\n'
+    }
+    run newest_kernel
+    [ "$output" = "6.12.101+deb13-amd64" ]
+}
+
+@test "no packaged kernel is reported as unanswered, not as up to date" {
+    dpkg-query() { return 1; }
+    HOME="$BATS_TEST_TMPDIR"
+    run cmd_radius
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"could not tell"* ]]
+    [[ "$output" != *"reboot pending"* ]]
+}
+
+@test "a running kernel matching the newest reports no pending reboot" {
+    HOME="$BATS_TEST_TMPDIR"
+    newest_kernel() { uname -r; }
+    run cmd_radius
+    [[ "$output" == *"reboot pending:   no"* ]]
+}
+
+@test "a running kernel behind the newest reports one" {
+    HOME="$BATS_TEST_TMPDIR"
+    newest_kernel() { printf '99.0.0-amd64\n'; }
+    run cmd_radius
+    [[ "$output" == *"reboot pending:   YES"* ]]
+}
+
 # --- wrong machine ----------------------------------------------------------
 
 @test "running in a container says so before reporting anything" {