Report findings, not reasoning, and fix the kernel currency check #10

2026-08-16T21:37:39-07:00

dejima-agent commented

2026-08-16 21:37:39 -07:00

Summary generated by LLM agent

The audit explained itself at every step. It now reports.

The output

Gone: why a positive control makes a timeout mean something, why an
INDETERMINATE is not a pass, why the two kernel versions being equal is "not a
detail", the paragraph on why the model API token is worth naming ahead of the
source it sits next to. The reasoning is worth having and it is now in the
source, next to the code it explains, where it does not cost the operator a
screenful on every run.

What stayed is what an operator acts on. FAIL still lists the three things to
check, in order, because that is a diagnosis rather than a justification. The
DHCP pair is still spelled out where it applies. INDETERMINATE still says it
is not a pass, in one clause instead of a paragraph.

The kernel check was wrong

Found by reading a real run. Debian ships the signed image as
linux-image-<version> and the unsigned one as
linux-image-<version>-unsigned, and uname -r prints neither suffix. The
check compared the package name against uname -r, so a host running the
unsigned image was told a restart was pending, permanently, whatever it had
booted.

dpkg-query -W also lists packages dpkg merely remembers, so a kernel removed
but not purged could present itself as the newest installed - announcing a
restart into a kernel that is not on the disk.

Both are fixed in newest_kernel, which is now a function with tests rather
than a pipeline inline in the report. The output is a yes/no line instead of two
version strings for the reader to compare.

This matters more than a cosmetic wrong answer. It is the one line in the
report an operator is expected to act on, so a version of it that cries wolf on
ordinary hosts teaches them to scroll past the section.

Where bootstrap leaves the host

It ended with a fixed paragraph explaining that the audit reports FAIL until
the node rules are applied. On a host where they already were, that
contradicted the PASS three lines above it.

It now branches on the audit's actual result: either the rules are in force and
the timer will keep checking, or the next step and the exact command to re-run.

Exit status stays 0 in both cases, and the reasoning is recorded in the source.
What this program installs lives in the guest, and it either installed or died
earlier. The audit's subject is the hypervisor, which the runbook configures
after this runs, so a FAIL there is the expected reading at that point rather
than an install that went wrong.

Tested

83 tests pass, up from 77. New coverage:

the unsigned image package is not read as a different kernel
a kernel removed but not purged does not count as installed
the newest of several installed kernels wins
no packaged kernel is reported as unanswered rather than as up to date
both readings of the restart check

Existing assertions that anchored on removed prose were moved to text that is
still there, not deleted. The "report runs to its end" test now anchors on
newest installed, which both branches of the kernel check print, so it still
proves the function got past the dpkg-query call that once killed it halfway
through.

**Summary generated by LLM agent** The audit explained itself at every step. It now reports. ## The output Gone: why a positive control makes a timeout mean something, why an INDETERMINATE is not a pass, why the two kernel versions being equal is "not a detail", the paragraph on why the model API token is worth naming ahead of the source it sits next to. The reasoning is worth having and it is now in the source, next to the code it explains, where it does not cost the operator a screenful on every run. What stayed is what an operator acts on. `FAIL` still lists the three things to check, in order, because that is a diagnosis rather than a justification. The DHCP pair is still spelled out where it applies. `INDETERMINATE` still says it is not a pass, in one clause instead of a paragraph. ## The kernel check was wrong Found by reading a real run. Debian ships the signed image as `linux-image-<version>` and the unsigned one as `linux-image-<version>-unsigned`, and `uname -r` prints neither suffix. The check compared the package name against `uname -r`, so a host running the unsigned image was told a restart was pending, permanently, whatever it had booted. `dpkg-query -W` also lists packages dpkg merely remembers, so a kernel removed but not purged could present itself as the newest installed - announcing a restart into a kernel that is not on the disk. Both are fixed in `newest_kernel`, which is now a function with tests rather than a pipeline inline in the report. The output is a yes/no line instead of two version strings for the reader to compare. This matters more than a cosmetic wrong answer. It is the one line in the report an operator is expected to act on, so a version of it that cries wolf on ordinary hosts teaches them to scroll past the section. ## Where bootstrap leaves the host It ended with a fixed paragraph explaining that the audit reports FAIL until the node rules are applied. On a host where they already were, that contradicted the PASS three lines above it. It now branches on the audit's actual result: either the rules are in force and the timer will keep checking, or the next step and the exact command to re-run. Exit status stays 0 in both cases, and the reasoning is recorded in the source. What this program installs lives in the guest, and it either installed or died earlier. The audit's subject is the hypervisor, which the runbook configures after this runs, so a FAIL there is the expected reading at that point rather than an install that went wrong. ## Tested 83 tests pass, up from 77. New coverage: - the unsigned image package is not read as a different kernel - a kernel removed but not purged does not count as installed - the newest of several installed kernels wins - no packaged kernel is reported as unanswered rather than as up to date - both readings of the restart check Existing assertions that anchored on removed prose were moved to text that is still there, not deleted. The "report runs to its end" test now anchors on `newest installed`, which both branches of the kernel check print, so it still proves the function got past the `dpkg-query` call that once killed it halfway through.

dejima-agent added 1 commit 2026-08-16 21:37:40 -07:00

Report findings, not reasoning, and fix the kernel currency check

tests / runner-env (pull_request_target) Successful in 34s

Details

tests / tests (pull_request_target) Successful in 48s

Details

c784e0cc88

Three things, all about what the tool says when it runs.

The audit's output explained itself at every step: why a positive control
makes a timeout mean something, why an INDETERMINATE is not a pass, why
keeping the two kernel versions equal is "not a detail". The reasoning is
worth having and it belongs in the source, where it now lives. Output is
for findings and for the next action.

The kernel check was wrong on ordinary Debian hosts. Debian ships the
signed image as linux-image-<version> and the unsigned one as
linux-image-<version>-unsigned; uname prints neither suffix, so comparing
the package name against uname -r reported a pending restart on every host
running an unsigned image, whatever it had booted. dpkg-query was also
counting packages removed but not purged, which could announce a restart
into a kernel that is not on the disk. Both are fixed in newest_kernel,
with tests, and the line now reads yes/no rather than asking the reader to
compare two version strings themselves.

Bootstrap ended with a fixed paragraph saying the audit reports FAIL until
the node rules are applied. On a host where they already were, that
contradicted the PASS three lines above it. It now branches on the audit's
result: rules in force, or the exact command to re-run after applying
them. Exit status stays 0 either way - what this program installs is in
the guest, and a FAIL from a hypervisor that has not been configured yet
is the expected reading at that point in the runbook, not a failed install.

Tested: 83 tests pass. New coverage for the unsigned-image comparison, for
removed-but-not-purged packages, for both readings of the kernel check,
and for a host where no packaged kernel is found.

branderson approved these changes 2026-08-16 21:51:24 -07:00

branderson merged commit 652629e681 into main

2026-08-16 21:51:29 -07:00

dejima-agent referenced this pull request

2026-08-17 00:08:32 -07:00

Hand off with dejima's dependencies met and its source on the box #11

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dejima/dejima-devbox-hardener#10