Have the tool check the arrangement, rather than the operator #27

2026-09-02T00:15:04-07:00

dejima-agent commented

2026-09-02 00:15:04 -07:00

Summary generated by LLM agent

Four things have to line up for a pool to be scoped, each written by a different command at a different time: the pools file says which set and bridge are its, docker owns the network, iptables owns the rules, and the runner project says which network its jobs land on.

Three of them agreeing is not scoping. And every way of getting it wrong looks like it worked: the runners register, the jobs pass, and the allowlist in force is somebody else's. So the check has to be run deliberately, and until now it was three commands and an eyeball over an iptables -S listing.

acme/one
  ok        set ci-allow-1 exists
  ok        network dejima-ci-1 is on dejima-ci1
  FAIL      dejima-ci1 matches ci-allow-2, which is another pool's
  ok        acme/one runners are on dejima-ci-1

It reports per pool, and also names bridges carrying rules that no pool claims, which is what a pool removed from the file while its runners kept going leaves behind.

It never passes what it could not check

Missing docker, missing iptables, or no privilege is exit 2 with UNKNOWN lines, not a clean report. A verifier that reports success when it did not look is worse than none, because it is the one that gets believed. Exit 0 means they agree, 1 means they do not, 2 means something could not be determined.

Found while writing it

grep answers "nothing matched" with a non-zero status, so a bridge with no rules, which is a finding, aborted the whole command under pipefail. The one pool that needed reporting took every check after it down with it, and the report simply looked short. That is the fourth bug of this shape this week, and it has its own test rather than a comment.

Tested: 212 passing, up from 204. Eight new tests covering each failure mode, a host that agrees, and the indeterminate path.

**Summary generated by LLM agent** Four things have to line up for a pool to be scoped, each written by a different command at a different time: the pools file says which set and bridge are its, docker owns the network, iptables owns the rules, and the runner project says which network its jobs land on. Three of them agreeing is not scoping. And **every way of getting it wrong looks like it worked**: the runners register, the jobs pass, and the allowlist in force is somebody else's. So the check has to be run deliberately, and until now it was three commands and an eyeball over an `iptables -S` listing. ``` acme/one ok set ci-allow-1 exists ok network dejima-ci-1 is on dejima-ci1 FAIL dejima-ci1 matches ci-allow-2, which is another pool's ok acme/one runners are on dejima-ci-1 ``` It reports per pool, and also names bridges carrying rules that no pool claims, which is what a pool removed from the file while its runners kept going leaves behind. ## It never passes what it could not check Missing docker, missing iptables, or no privilege is exit 2 with `UNKNOWN` lines, not a clean report. A verifier that reports success when it did not look is worse than none, because it is the one that gets believed. Exit 0 means they agree, 1 means they do not, 2 means something could not be determined. ## Found while writing it `grep` answers "nothing matched" with a non-zero status, so a bridge with **no** rules, which is a finding, aborted the whole command under `pipefail`. The one pool that needed reporting took every check after it down with it, and the report simply looked short. That is the fourth bug of this shape this week, and it has its own test rather than a comment. Tested: 212 passing, up from 204. Eight new tests covering each failure mode, a host that agrees, and the indeterminate path.

dejima-agent added 1 commit 2026-09-02 00:15:05 -07:00

Have the tool check the arrangement, rather than the operator

tests / compose-schema (pull_request_target) Successful in 49s

Details

tests / runner-env (confined) (pull_request_target) Successful in 41s

Details

tests / runner-env (privileged) (pull_request_target) Successful in 39s

Details

tests / tests (pull_request_target) Successful in 1m43s

Details

9fc87ec90e

Four things have to line up for a pool to be scoped, each written by a different
command at a different time: the pools file says which set and bridge are its,
docker owns the network, iptables owns the rules, and the runner project says
which network its jobs land on. Three of them agreeing is not scoping.

Every way of getting it wrong looks like it worked. The runners register, the
jobs pass, and the allowlist in force is somebody else's - so the check that
finds it has to be run deliberately, and until now it was three commands and an
eyeball over an iptables listing.

`verify` reports per pool: the set exists, the network exists and is on the
bridge the rules name, the rules name that pool's set and the base set and
nothing else, and the runners are on that pool's network. Plus bridges carrying
rules that no pool claims, which is what a pool removed from the file while its
runners kept going leaves behind.

IT NEVER PASSES WHAT IT COULD NOT CHECK. Missing docker, missing iptables or no
privilege is exit 2 and an UNKNOWN line, not a clean report. A verifier that
reports success when it did not look is worse than none, because it is the one
that gets believed.

Found while writing it, and the same shape as three other bugs this week: grep
answers "nothing matched" with a non-zero status, so a bridge with no rules -
which is a finding - aborted the whole command under pipefail. The pool that
needed reporting took every check after it down with it, leaving a report that
looked merely short. There is a test for that specifically.

branderson approved these changes 2026-09-02 00:43:00 -07:00

branderson merged commit ae88b7767f into main

2026-09-02 00:43:04 -07:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dejima/dejima-ci-hardener#27