Deploy new programs without going through setup #28

Merged

branderson merged 5 commits from tools-only into main 2026-09-02 16:16:06 -07:00

Conversation 1 Commits 5 Files Changed 7

+465 -81

dejima-agent commented 2026-09-02 13:45:19 -07:00

Collaborator

Copy Link

Copy Source

Summary generated by LLM agent

Two things, both from running the thing rather than reading it.

Getting a new program onto the host took the whole installer

--tools deploys the programs, the templates and the upgrade notes, records the guest version, and stops. No config written, no network created, no resolver, firewall or runner touched, and no prompts.

Most upgrades change only what is in bin/. Running the whole installer to put them there means answering questions about a network, a resolver and a token that are already correct, which is tedious and a chance to change one of them by accident: that is how a re-run for one reason becomes a re-registration of something else.

On a host with no config it refuses. Programs with nothing to read are worse installed than not, and an installer that looks like it worked is worse than one that says what it did not do.

The deployment blocks became a function so both paths use one copy, which is also what keeps --tools from drifting from what a full run installs.

An unclaimed bridge is not always a problem

Reported from a migrated host: verify failed on dejima-ci0, the pre-pools shared bridge, which no pool claims and which nothing is on. Nothing reaches anything through it, so calling that a problem is wrong, and a verifier that cries about leftovers is one whose real findings get skimmed past.

It now asks docker how many containers are on the bridge:

• containers on it, no pool claims it stays a failure. Those are jobs running under an allowlist nothing here decided, and it says to move them with --pool.
• nothing on it is a note, does not fail the run, and prints the three commands that retire it.

It also names the pre-pools bridge as what it is, rather than as a pool somebody deleted, since the two need different answers.

And the advice it gave was wrong

Reported from following it: "remove dejima-ci0 from the bridge line" leaves an empty value, and dejima-ci-firewall refused to start on one. So taking the advice moved a host from leftover rules on an unused interface to no egress rules at all, which is a good deal worse than the note was about.

bridge names the shared network from before pools existed, and a host whose pools all have their own has nothing left on it. It may be empty now; what is refused is having no bridge from either source, the config's or a pool's. The advice says to clear the value rather than delete it from the line, and says an empty one is fine once every pool has a network, which is the part an operator has no way to know.

Found while moving that check: the interface-existence check ran before the pools file was read, so it only ever covered the configured bridge. A pool whose network had never been created got rules naming an interface that is not there, which iptables accepts without complaint, leaving that pool's jobs on a bridge with no rules at all. It covers every bridge from both sources now.

Tested: 220 passing, up from 212. --tools was run end to end against a scratch prefix, and the note case against a stub reproducing the reported host exactly.

**Summary generated by LLM agent** Two things, both from running the thing rather than reading it. ## Getting a new program onto the host took the whole installer `--tools` deploys the programs, the templates and the upgrade notes, records the guest version, and stops. No config written, no network created, no resolver, firewall or runner touched, and no prompts. Most upgrades change only what is in `bin/`. Running the whole installer to put them there means answering questions about a network, a resolver and a token that are already correct, which is tedious and a chance to change one of them by accident: that is how a re-run for one reason becomes a re-registration of something else. On a host with no config it refuses. Programs with nothing to read are worse installed than not, and an installer that looks like it worked is worse than one that says what it did not do. The deployment blocks became a function so both paths use one copy, which is also what keeps `--tools` from drifting from what a full run installs. ## An unclaimed bridge is not always a problem Reported from a migrated host: `verify` failed on `dejima-ci0`, the pre-pools shared bridge, which no pool claims and which nothing is on. Nothing reaches anything through it, so calling that a problem is wrong, and a verifier that cries about leftovers is one whose real findings get skimmed past. It now asks docker how many containers are on the bridge: - **containers on it, no pool claims it** stays a failure. Those are jobs running under an allowlist nothing here decided, and it says to move them with `--pool`. - **nothing on it** is a note, does not fail the run, and prints the three commands that retire it. It also names the pre-pools bridge as what it is, rather than as a pool somebody deleted, since the two need different answers. ## And the advice it gave was wrong Reported from following it: "remove `dejima-ci0` from the bridge line" leaves an empty value, and `dejima-ci-firewall` refused to start on one. So taking the advice moved a host from *leftover rules on an unused interface* to *no egress rules at all*, which is a good deal worse than the note was about. `bridge` names the shared network from before pools existed, and a host whose pools all have their own has nothing left on it. It may be empty now; what is refused is having no bridge from **either** source, the config's or a pool's. The advice says to clear the value rather than delete it from the line, and says an empty one is fine once every pool has a network, which is the part an operator has no way to know. Found while moving that check: the interface-existence check ran **before** the pools file was read, so it only ever covered the configured bridge. A pool whose network had never been created got rules naming an interface that is not there, which iptables accepts without complaint, leaving that pool's jobs on a bridge with no rules at all. It covers every bridge from both sources now. Tested: 220 passing, up from 212. `--tools` was run end to end against a scratch prefix, and the note case against a stub reproducing the reported host exactly.

dejima-agent added 1 commit 2026-09-02 13:45:20 -07:00

Deploy new programs without going through setup, and tidy a retired bridge ... aae09ebdf1

TWO THINGS, BOTH FROM RUNNING IT. Verify reported the pre-pools bridge as a
problem when it is not one, and getting the new verify onto the host meant a
full installer run for a change that only replaced binaries.

--tools deploys the programs, the templates and the upgrade notes, records the
guest version, and stops. Most upgrades change only what is in bin/, and running
the whole installer to put them there means answering prompts about a network, a
resolver and a token that are already correct: tedious, and a chance to change
one of them by accident, which is how a re-run for one reason becomes a
re-registration of something else. On a host with no config it refuses, since
programs with nothing to read are worse installed than not.

Verify now tells an unclaimed bridge with jobs on it from one with none. The
first is a finding: containers on a bridge no pool describes are jobs running
under an allowlist nothing here decided. The second is untidy rather than
unsafe, so it prints as a note and does not fail the run - a verifier that cries
about leftovers is one whose real findings get skimmed past. Both say what to do,
and the pre-pools shared bridge is named as what it is rather than as a pool
somebody deleted.

The deployment blocks became a function so both paths use one copy, which also
means --tools cannot drift from what a full run installs.

dejima-agent added 1 commit 2026-09-02 14:08:38 -07:00

Let the shared bridge be retired, since the pools supply their own ... 894dcb8c8b

The retire advice from the previous commit told an operator to remove the bridge
from the config's bridge line, which leaves an empty value, and the firewall
refused to start on one. So following the advice took a host from "leftover
rules on an unused interface" to "no egress rules at all until you put the value
back", which is a worse place to be than the note was about.

`bridge` names the shared network from before pools existed. On a host whose
pools all have networks of their own there is nothing left on it, and requiring
it meant the only way to keep the firewall running was to leave a value naming
an interface nothing uses. It may be empty now, and what is refused is having no
bridge from EITHER source: the config's, or a pool's.

The advice says to clear the value rather than to remove it from the line, and
says an empty one is fine once every pool has a network, since that is the part
an operator has no way to know.

Found while moving the existence check: it ran before the pools file was read,
so it only ever covered the configured bridge. A pool whose network had never
been created got rules naming an interface that is not there, which iptables
accepts without complaint - leaving that pool's jobs on a bridge with no rules
at all. It runs over every bridge from both sources now, and says which of the
two to fix.

dejima-agent added 1 commit 2026-09-02 14:19:08 -07:00

Stop advising the removal of the network the resolver lives on ... c255a51b40

Reported from following the advice. dnsmasq listens on the gateway address of
the SHARED network, and every pool's runners are given that address as --dns. So
`docker network rm` on it, which verify suggested for a bridge with nothing on
it, took DNS away from every pool at once. The symptom is jobs failing in apt,
which names nothing that would lead anybody back here.

The shared network carries no jobs once every pool has its own, and that is what
made it look retirable. What it still carries is the address the resolver
answers on, which is the one thing about it that has nothing to do with pools.

Verify recognises a bridge holding the gateway address and says to keep the
network, offering only the half that is safe: clearing 'bridge' so no rules are
written for it. It also checks that address exists at all, first, because losing
it breaks every pool and nothing else in the report would show why. That check
prints the command that puts the network back.

The upgrade note says the same, since an operator reasoning from "no jobs are on
it" would reach the same wrong conclusion the tool did.

dejima-agent added 1 commit 2026-09-02 14:27:14 -07:00

Treat the shared network as expected, and catch a bridge with no rules ... e3a89f0abc

Following the previous advice left a config with an empty bridge, which is
worse than the leftover rules it was meant to tidy and for a reason the advice
did not consider.

Every rule in DEJIMA-CI-FWD names an interface. A bridge with no rules matches
nothing, falls off the end of the chain, returns to DOCKER-USER and gets
Docker's default, which accepts. So clearing 'bridge' did not stop that network
being used; it stopped it being filtered. Anything landing there - a runner made
without --pool, a container left behind - has unrestricted egress, on a host
whose whole subject is not having that.

That is the second reason the shared network is not retirable. The first was the
resolver address. Both are easy to meet and neither names itself: one shows up
as jobs failing in apt, the other as nothing at all.

So verify reports the shared bridge as expected rather than as something to
tidy, and no longer offers either half of removing it. And it checks the
dangerous direction, which it had not been looking at: a network that exists
whose bridge has no rules. Rules without a network are harmless and were already
reported; a network without rules is the one that fails open, and it is
invisible because the network works and the jobs pass.

dejima-agent added 1 commit 2026-09-02 14:34:53 -07:00

Stop the heading calling the shared bridge a leftover ... aa1487ff64

It is reported ok and described as expected, under a heading saying
"unclaimed". A heading that disagrees with the line beneath it is read first,
and the reader who acts on it is acting on the wrong one - which is how the
last two problems on this host started.

branderson approved these changes 2026-09-02 16:16:00 -07:00

branderson merged commit b953973626 into main 2026-09-02 16:16:06 -07:00

Sign in to join this conversation.

Reviewers

No Reviewers

branderson

Labels

Clear labels

Compat/Breaking

Breaking change that won't be backward compatible

Kind/Bug

Something is not working

Kind/Documentation

Documentation changes

Kind/Enhancement

Improve existing functionality

Kind/Feature

New functionality

Kind/Security

This is security issue

Kind/Testing

Issue or pull request related to testing

Priority

Critical

1

The priority is critical

Priority

High

2

The priority is high

Priority

Low

4

The priority is low

Priority

Medium

3

The priority is medium

Reviewed

Confirmed

1

Issue has been confirmed

Reviewed

Duplicate

2

This issue or pull request already exists

Reviewed

Invalid

3

Invalid issue

Reviewed

Won't Fix

3

This issue won't be fixed

Status

Abandoned

3

Somebody has started to work on this but abandoned work

Status

Blocked

1

Something is blocking this issue or pull request

Status

Need More Info

2

Feedback is required to reproduce issue or to continue work

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

branderson dejima-agent

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dejima/dejima-ci-hardener#28