Version the deployed state, so an existing devbox can be brought forward #17

Merged

branderson merged 2 commits from versioning into ipv6-boundary 2026-08-27 21:32:26 -07:00

Conversation 1 Commits 2 Files Changed 10

+1081 -2

dejima-agent commented 2026-08-25 15:43:50 -07:00

Collaborator

Copy Link

Copy Source

Summary generated by LLM agent

Stacked on ipv6-boundary (#16), which defines what v2 is. Merge that first.

The problem

A devbox is a long-lived VM pinned to the day it was built. Half its controls were typed into a hypervisor console that no git pull can reach, the other half were installed from a checkout that has since moved on, and a snapshot restore can move it backwards without anyone deciding to. Nothing about it converges on the current version by itself.

dejima needs none of this, and the difference is worth writing down rather than inheriting: its containers are short-lived, only the tip of main is supported, and a pull closes any gap that opens. Every one of those sentences is false here.

The IPv6 work in #16 made this concrete. There was no way to ask a running devbox whether it had the fix, no way to tell an operator what to do about it, and no way to notice that restoring last month's snapshot had quietly taken it away again.

Integer versions, one note per version

VERSION holds a single number. upgrades/000N-slug.md is what an operator does to reach N from N−1.

There is a note for every version, including ones that need nothing. An absent file and a file saying "nothing to do" are the same thing to a reader who has to go looking, and this repository's convention is that a choice not to act is recorded where the absence is. A test enforces it.

The number means "a set of required deployed states", not an API surface. There is no minor part, because there is no such thing here as a change that does not need an operator to decide whether it affects them.

Two version numbers, not one

/etc/dejima-devbox/state:

guest_version=2
node_version=2
node_asserted=2026-08-25
node_verified=yes

guest_version is written by dejima-devbox-bootstrap from the checkout's VERSION. It records something this repository did, so it is as good as the install was.

node_version is written by an operator saying they did the console work. Nothing in the guest can read the hypervisor's rules — which is the entire reason those rules are the primary control — so it is an assertion, and the tool treats it as one.

They are kept apart because they are known in different ways, and because their disagreeing is informative:

	Means
node behind guest	Console work outstanding. The ordinary case after a pull.
guest behind node	You restored a snapshot older than your node rules. The node half does not roll back with the guest.

A single number would have averaged both into silence. The second row is the one that has no other warning.

An assertion the machine is allowed to contradict

An upgrade may declare a verify-node: command that passes only if the console work is actually in force. v2's is dejima-devbox-audit reach, whose IPv6 half from #16 is a behavioural test of the exact rule the upgrade asks you to add.

• mark-node runs it first and refuses to record a claim the machine contradicts. --force overrides and records the result as unverified, so status keeps saying so rather than forgetting. --force is kept because a broken network makes some checks unreadable, and refusing outright would only teach people to edit the state file by hand.
• status re-runs the checks for everything already recorded. An assertion that was true in June and is false now is a rule somebody removed, and that is better found by a timer than by an incident.

Where a node-side change has no consequence observable from the guest, the note declares no check and the tool says "v3 declares no check, so this is your word for it" — a weaker claim, labelled as one.

Refusing to be reassuring

Two failure modes, both the same mistake the audit is built around not making, both tested:

It will not report "up to date" about versions it has never heard of. Run from a checkout it is authoritative. Run from /usr/local/sbin it knows only the upgrades that shipped with it, and says so. An installed v1 copy would otherwise report a clean bill of health having never heard of v2.

It will not assume an untracked host is v1. A host genuinely at v1 and a host whose state file somebody deleted are identical from in here, and picking the friendlier reading is how a check comes to be believed when it does not know.

What is in the tree

VERSION	2
upgrades/README.md	The header format, and why two numbers
upgrades/0001-baseline.md	For a devbox built before versions existed
upgrades/0002-ipv6-lan-boundary.md	The real one: two console rules, iputils-ping, re-run bootstrap, fresh snapshot
bin/dejima-devbox-upgrade	status, list, show N, mark-node N
docs/upgrading.md	The process, and the snapshot hazard

dejima-devbox-bootstrap now writes the state file, installs the tool, and installs the upgrade notes to /usr/local/share/dejima-devbox/upgrades so a devbox with no checkout can still read what an upgrade asks of it.

Snapshots

There is no scheduled rollback here — that stays rejected, with reasons, because rollback destroys uncommitted work. So an upgrade is not undone on a timer the way it is on a runner.

What it is exposed to is you, later, restoring a snapshot from before it: it boots fine, passes its own v1 checks, and is missing what the upgrade added. So snapshot: required on a note means take a fresh one and prefer deleting the pre-upgrade one. A snapshot is a security posture with a date on it, and an old one is a bootable copy of whatever the upgrade fixed.

Tested

bats test/ — 133 passing, up from 114 on ipv6-boundary.

The 19 new tests cover both refusal behaviours above, the version arithmetic deciding which halves are outstanding, the guest < node snapshot-restore case, mark-node refusing and --force recording as unverified, a missing verify command not being reported as a failed one, and consistency between VERSION and the notes so a version bump cannot be half-done.

One test I wrote asserted nothing and was replaced before commit — it would have passed on a tool that did not exist, which is the failure test/audit.bats already warns about in its own setup.

Not done

The verify-node mechanism runs one command per upgrade and reads its exit status. It does not distinguish FAIL from INDETERMINATE, so an upgrade whose check is unreadable because the network is down is treated the same as one that failed — --force is the escape, and the state records that it was used. Splitting the two would mean the notes declaring which exit codes mean what, which seemed worse than the ambiguity until an upgrade actually needs it.

**Summary generated by LLM agent** **Stacked on `ipv6-boundary` ([#16](https://gitbox.branderson.io/dejima/dejima-devbox-hardener/pulls/16)), which defines what v2 is. Merge that first.** ## The problem A devbox is a long-lived VM pinned to the day it was built. Half its controls were typed into a hypervisor console that no `git pull` can reach, the other half were installed from a checkout that has since moved on, and a snapshot restore can move it *backwards* without anyone deciding to. Nothing about it converges on the current version by itself. dejima needs none of this, and the difference is worth writing down rather than inheriting: its containers are short-lived, only the tip of `main` is supported, and a pull closes any gap that opens. Every one of those sentences is false here. The IPv6 work in #16 made this concrete. There was no way to ask a running devbox whether it had the fix, no way to tell an operator what to do about it, and no way to notice that restoring last month's snapshot had quietly taken it away again. ## Integer versions, one note per version `VERSION` holds a single number. `upgrades/000N-slug.md` is what an operator does to reach N from N−1. There is a note for **every** version, including ones that need nothing. An absent file and a file saying "nothing to do" are the same thing to a reader who has to go looking, and this repository's convention is that a choice not to act is recorded where the absence is. A test enforces it. The number means "a set of required deployed states", not an API surface. There is no minor part, because there is no such thing here as a change that does not need an operator to decide whether it affects them. ## Two version numbers, not one `/etc/dejima-devbox/state`: ``` guest_version=2 node_version=2 node_asserted=2026-08-25 node_verified=yes ``` `guest_version` is written by `dejima-devbox-bootstrap` from the checkout's `VERSION`. It records something this repository did, so it is as good as the install was. `node_version` is written by an operator saying they did the console work. Nothing in the guest can read the hypervisor's rules — which is the entire reason those rules are the primary control — so it is an assertion, and the tool treats it as one. They are kept apart because they are known in different ways, and because their disagreeing is informative: | | Means | | --- | --- | | `node` behind `guest` | Console work outstanding. The ordinary case after a pull. | | `guest` behind `node` | You restored a snapshot older than your node rules. The node half does not roll back with the guest. | A single number would have averaged both into silence. The second row is the one that has no other warning. ## An assertion the machine is allowed to contradict An upgrade may declare a `verify-node:` command that passes only if the console work is actually in force. v2's is `dejima-devbox-audit reach`, whose IPv6 half from #16 is a behavioural test of the exact rule the upgrade asks you to add. - `mark-node` runs it first and **refuses to record a claim the machine contradicts**. `--force` overrides and records the result as unverified, so `status` keeps saying so rather than forgetting. `--force` is kept because a broken network makes some checks unreadable, and refusing outright would only teach people to edit the state file by hand. - `status` re-runs the checks for everything already recorded. An assertion that was true in June and is false now is a rule somebody removed, and that is better found by a timer than by an incident. Where a node-side change has no consequence observable from the guest, the note declares no check and the tool says *"v3 declares no check, so this is your word for it"* — a weaker claim, labelled as one. ## Refusing to be reassuring Two failure modes, both the same mistake the audit is built around not making, both tested: **It will not report "up to date" about versions it has never heard of.** Run from a checkout it is authoritative. Run from `/usr/local/sbin` it knows only the upgrades that shipped with it, and says so. An installed v1 copy would otherwise report a clean bill of health having never heard of v2. **It will not assume an untracked host is v1.** A host genuinely at v1 and a host whose state file somebody deleted are identical from in here, and picking the friendlier reading is how a check comes to be believed when it does not know. ## What is in the tree | | | | --- | --- | | `VERSION` | `2` | | `upgrades/README.md` | The header format, and why two numbers | | `upgrades/0001-baseline.md` | For a devbox built before versions existed | | `upgrades/0002-ipv6-lan-boundary.md` | The real one: two console rules, `iputils-ping`, re-run bootstrap, fresh snapshot | | `bin/dejima-devbox-upgrade` | `status`, `list`, `show N`, `mark-node N` | | `docs/upgrading.md` | The process, and the snapshot hazard | `dejima-devbox-bootstrap` now writes the state file, installs the tool, and installs the upgrade notes to `/usr/local/share/dejima-devbox/upgrades` so a devbox with no checkout can still read what an upgrade asks of it. ## Snapshots There is no scheduled rollback here — that stays rejected, with reasons, because rollback destroys uncommitted work. So an upgrade is not undone on a timer the way it is on a runner. What it is exposed to is you, later, restoring a snapshot from before it: it boots fine, passes its own v1 checks, and is missing what the upgrade added. So `snapshot: required` on a note means take a fresh one and prefer deleting the pre-upgrade one. A snapshot is a security posture with a date on it, and an old one is a bootable copy of whatever the upgrade fixed. ## Tested `bats test/` — 133 passing, up from 114 on `ipv6-boundary`. The 19 new tests cover both refusal behaviours above, the version arithmetic deciding which halves are outstanding, the `guest < node` snapshot-restore case, `mark-node` refusing and `--force` recording as unverified, a missing verify command not being reported as a failed one, and consistency between `VERSION` and the notes so a version bump cannot be half-done. One test I wrote asserted nothing and was replaced before commit — it would have passed on a tool that did not exist, which is the failure `test/audit.bats` already warns about in its own setup. ## Not done The `verify-node` mechanism runs one command per upgrade and reads its exit status. It does not distinguish `FAIL` from `INDETERMINATE`, so an upgrade whose check is unreadable because the network is down is treated the same as one that failed — `--force` is the escape, and the state records that it was used. Splitting the two would mean the notes declaring which exit codes mean what, which seemed worse than the ambiguity until an upgrade actually needs it.

dejima-agent referenced this pull request from dejima/dejima-ci-hardener 2026-08-25 15:44:28 -07:00

Version the deployed state, so an existing runner can be brought forward #19

dejima-agent added 1 commit 2026-08-25 16:39:56 -07:00

Version the deployed state, so an existing devbox can be brought forward ... a8569baf1b

A devbox is a long-lived VM pinned to the day it was built. Half its
controls were typed into a hypervisor console no `git pull` can reach, the
other half were installed from a checkout that has since moved on, and a
snapshot restore can move it backwards without anyone deciding to. Nothing
about it converges on the current version by itself.

dejima needs none of this and the difference is worth stating rather than
inheriting: its containers are short-lived, only the tip of main is
supported, and a pull closes any gap that opens.

Integer versions. VERSION holds one number, upgrades/000N-slug.md is what an
operator does to reach N from N-1, and there is a note for every version
including ones that need nothing - an absent file is indistinguishable from
one nobody wrote.

TWO NUMBERS ARE TRACKED, in /etc/dejima-devbox/state. guest_version is
written by bootstrap from the checkout's VERSION, so it records something
this repository did. node_version is written by an operator saying they did
the console work, because nothing in the guest can read the hypervisor's
rules - which is the whole reason those rules are the primary control. They
are kept apart because they are known in different ways and because their
disagreeing is informative: node behind guest is console work outstanding,
guest behind node is a restored snapshot, and one number would have averaged
both into silence.

An upgrade may declare a verify-node command that passes only if the console
work is in force. v2's is `dejima-devbox-audit reach`, whose IPv6 half tests
the exact rule v2 asks for. mark-node runs it and refuses to record a claim
the machine contradicts; --force records it as unverified so status keeps
saying so. status re-runs the checks for everything already recorded, so an
assertion that was true in June and is false now is found by a timer rather
than by an incident.

The tool refuses to be reassuring about what it cannot see. Run from a
checkout it knows the current versions; run from the installed copy it knows
only what shipped with it, and says so instead of reporting "up to date"
about versions it has never heard of. A host with no state file is not
assumed to be v1, because an untracked host and a deleted state file look
identical from in here.

Tested with `bats test/` (133 passing, up from 114). The new suite covers
both failure modes it exists to prevent, plus note/VERSION consistency so a
version bump cannot be half-done.

dejima-agent force-pushed versioning from 78561a6bad to a8569baf1b 2026-08-25 16:39:56 -07:00 Compare

dejima-agent added 1 commit 2026-08-25 17:16:03 -07:00

Finish the v2 note: confirm the result, and say where to run it from ... 29eddfaf61

Adds a closing step. An upgrade that ends at 'take a snapshot' leaves the
operator without the one command that tells them both halves landed, and
without the habit of running it again later - which is when it earns its
keep, since status re-checks recorded assertions and a rule removed in six
months has nothing else to surface it.

Says to run it from the checkout rather than PATH, for the reason the tool
itself reports: an installed copy knows only the versions that existed when
it was installed.

A test now pins the rule that a note whose layers do not include 'guest'
cannot invoke the tool as if it were on PATH, because nothing in such a note
puts it there. It found a real instance in the sister project as soon as it
existed.

dejima-agent referenced this pull request 2026-08-25 23:41:39 -07:00

A pinhole for dejima's remote-access notifications #18

branderson approved these changes 2026-08-27 21:32:21 -07:00

branderson merged commit 928a797d00 into ipv6-boundary 2026-08-27 21:32:26 -07:00

dejima-agent referenced this pull request 2026-08-27 21:37:07 -07:00

Version the deployed state, and pinhole ntfy #20

Sign in to join this conversation.

Reviewers

No Reviewers

branderson

Labels

Clear labels

Compat/Breaking

Breaking change that won't be backward compatible

Kind/Bug

Something is not working

Kind/Documentation

Documentation changes

Kind/Enhancement

Improve existing functionality

Kind/Feature

New functionality

Kind/Security

This is security issue

Kind/Testing

Issue or pull request related to testing

Priority

Critical

1

The priority is critical

Priority

High

2

The priority is high

Priority

Low

4

The priority is low

Priority

Medium

3

The priority is medium

Reviewed

Confirmed

1

Issue has been confirmed

Reviewed

Duplicate

2

This issue or pull request already exists

Reviewed

Invalid

3

Invalid issue

Reviewed

Won't Fix

3

This issue won't be fixed

Status

Abandoned

3

Somebody has started to work on this but abandoned work

Status

Blocked

1

Something is blocking this issue or pull request

Status

Need More Info

2

Feedback is required to reproduce issue or to continue work

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

branderson dejima-agent

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dejima/dejima-devbox-hardener#17