dejima/dejima-devbox-hardener
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Version the deployed state, so an existing devbox can be brought forward
tests / runner-env (pull_request_target) Successful in 38s
Details
tests / tests (pull_request_target) Successful in 1m7s
Details

Browse Source 
A devbox is a long-lived VM pinned to the day it was built. Half its
controls were typed into a hypervisor console no `git pull` can reach, the
other half were installed from a checkout that has since moved on, and a
snapshot restore can move it backwards without anyone deciding to. Nothing
about it converges on the current version by itself.

dejima needs none of this and the difference is worth stating rather than
inheriting: its containers are short-lived, only the tip of main is
supported, and a pull closes any gap that opens.

Integer versions. VERSION holds one number, upgrades/000N-slug.md is what an
operator does to reach N from N-1, and there is a note for every version
including ones that need nothing - an absent file is indistinguishable from
one nobody wrote.

TWO NUMBERS ARE TRACKED, in /etc/dejima-devbox/state. guest_version is
written by bootstrap from the checkout's VERSION, so it records something
this repository did. node_version is written by an operator saying they did
the console work, because nothing in the guest can read the hypervisor's
rules - which is the whole reason those rules are the primary control. They
are kept apart because they are known in different ways and because their
disagreeing is informative: node behind guest is console work outstanding,
guest behind node is a restored snapshot, and one number would have averaged
both into silence.

An upgrade may declare a verify-node command that passes only if the console
work is in force. v2's is `dejima-devbox-audit reach`, whose IPv6 half tests
the exact rule v2 asks for. mark-node runs it and refuses to record a claim
the machine contradicts; --force records it as unverified so status keeps
saying so. status re-runs the checks for everything already recorded, so an
assertion that was true in June and is false now is found by a timer rather
than by an incident.

The tool refuses to be reassuring about what it cannot see. Run from a
checkout it knows the current versions; run from the installed copy it knows
only what shipped with it, and says so instead of reporting "up to date"
about versions it has never heard of. A host with no state file is not
assumed to be v1, because an untracked host and a deleted state file look
identical from in here.

Tested with `bats test/` (133 passing, up from 114). The new suite covers
both failure modes it exists to prevent, plus note/VERSION consistency so a
version bump cannot be half-done.
This commit is contained in:
branderson
2026-08-25 16:38:58 -07:00
parent 8b253765cb
commit a8569baf1b
10 changed files with 1047 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+20
View File
@@ -103,6 +103,8 @@ cd ~/dejima && ./install.sh # dejima's own step, as your own account
sudo dejima-devbox-audit # both halves
sudo dejima-devbox-audit reach # containment only, what the timer runs
sudo dejima-devbox-audit radius # what root here would get
./bin/dejima-devbox-upgrade status # what this box is missing, from a checkout
```
Bootstrap installs what dejima needs from the host - Docker with the `compose`
@@ -155,6 +157,24 @@ map of valuable targets, kept on the machine this project assumes will be owned.
The audit derives its targets from the routing table and its controls from your
forge, so there is nothing to type and nothing to leak.
## Versions, and the boxes already out there
This is at **v2**. `VERSION` says so, and `/etc/dejima-devbox/state` says what a
given devbox is at — two numbers, because the guest half is installed from a
checkout and the node half is typed into a console, and only one of them is
knowable from inside the VM.
dejima needs none of this: its containers are short-lived, only the tip of
`main` is supported, and `git pull` closes any gap. A devbox is pinned to the
day it was built, and a snapshot restore can move it backwards without anyone
deciding to. [docs/upgrading.md](docs/upgrading.md) is the process;
[upgrades/](upgrades/) is one note per version, written for someone standing at
a console who has not read the diff.
```sh
./bin/dejima-devbox-upgrade status # run from a checkout, not the installed copy
```
## Tests
```sh
VERSION
+1
View File
@@ -0,0 +1 @@
2
bin/dejima-devbox-bootstrap
+31 -2
View File
@@ -22,6 +22,7 @@ SRC="$(cd "$(dirname "$0")/.." && pwd)"
BIN_DIR=${BIN_DIR:-/usr/local/sbin}
CONF_DIR=${CONF_DIR:-/etc/dejima-devbox}
UNIT_DIR=${UNIT_DIR:-/etc/systemd/system}
SHARE_DIR=${SHARE_DIR:-/usr/local/share/dejima-devbox}
docker_group_added=0
@@ -46,11 +47,11 @@ writable() { mkdir -p "$1" 2>/dev/null || true; [ -d "$1" ] && [ -w "$1" ]; }
# ---------------------------------------------------------------------------
# 1. Prerequisites, before anything that would need the network to work.
for d in "$CONF_DIR" "$BIN_DIR" "$UNIT_DIR"; do
for d in "$CONF_DIR" "$BIN_DIR" "$UNIT_DIR" "$SHARE_DIR"; do
# Distinguish "cannot write" from "does not exist": [ -e ] cannot tell an
# absent directory from an unsearchable one, and reporting the wrong one
# sends people to reinstall something that is fine.
writable "$d" || die "cannot write $d - run with sudo, or set CONF_DIR/BIN_DIR/UNIT_DIR"
writable "$d" || die "cannot write $d - run with sudo, or set CONF_DIR/BIN_DIR/UNIT_DIR/SHARE_DIR"
done
for c in ip timeout getent stat find; do
@@ -312,6 +313,34 @@ install -m 0755 "$SRC/bin/dejima-devbox-audit" "$BIN_DIR/dejima-devbox-audit"
[ -x "$BIN_DIR/dejima-devbox-audit" ] \
|| die "installed a file which is not executable: $BIN_DIR/dejima-devbox-audit"
install -m 0755 "$SRC/bin/dejima-devbox-upgrade" "$BIN_DIR/dejima-devbox-upgrade"
# The upgrade notes travel with the program, so an operator on a devbox with no
# checkout can still read what an upgrade asks of them. The installed copy
# knows only the versions that existed when it ran, and says so rather than
# reporting "up to date" about versions it has never heard of.
install -m 0755 -d "$SHARE_DIR/upgrades"
install -m 0644 "$SRC"/upgrades/*.md "$SHARE_DIR/upgrades/"
# What this host is now at, guest-side.
#
# Written from the checkout's VERSION rather than asked, because it records
# something this program just did rather than something a person remembers. The
# node half is not written here and cannot be: nothing in this guest can read
# the hypervisor's rules, which is the entire reason those rules are the
# primary control. dejima-devbox-upgrade mark-node is where that assertion goes.
version=$(tr -d '[:space:]' < "$SRC/VERSION")
[ -n "$version" ] || die "no version in $SRC/VERSION - refusing to record an empty one"
if [ -r "$CONF_DIR/state" ]; then
grep -v '^guest_version=' "$CONF_DIR/state" > "$CONF_DIR/state.new" || true
else
: > "$CONF_DIR/state.new"
fi
printf 'guest_version=%s\n' "$version" >> "$CONF_DIR/state.new"
sort -o "$CONF_DIR/state.new" "$CONF_DIR/state.new"
chmod 0640 "$CONF_DIR/state.new"
mv "$CONF_DIR/state.new" "$CONF_DIR/state"
for u in "$SRC"/systemd/*; do
sed "s|@@BIN_DIR@@|$BIN_DIR|g" "$u" > "$UNIT_DIR/$(basename "$u")"
done
bin/dejima-devbox-upgrade
Executable
+379
View File
@@ -0,0 +1,379 @@
#!/usr/bin/env bash
# SPDX-License-Identifier: GPL-3.0-or-later
# Copyright (C) 2026 Brad Anderson
#
# What this devbox is missing, and what to do about it.
#
# dejima needs nothing like this: its containers are short-lived, only the tip
# of main is supported, and `git pull` closes any gap that opens. A devbox is a
# long-lived VM pinned to the moment it was built, with half its controls
# entered by hand in a hypervisor console that no pull can reach. Nothing about
# it converges on the current version on its own.
#
# TWO VERSIONS ARE TRACKED, NOT ONE, and the reason is the same reason the
# audit exists at all:
#
# guest_version written by dejima-devbox-bootstrap when it installs. A
# record of something this repository did, so it is as
# reliable as the install was.
# node_version written by `mark-node`, by an operator saying they did the
# console work. Nothing in the guest can read the
# hypervisor's rules - that is what makes those rules worth
# having - so this is an assertion and is treated as one.
#
# Where an upgrade names a verify-node command, that command is run and is
# allowed to CONTRADICT the assertion. An operator who marked something done
# and was wrong is the case this is for; an operator who is right loses three
# seconds.
#
# Keeping the two apart is also what makes a snapshot restore legible. Restore
# an image from before an upgrade and guest_version goes backwards while
# node_version, living on the hypervisor, does not. A single number would have
# averaged that into silence.
set -euo pipefail
PROG=${0##*/}
CONF_DIR=${CONF_DIR:-/etc/dejima-devbox}
STATE="$CONF_DIR/state"
SHARE_DIR=${SHARE_DIR:-/usr/local/share/dejima-devbox}
say() { printf '%s\n' "$*" >&2; }
# Where the upgrade notes are, and which kind of copy they are.
#
# A checkout is authoritative about what versions exist; an installed copy is
# not, because it only ever shipped with the notes that existed when it was
# installed. That distinction is load-bearing: an installed v1 tool would
# otherwise report "up to date" with total confidence, having never heard of
# v2. So the kind is reported alongside the answer rather than hidden.
UPGRADE_DIR=""
UPGRADE_SRC=""
find_upgrades() {
local self_dir
self_dir=$(cd "$(dirname "$(readlink -f "$0")")" && pwd)
if [ -d "$self_dir/../upgrades" ]; then
UPGRADE_DIR=$(cd "$self_dir/../upgrades" && pwd)
UPGRADE_SRC=checkout
return 0
fi
if [ -d "$SHARE_DIR/upgrades" ]; then
UPGRADE_DIR="$SHARE_DIR/upgrades"
UPGRADE_SRC=installed
return 0
fi
return 1
}
# --- the notes --------------------------------------------------------------
note_for() { # note_for VERSION
local f
f=$(printf '%s/%04d-*.md' "$UPGRADE_DIR" "$1")
# shellcheck disable=SC2086
f=$(ls $f 2>/dev/null | head -1) || true
[ -n "$f" ] || return 1
printf '%s\n' "$f"
}
# A value from the note's header block.
#
# Read only between the first two --- lines, so a line in the prose that
# happens to look like a key cannot change what the tool does. The prose is
# written for a person and should not have to watch its wording.
note_field() { # note_field FILE KEY
sed -n '/^---$/,/^---$/p' "$1" | sed -n "s/^$2: //p" | head -1
}
known_versions() {
# shellcheck disable=SC2012
ls "$UPGRADE_DIR" 2>/dev/null \
| sed -n 's/^0*\([0-9][0-9]*\)-.*\.md$/\1/p' \
| sort -n
}
newest_version() { known_versions | tail -1; }
# --- state ------------------------------------------------------------------
state_get() { # state_get KEY
local v=""
[ -r "$STATE" ] && v=$(sed -n "s/^$1=//p" "$STATE" | tail -1)
[ -n "$v" ] || return 1
printf '%s\n' "$v"
}
state_set() { # state_set KEY VALUE
local tmp="$STATE.new" key=$1 value=$2
[ -d "$CONF_DIR" ] || install -m 0755 -d "$CONF_DIR"
umask 027
{ [ -r "$STATE" ] && grep -v "^$key=" "$STATE" || true; } > "$tmp"
printf '%s=%s\n' "$key" "$value" >> "$tmp"
sort -o "$tmp" "$tmp"
chmod 0640 "$tmp"
mv "$tmp" "$STATE"
}
# --- verification -----------------------------------------------------------
# Run an upgrade's verify-node command, if it has one.
#
# ok - the command passed, so the console work is in force
# failed - it did not, whatever the state file says
# none - the upgrade declares no check
# missing - it declares one and the command is not installed here
#
# `missing` is kept apart from `failed` on purpose. A check that could not run
# has not found anything, and reporting that as a failure would train an
# operator to ignore the word.
verify_node() { # verify_node VERSION
local note cmd
note=$(note_for "$1") || { printf 'none\n'; return 0; }
cmd=$(note_field "$note" verify-node)
[ -n "$cmd" ] || { printf 'none\n'; return 0; }
command -v "${cmd%% *}" >/dev/null 2>&1 || { printf 'missing\n'; return 0; }
if $cmd >/dev/null 2>&1; then
printf 'ok\n'
else
printf 'failed\n'
fi
}
# --- status -----------------------------------------------------------------
VERIFY=1
cmd_status() {
local guest node newest v note rc=0 outstanding=0
if ! find_upgrades; then
say "$PROG: no upgrade notes found."
say " Looked beside this program and in $SHARE_DIR/upgrades."
say " Run this from a checkout of the repository."
return 2
fi
newest=$(newest_version)
say "== Version =="
say ""
if [ ! -r "$STATE" ]; then
# Not assumed to be v1. A host that is genuinely at v1 and a host whose
# state file was deleted look identical from here, and choosing the
# friendlier reading is how a check comes to be believed when it does
# not know.
say " No $STATE, so this host predates version tracking."
say " It is NOT assumed to be v1: an untracked host and a host whose"
say " state file went missing are the same thing from in here."
say ""
say " Read upgrades/0001-baseline.md, which says how to establish"
say " where you actually are."
say ""
say " Newest version known to this copy: v$newest"
return 1
fi
guest=$(state_get guest_version) || guest=""
node=$(state_get node_version) || node=""
say " guest ${guest:+v}${guest:-unrecorded}"
if [ -n "$node" ]; then
say " node v$node asserted $(state_get node_asserted || echo 'date unknown')$(
[ "$(state_get node_verified 2>/dev/null || echo no)" = no ] && printf ', UNVERIFIED' || true)"
else
say " node unrecorded"
fi
say ""
say " notes $UPGRADE_DIR ($UPGRADE_SRC, newest v$newest)"
if [ "$UPGRADE_SRC" = installed ]; then
say ""
say " This is the installed copy, so it only knows the upgrades that"
say " existed when it was installed. It cannot tell you about a version"
say " released since. Pull the repository and run it from there for an"
say " answer that means anything."
fi
# A guest behind the node is the shape a snapshot restore leaves. Worth
# naming, because it is the one direction nobody does on purpose.
if [ -n "$guest" ] && [ -n "$node" ] && [ "$guest" -lt "$node" ]; then
say ""
say "== The guest is behind the node =="
say ""
say " guest v$guest, node v$node. The hypervisor's rules are newer than"
say " what is installed here, which is what restoring an old snapshot"
say " looks like - the node half does not roll back with the guest."
say " Re-run dejima-devbox-bootstrap from a current checkout."
rc=1
fi
say ""
say "== Outstanding =="
say ""
for v in $(known_versions); do
note=$(note_for "$v") || continue
local layers title
layers=$(note_field "$note" layers)
title=$(note_field "$note" title)
local need_guest=0 need_node=0
case " $layers " in *" guest "*) need_guest=1 ;; esac
case " $layers " in *" node "*) need_node=1 ;; esac
[ -n "$guest" ] && [ "$v" -le "$guest" ] && need_guest=0
[ -n "$node" ] && [ "$v" -le "$node" ] && need_node=0
[ "$need_guest" = 0 ] && [ "$need_node" = 0 ] && continue
outstanding=1
rc=1
say " v$v $title"
[ "$need_node" = 1 ] && say " node: console work outstanding"
[ "$need_guest" = 1 ] && say " guest: re-run dejima-devbox-bootstrap from a v$v checkout"
[ "$(note_field "$note" snapshot)" = required ] \
&& say " snapshot: REQUIRED after this one"
say " $PROG show $v"
say ""
done
[ "$outstanding" = 0 ] && say " Nothing. Both halves are at v$newest." && say ""
# Verification last, because it is the slow part and because it reads as a
# verdict on everything above it.
if [ "$VERIFY" = 1 ] && [ -n "$node" ]; then
say "== Node checks =="
say ""
local any=0
for v in $(known_versions); do
[ "$v" -le "$node" ] || continue
case "$(verify_node "$v")" in
ok) any=1; say " v$v in force" ;;
failed) any=1; rc=1
say " v$v FAILED. The check for this upgrade does not pass, so its"
say " console work is not in force whatever the state file says."
say " $PROG show $v" ;;
missing) any=1
say " v$v could not check - $(note_field "$(note_for "$v")" verify-node) is not installed here." ;;
esac
done
[ "$any" = 0 ] && say " None of the recorded upgrades declares a check."
say ""
fi
return $rc
}
# --- show -------------------------------------------------------------------
cmd_show() {
local v=${1:?usage: $PROG show VERSION} note
find_upgrades || { say "$PROG: no upgrade notes found."; return 2; }
note=$(note_for "$v") || {
say "$PROG: no upgrade note for v$v."
say " Known here: $(known_versions | tr '\n' ' ')"
return 2
}
cat "$note"
}
cmd_list() {
local v note
find_upgrades || { say "$PROG: no upgrade notes found."; return 2; }
for v in $(known_versions); do
note=$(note_for "$v") || continue
printf 'v%-4s %s\n' "$v" "$(note_field "$note" title)"
done
}
# --- mark-node --------------------------------------------------------------
FORCE=0
cmd_mark_node() {
local v=${1:?usage: $PROG mark-node VERSION} note result
find_upgrades || { say "$PROG: no upgrade notes found."; return 2; }
note=$(note_for "$v") || { say "$PROG: no upgrade note for v$v."; return 2; }
result=$(verify_node "$v")
case $result in
ok)
say "$PROG: verified - $(note_field "$note" verify-node) passes."
;;
failed)
say "$PROG: refusing to record v$v."
say ""
say " $(note_field "$note" verify-node) does not pass, so the console work"
say " is not in force. Recording it would put a claim in $STATE"
say " that the machine already contradicts."
say ""
say " Run it yourself to see what it says:"
say " $(note_field "$note" verify-node)"
say ""
say " If you are sure and the check is wrong - a broken network"
say " makes some of them unreadable - use --force, which records"
say " the assertion and marks it unverified so status keeps saying so."
[ "$FORCE" = 1 ] || return 1
say ""
say " --force given. Recording it as UNVERIFIED."
;;
missing)
say "$PROG: cannot check - $(note_field "$note" verify-node) is not installed here."
say " Recording as unverified rather than refusing, because a missing"
say " program is not evidence about your firewall."
;;
none)
say "$PROG: v$v declares no check, so this is your word for it."
;;
esac
state_set node_version "$v"
state_set node_asserted "$(date -u +%F)"
state_set node_verified "$([ "$result" = ok ] && echo yes || echo no)"
say ""
say "Recorded: node_version=$v in $STATE"
}
# --- main -------------------------------------------------------------------
usage() {
cat >&2 <<EOF
usage: $PROG [status] [--no-verify]
$PROG list
$PROG show VERSION
$PROG mark-node VERSION [--force]
status what this host is at, what is outstanding, and whether the
node-side work it claims is actually in force
list every upgrade this copy knows about
show an upgrade note in full
mark-node record that the console work for VERSION is done
Run from a checkout for an answer about the current version. The installed
copy only knows the upgrades that existed when it was installed.
EOF
}
main() {
local cmd=status args=()
for a in "$@"; do
case "$a" in
--no-verify) VERIFY=0 ;;
--force) FORCE=1 ;;
-h|--help) usage; return 0 ;;
-*) say "$PROG: unknown option: $a"; usage; return 64 ;;
*) args+=("$a") ;;
esac
done
[ "${#args[@]}" -gt 0 ] && { cmd=${args[0]}; args=("${args[@]:1}"); }
case "$cmd" in
status) cmd_status ;;
list) cmd_list ;;
show) cmd_show "${args[@]:-}" ;;
mark-node) cmd_mark_node "${args[@]:-}" ;;
*) say "$PROG: unknown command: $cmd"; usage; return 64 ;;
esac
}
# Sourced by test/upgrade.bats, which replaces the functions that touch the
# outside world. The :- guard matters under set -u.
if [ "${BASH_SOURCE[0]:-$0}" = "$0" ]; then
main "$@"
fi
docs/README.md
+1
View File
@@ -5,6 +5,7 @@
| [building-a-devbox.md](building-a-devbox.md) | The guest-side runbook: install, bootstrap, verify, and what to keep off the machine |
| [threat-model.md](threat-model.md) | What each layer grants, why container escape is deprioritised, and which options were rejected |
| [reference.md](reference.md) | The three verdicts, the three addressing modes, and the one config key |
| [upgrading.md](upgrading.md) | Moving a devbox that already exists to a newer version, and why the two halves are counted separately |
The node-side half is [../proxmox/README.md](../proxmox/README.md), kept
separate so another hypervisor can be added without touching a guest-side step.
docs/upgrading.md
+139
View File
@@ -0,0 +1,139 @@
# Upgrading a devbox that already exists
A devbox is a long-lived VM pinned to the moment it was built. Half its controls
were entered by hand in a hypervisor console, and the other half were installed
from a checkout that has since moved on. Neither half updates itself, and one of
them can move *backwards* without anybody deciding to.
So this page, and the [`upgrades/`](../upgrades/) directory it describes.
## Why this project has one and dejima does not
dejima's containers are short-lived and only the tip of `main` is supported. A
gap opens, `git pull` closes it, and the next container starts from the closed
state. Nothing is pinned, so nothing drifts.
Every one of those sentences is false here:
| | dejima | a devbox |
| --- | --- | --- |
| Lifetime | one session | months |
| Where the controls live | in the image, rebuilt each run | half in the guest, half typed into a console |
| How a gap closes | `git pull` | a person, deliberately, on two machines |
| Can it move backwards? | no | yes — a snapshot restore |
The last row is the one that makes versioning necessary rather than tidy. A
devbox restored from a snapshot taken before an upgrade boots cleanly, passes
its own checks, and is missing whatever the upgrade added. Nothing about it
looks wrong.
## The two version numbers
`/etc/dejima-devbox/state`:
```
guest_version=2
node_version=2
node_asserted=2026-08-25
node_verified=yes
```
**`guest_version`** is written by `dejima-devbox-bootstrap` from the checkout's
`VERSION` file. It records something this repository did, so it is as good as
the install was.
**`node_version`** is written by `dejima-devbox-upgrade mark-node`, by you,
saying you did the console work. Nothing in this guest can read the
hypervisor's rules — that is what makes those rules the primary control — so it
is an assertion and the tool treats it as one.
They are separate because they are known in different ways, and because their
disagreeing is informative:
| | Means |
| --- | --- |
| `node` behind `guest` | Console work outstanding. The ordinary case after a `git pull`. |
| `guest` behind `node` | You restored a snapshot older than your node rules. The node half does not roll back with the guest. |
A single number would have averaged both into silence.
## Where the assertion gets checked
An upgrade note may declare a `verify-node` command: something that passes only
if the console work is actually in force. For v2 that is
`dejima-devbox-audit reach`, whose IPv6 half is a behavioural test of the exact
rule the upgrade asks you to add.
Two consequences worth stating plainly:
- `mark-node` **runs the check first and refuses to record a claim the machine
contradicts.** `--force` overrides it and records the result as unverified, so
`status` keeps saying so rather than forgetting.
- `status` re-runs the checks for everything you have already recorded. An
assertion that was true in June and is not true now is a rule somebody removed,
and that is worth finding out from a tool rather than from an incident.
Not every upgrade can have one. Where the node-side change has no observable
consequence from inside the guest, the note declares no check and the tool says
so: *"v3 declares no check, so this is your word for it."* That is a weaker
claim and is labelled as one.
## Doing an upgrade
```sh
git -C ~/dejima-devbox-hardener pull
cd ~/dejima-devbox-hardener
./bin/dejima-devbox-upgrade status
```
`status` from a **checkout** is the authoritative answer. The installed copy at
`/usr/local/sbin/dejima-devbox-upgrade` only knows the upgrades that existed
when it was installed, so it cannot tell you about a version released since —
and it says so rather than reporting "up to date" about versions it has never
heard of.
Then, for each outstanding version, in this order:
1. **`dejima-devbox-upgrade show N`** and read it. Node work first — the guest
half often installs the check for the node half, and running it against rules
that do not exist yet produces a `FAIL` that is correct and misleading.
2. **The console work**, in the Proxmox web UI.
3. **The guest work**, usually `sudo ./bin/dejima-devbox-bootstrap` from the
pulled checkout. Bootstrap is idempotent; Enter at every prompt keeps what
you have.
4. **`sudo dejima-devbox-upgrade mark-node N`**, which verifies before it
records.
5. **A fresh snapshot**, if the note says `snapshot: required`.
## Snapshots, and the direction this project cannot roll back
There is no scheduled rollback here. That is
[rejected, with reasons](threat-model.md#rejected-with-reasons): rollback
destroys uncommitted work, which is the thing a devbox exists to hold. So an
upgrade is not undone on a timer the way it is on a CI runner.
What it *is* exposed to is you, later, restoring a snapshot from before it.
**Take a fresh snapshot after upgrading, and prefer deleting the pre-upgrade one
to keeping it.** A snapshot is a security posture with a date on it, and an old
one is a bootable copy of whatever the upgrade fixed. If you keep it, name it so
the date is unmissable in the list.
If you do restore an old one, `dejima-devbox-upgrade status` will report
`guest_version` behind `node_version` and tell you to re-run bootstrap. That is
the pair disagreeing on purpose, and it is the only warning you will get.
## Adding an upgrade, if you are changing this repository
1. Bump `VERSION`.
2. Write `upgrades/000N-slug.md` with the header
[documented here](../upgrades/README.md#the-header).
3. Give it a `verify-node:` if the node-side change has any consequence
observable from inside the guest. Prefer an existing audit check to a new
one.
4. Write it for a person who has not read the diff and is standing at a console.
**Write one for every version, including versions that need nothing.** A missing
file and a file saying "nothing to do" are the same thing to a reader who has to
go looking, and this repository's convention is that a choice not to act is
recorded where the absence is.
test/upgrade.bats
+213
View File
@@ -0,0 +1,213 @@
#!/usr/bin/env bats
# SPDX-License-Identifier: GPL-3.0-or-later
# Copyright (C) 2026 Brad Anderson
#
# The two things this tool must not do, both of which are the same mistake the
# audit is built around not making:
#
# - report "up to date" about versions it has never heard of;
# - record an operator's claim about the hypervisor that the machine in front
# of it already contradicts.
#
# Everything below is one of those two, or the arithmetic that decides which
# halves of an upgrade are outstanding.
setup() {
REPO="$(cd "$BATS_TEST_DIRNAME/.." && pwd)"
TOOL="$REPO/bin/dejima-devbox-upgrade"
STATE_DIR="$BATS_TEST_TMPDIR/etc"
STUB_DIR="$BATS_TEST_TMPDIR/bin"
mkdir -p "$STATE_DIR" "$STUB_DIR"
}
# Run the tool against a throwaway state dir, never the host's.
run_tool() { CONF_DIR="$STATE_DIR" PATH="$STUB_DIR:$PATH" run "$TOOL" "$@"; }
write_state() { printf '%s\n' "$@" > "$STATE_DIR/state"; }
# A stand-in for the verify-node command, exiting how the test wants.
stub_audit() {
printf '#!/bin/sh\nexit %s\n' "$1" > "$STUB_DIR/dejima-devbox-audit"
chmod +x "$STUB_DIR/dejima-devbox-audit"
}
# --- knowing where you are --------------------------------------------------
@test "a host with no state file is not assumed to be v1" {
# An untracked host and a host whose state file was deleted are identical
# from in here. Picking the friendlier reading is how a check comes to be
# believed when it does not know.
run_tool status --no-verify
[ "$status" -ne 0 ]
[[ "$output" == *"predates version tracking"* ]]
[[ "$output" == *"NOT assumed to be v1"* ]]
}
@test "an up-to-date host reports nothing outstanding" {
write_state guest_version=2 node_version=2 node_asserted=2026-08-25 node_verified=yes
run_tool status --no-verify
[ "$status" -eq 0 ]
[[ "$output" == *"Nothing. Both halves"* ]]
}
@test "a node behind its guest has console work outstanding" {
write_state guest_version=2 node_version=1 node_asserted=2026-06-01 node_verified=yes
run_tool status --no-verify
[ "$status" -ne 0 ]
[[ "$output" == *"console work outstanding"* ]]
# The guest half of v2 is done, so it must not also be listed.
[[ "$output" != *"re-run dejima-devbox-bootstrap from a v2 checkout"* ]]
}
@test "a guest behind its node is named as a restored snapshot" {
# The one direction nobody does on purpose, and the only warning there is.
write_state guest_version=1 node_version=2 node_asserted=2026-08-25 node_verified=yes
run_tool status --no-verify
[ "$status" -ne 0 ]
[[ "$output" == *"guest is behind the node"* ]]
[[ "$output" == *"restoring an old snapshot"* ]]
}
@test "both halves outstanding lists both" {
write_state guest_version=1 node_version=1 node_asserted=2026-06-01 node_verified=yes
run_tool status --no-verify
[[ "$output" == *"console work outstanding"* ]]
[[ "$output" == *"re-run dejima-devbox-bootstrap"* ]]
}
@test "an upgrade needing a snapshot says so in status, not only in the note" {
write_state guest_version=1 node_version=1
run_tool status --no-verify
[[ "$output" == *"snapshot: REQUIRED"* ]]
}
# --- not claiming to know what it cannot ------------------------------------
@test "the installed copy admits it cannot see newer versions" {
# An installed v1 tool would otherwise report "up to date" with total
# confidence, having never heard of v2. This is the same failure as reading
# a timeout as a pass.
share="$BATS_TEST_TMPDIR/share"
mkdir -p "$share/upgrades" "$BATS_TEST_TMPDIR/sbin"
cp "$REPO"/upgrades/0001-*.md "$share/upgrades/"
cp "$TOOL" "$BATS_TEST_TMPDIR/sbin/dejima-devbox-upgrade"
write_state guest_version=1 node_version=1
CONF_DIR="$STATE_DIR" SHARE_DIR="$share" \
run "$BATS_TEST_TMPDIR/sbin/dejima-devbox-upgrade" status --no-verify
[[ "$output" == *"installed copy"* ]]
[[ "$output" == *"cannot tell you about a version"* ]]
}
@test "with no notes reachable it refuses rather than reporting a clean run" {
# A copy that can find no notes knows nothing about any version. The one
# answer it must never give is the reassuring one.
lonely="$BATS_TEST_TMPDIR/lonely"
mkdir -p "$lonely"
cp "$TOOL" "$lonely/dejima-devbox-upgrade"
write_state guest_version=1 node_version=1
CONF_DIR="$STATE_DIR" SHARE_DIR="$BATS_TEST_TMPDIR/absent" \
run "$lonely/dejima-devbox-upgrade" status --no-verify
[ "$status" -eq 2 ]
[[ "$output" == *"no upgrade notes found"* ]]
}
# --- not recording a claim the machine contradicts --------------------------
@test "mark-node refuses when the check for that upgrade fails" {
stub_audit 1
write_state guest_version=2
run_tool mark-node 2
[ "$status" -ne 0 ]
[[ "$output" == *"refusing to record"* ]]
# And nothing was written.
! grep -q '^node_version=' "$STATE_DIR/state"
}
@test "mark-node records when the check passes, and says it verified" {
stub_audit 0
write_state guest_version=2
run_tool mark-node 2
[ "$status" -eq 0 ]
grep -q '^node_version=2$' "$STATE_DIR/state"
grep -q '^node_verified=yes$' "$STATE_DIR/state"
}
@test "--force records the claim but marks it unverified" {
# Kept because a broken network makes some checks unreadable, and refusing
# outright would only teach people to edit the state file by hand.
stub_audit 1
write_state guest_version=2
run_tool mark-node 2 --force
[ "$status" -eq 0 ]
grep -q '^node_version=2$' "$STATE_DIR/state"
grep -q '^node_verified=no$' "$STATE_DIR/state"
}
@test "a missing verify command is not reported as a failed check" {
# A check that could not run has not found anything. Calling that a failure
# trains an operator to ignore the word.
write_state guest_version=2
run_tool mark-node 2
[ "$status" -eq 0 ]
[[ "$output" == *"cannot check"* ]]
grep -q '^node_verified=no$' "$STATE_DIR/state"
}
@test "status contradicts a recorded claim the machine no longer supports" {
# An assertion true in June and false now is a rule somebody removed. Better
# found by a timer than by an incident.
stub_audit 1
write_state guest_version=2 node_version=2 node_asserted=2026-06-01 node_verified=yes
run_tool status
[ "$status" -ne 0 ]
[[ "$output" == *FAILED* ]]
[[ "$output" == *"whatever the state file says"* ]]
}
@test "status confirms a recorded claim the machine still supports" {
stub_audit 0
write_state guest_version=2 node_version=2 node_asserted=2026-08-25 node_verified=yes
run_tool status
[ "$status" -eq 0 ]
[[ "$output" == *"in force"* ]]
}
# --- the notes themselves ---------------------------------------------------
@test "every version from 1 to VERSION has an upgrade note" {
# A missing note and a note saying "nothing to do" are the same thing to
# someone who has to go looking, so absence is not allowed to mean either.
newest=$(tr -d '[:space:]' < "$REPO/VERSION")
for v in $(seq 1 "$newest"); do
ls "$REPO/upgrades/$(printf '%04d' "$v")"-*.md >/dev/null 2>&1 \
|| { echo "no upgrade note for v$v"; false; }
done
}
@test "every note's version field matches its filename" {
for f in "$REPO"/upgrades/0*.md; do
from_name=$(basename "$f" | sed 's/^0*\([0-9][0-9]*\)-.*/\1/')
from_head=$(sed -n '/^---$/,/^---$/p' "$f" | sed -n 's/^version: //p' | head -1)
[ "$from_name" = "$from_head" ] \
|| { echo "$f: filename says $from_name, header says $from_head"; false; }
done
}
@test "every note declares the keys the tool reads" {
for f in "$REPO"/upgrades/0*.md; do
for k in version title layers snapshot; do
sed -n '/^---$/,/^---$/p' "$f" | grep -q "^$k: " \
|| { echo "$f: no '$k' in the header"; false; }
done
done
}
@test "VERSION is a bare integer, since the scheme has no minor part" {
run cat "$REPO/VERSION"
[[ "$output" =~ ^[0-9]+$ ]]
}
@test "the newest note matches VERSION, so a bump cannot be half-done" {
newest_note=$(ls "$REPO"/upgrades/0*.md | sed -n 's/.*\/0*\([0-9][0-9]*\)-.*/\1/p' | sort -n | tail -1)
[ "$newest_note" = "$(tr -d '[:space:]' < "$REPO/VERSION")" ]
}
upgrades/0001-baseline.md
+48
View File
@@ -0,0 +1,48 @@
---
version: 1
title: Baseline, for a devbox built before versions existed
layers: none
snapshot: none
---
# v1 — the baseline
Nothing to do. This version exists so that "no version recorded" has a name and
a place to be written down, rather than being a gap the tooling has to guess
about.
Every devbox built before `VERSION` existed is a v1 devbox. That is true whether
it was built the week this repository started or the week before the change that
introduced versioning, because until then there was one supported state — the
tip of `main` — and no way to be at any other.
## Establishing where you are
`dejima-devbox-upgrade status` on a host with no `/etc/dejima-devbox/state` will
tell you it predates version tracking and stop there. It does not assume v1,
because a host that is *actually* at v1 and a host whose state file somebody
deleted are the same thing from in here, and picking the friendlier reading is
how a check comes to be believed when it does not know.
To record it, run the guest half of the next upgrade. `dejima-devbox-bootstrap`
writes `guest_version` from this repository's `VERSION` file, so a host that has
been bootstrapped from a checkout is at that checkout's version by
construction — there is nothing to type and nothing to get wrong.
For the node half, read `0002` and the ones after it, do the console work each
one describes, and record it with `dejima-devbox-upgrade mark-node`. There is no
shortcut, because there is nothing in the guest that can read the hypervisor.
## If you are not sure your host is at v1
Then say so with the tool rather than assuming, and check the two things a v1
host is most likely to be wrong about:
```sh
sudo dejima-devbox-audit # both halves, as they stand
```
A v1 host in good order reports `PASS` on the v4 half. It has no v6 half at all,
because that check arrived in v2 — so an old audit reporting a clean `PASS` is
reporting on less than a current one would. That is precisely the situation
versions exist to make visible.
upgrades/0002-ipv6-lan-boundary.md
+140
View File
@@ -0,0 +1,140 @@
---
version: 2
title: The LAN boundary gains IPv6 and the CGNAT range
layers: node guest
snapshot: required
verify-node: dejima-devbox-audit reach
---
# v2 — the LAN boundary gains IPv6 and the CGNAT range
Two rules on the hypervisor, one package and a re-run of bootstrap in the guest,
and a fresh snapshot. Budget twenty minutes, most of it waiting for the console.
**Do the node half first.** The guest half installs an audit that checks the
node half, and running it before the rules exist produces a `FAIL` that is
correct, alarming, and not about anything you did wrong.
## Why
The node firewall's outbound DROPs named IPv4 ranges only, so the control this
project exists to provide did not apply to IPv6 at all. Every Linux host brings
up an `fe80::` address on every interface whether or not anything routes v6, and
a daemon bound to `::` answers on it — so the machine beside your devbox was
reachable across the segment with all four DROPs in force, on a network with no
IPv6 in it. `dejima-devbox-audit reach` reported `PASS` throughout, because its
probe was `ip -4 route show default`.
The full argument is in
[the threat model](../docs/threat-model.md#the-lan-boundary-is-two-boundaries).
The short version: IPv6 has no RFC 1918, so the family is dropped whole rather
than enumerated.
## 1. Node — two rules in the Proxmox console
**Datacenter → Firewall → Security Group → `dejima-devbox-hardener`.**
Add these two. They are rules 11 and 12 of
[the table in the runbook](../proxmox/README.md#build-the-group-once):
| # | Direction | Action | Protocol | Dest. port | Source / Dest |
| --- | --- | --- | --- | --- | --- |
| 11 | out | DROP | — | — | **Dest:** `100.64.0.0/10` |
| 12 | out | DROP | — | — | **Dest:** `::/0` |
**Both go at the bottom, below every existing rule.** The console inserts each
new rule at the *top* of the list, so a rule left where it lands sits above your
forge and resolver pinholes. These two are DROPs; put them above the pinholes
and the devbox stops reaching its forge. Drag each one down as you add it.
Nothing else in the group changes. You are appending, not re-entering.
**If the console rejects `::/0`**, enter four rules in rule 12's position
instead — `fe80::/10`, `fc00::/7`, `ff00::/8`, `2000::/3` — which cover the same
space. Prefer the single line where it is accepted: four entries are four
chances to enter three.
**Nothing to restart.** A security group edit applies to every VM referencing it
immediately. If you built the rules per-VM rather than as a group, you have this
to do on each VM, and building the group is worth doing while you are here.
### Check the node half landed
On the node:
```sh
sudo ip6tables -S | grep "tap<vmid>i0"
```
An IPv6 rule is compiled into `ip6tables` and is **absent from `iptables -S`**,
so the v4 table shows rule 12 missing whether it is there or not. Checking the
wrong table is the most likely way to conclude this worked when it did not.
## 2. Guest — the package, then bootstrap
```sh
sudo apt-get install -y iputils-ping
```
New dependency. The audit's IPv6 check asks the segment who is there and needs
`ping` to do it; without it the check reports `INDETERMINATE`, which is not a
pass. A standard Debian install has it and a minimal or cloud image often does
not.
Then, from a checkout of this repository at v2 or later:
```sh
git -C ~/dejima-devbox-hardener pull # or clone it fresh
cd ~/dejima-devbox-hardener
sudo ./bin/dejima-devbox-bootstrap
```
Bootstrap is idempotent. It will re-prompt for the forge hostname and the
account, offering what you already have as the defaults, and pressing Enter at
both changes nothing. What it does that matters here is install the new
`dejima-devbox-audit`, install `dejima-devbox-upgrade`, and write
`guest_version=2` into `/etc/dejima-devbox/state`.
It also reinstalls the systemd timer, which is harmless and is how the unit
picks up any change to it.
### Check the guest half landed
```sh
sudo dejima-devbox-audit reach
```
Two `PASS` lines now, not one. The second is the IPv6 half. If it says `FAIL`,
the node work above is not in force — go back to §1 and check `ip6tables`, not
`iptables`. If it says `INDETERMINATE`, `ping` is missing.
## 3. Record the node half
```sh
sudo dejima-devbox-upgrade mark-node 2
```
This runs `dejima-devbox-audit reach` first and refuses to record the assertion
if it does not pass, so the thing being written down has been corroborated
rather than merely claimed.
## 4. Snapshot
**Take a fresh snapshot, and delete the pre-upgrade one.**
A devbox has no scheduled rollback, so nothing undoes this on a timer. What
would undo it is you, later, restoring a snapshot taken this morning — which
boots fine, passes its own v1 checks, and is missing the IPv6 rule's guest-side
counterpart while the node keeps the rule. `dejima-devbox-upgrade status` would
then report `guest_version` behind `node_version`, which is the pair disagreeing
on purpose.
Keeping the old snapshot is keeping a bootable copy of the vulnerability. If you
have a reason to keep it, name it so that its date is obvious in the list.
## What this does not change
Internet egress. Still unbounded from the VM, still deliberate, and the reason
is in `proxmox/firewall.fw.template` above `policy_out`. Rule 12 removes a second
route to the internet and a first route to your LAN; it does not narrow the
first route.
upgrades/README.md
+75
View File
@@ -0,0 +1,75 @@
# upgrades
One file per version. `000N-slug.md` is what an operator has to do to reach
version `N` from version `N-1`, and there is one for every version including the
ones that need nothing — an absent file would be indistinguishable from a file
nobody wrote.
`dejima-devbox-upgrade` reads these. So does a person, and the file is written
for the person; the header exists so the tool can sort, filter and check without
the prose having to be machine-readable.
## Why this exists at all
dejima has no equivalent and does not need one. Its containers are short-lived,
only the tip of `main` is supported, and `git pull` closes any gap that opens.
A devbox is the opposite. It is a long-lived VM pinned to the moment it was
built, its hypervisor rules were entered by hand in a console no `git pull` can
reach, and a snapshot restore can move it *backwards* without anybody deciding
to. Nothing about it converges on the current version on its own, so the gap has
to be measured and closed deliberately.
## The header
```
---
version: 2
title: The LAN boundary gains IPv6 and the CGNAT range
layers: node guest
snapshot: required
verify-node: dejima-devbox-audit reach
---
```
| Key | Means |
| --- | --- |
| `version` | The version this upgrade arrives at. Must match the filename. |
| `title` | One line, shown in `status`. |
| `layers` | `node`, `guest`, both, or `none`. Which halves need work. |
| `snapshot` | `required`, `recommended`, or `none`. See below. |
| `verify-node` | Optional. A command that passes only if the node-side work is actually in force. |
## Two version numbers, not one
The guest half and the node half are tracked separately, in
`/etc/dejima-devbox/state`, because they are known in different ways.
`guest_version` is written by `dejima-devbox-bootstrap` when it installs. It is
a record of something this repository did, so it is as reliable as the install.
`node_version` is written by `dejima-devbox-upgrade mark-node`, by an operator
saying they did the console work. Nothing in the guest can read the hypervisor's
rules — that is the whole reason those rules are worth having — so this is an
assertion, and the tool treats it as one. Where an upgrade names a
`verify-node` command, the tool runs it and will contradict the assertion.
Keeping them apart is what makes a snapshot restore legible. Restore a devbox to
an image from before an upgrade and `guest_version` goes backwards while
`node_version`, which lives on the hypervisor, does not. The pair disagreeing is
a fact worth reporting; a single number would have quietly averaged it.
## `snapshot:`
`required` means the change is lost without a new snapshot, or that an old
snapshot becomes dangerous to restore once the change is made.
A devbox has no scheduled rollback — that is
[rejected, with reasons](../docs/threat-model.md#rejected-with-reasons), because
rollback destroys uncommitted work. So an upgrade is not undone on a timer here
the way it is on a CI runner. What it *is* exposed to is a manual restore: a
snapshot taken before an upgrade will still boot, still pass its own checks, and
still be missing the thing the upgrade added.
Take a fresh snapshot after upgrading, and prefer deleting the pre-upgrade one
to keeping it. A snapshot is a security posture with a date on it.