Give the privileged tier a working v1 -> v2 path, and link the upgrade docs
Two findings, both from reading the path rather than the code. THE "RECORD IT PROPERLY" BLOCK WAS CONFINED-SHAPED. It ran install.sh - which does not run on a privileged host, deliberately - and then invoked dejima-ci-upgrade bare, which only works because install.sh had just put it there. A privileged operator following it got exactly the error that started this: the tool reporting it could not find itself. There is now a block per tier. Confined keeps install.sh. Privileged gets the three install commands bootstrap uses, because bootstrap is the only thing that installs the tool on that tier and re-running it on a built machine also redoes runner setup - a real cost for a machine that is already serving. Those three duplicate bootstrap's lines and can drift from them, so a test pins both ends to the same destinations. A tool installed to the wrong path is a tool that is not there, reported three sections later as "not installed here", which is how this whole thread started. THE UPGRADE DOCS WERE NOT LINKED FROM THE README AT ALL. Only docs/README.md mentioned them, so an operator with an existing runner had no path from the front page to the flow. There is now an "Updating an existing runner" section with the command to check a version, the instruction to walk the notes in order, and the two things a runner operator needs before starting: run from a checkout, and recording is itself a guest-side change. The runbook and the docs index point at it, and so does the intro, where someone skimming will be. `bats test/` 136 passing, 1 new.
This commit is contained in:
@@ -43,6 +43,9 @@ For *why* any of this is necessary - what a CI run adds over a sandbox, and
|
||||
why no approval gate is offered - see [dejima's `docs/ci.md`](https://gitbox.branderson.io/dejima/dejima/src/branch/main/docs/ci.md). You do
|
||||
not need it to follow this.
|
||||
|
||||
**Already have a runner?** [Updating an existing runner](#updating-an-existing-runner)
|
||||
is the path forward: check its version, then walk the notes in order.
|
||||
|
||||
**Early development. Not ready to be depended on.** Nothing here has been
|
||||
through outside review, an external audit, or adversarial testing by anyone but
|
||||
the person who wrote it. Every security property this describes is a claim
|
||||
@@ -55,6 +58,35 @@ where to report a vulnerability rather than in a public issue.
|
||||
|
||||
[](https://gitbox.branderson.io/dejima/dejima)
|
||||
|
||||
## Updating an existing runner
|
||||
|
||||
**Check what it is at**, from a checkout rather than the installed copy:
|
||||
|
||||
```sh
|
||||
git -C ~/dejima-ci-hardener pull
|
||||
cd ~/dejima-ci-hardener
|
||||
sudo ./bin/dejima-ci-upgrade set-tier confined # or privileged, once per host
|
||||
sudo ./bin/dejima-ci-upgrade status
|
||||
```
|
||||
|
||||
That names every version this runner is missing. **Do them in order** — each
|
||||
note assumes the one before it.
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| [upgrades/](upgrades/) | One note per version: the console work, the guest work, and what it costs |
|
||||
| [docs/upgrading.md](docs/upgrading.md) | The process, including the seven steps a guest-side change takes when a rollback is going to rebuild the machine |
|
||||
|
||||
**From a checkout, not the installed copy**, for two reasons: an installed copy
|
||||
knows only the versions that existed when it was installed, and on a privileged
|
||||
host there may be no installed copy at all — `install.sh` does not run there.
|
||||
|
||||
**Recording a version is itself a guest-side change**, so on a runner it is
|
||||
thrown away at the next rollback unless it is baked into the snapshot. The
|
||||
console work is not: hypervisor rules are in force either way. `docs/upgrading.md`
|
||||
covers the choice; `status` re-runs each upgrade's check before calling its work
|
||||
outstanding, so a rebuilt runner is not told to redo something already done.
|
||||
|
||||
## Do not install this next to dejima
|
||||
|
||||
This runs on a machine whose job is to execute untrusted code. dejima runs on
|
||||
|
||||
+2
-1
@@ -9,7 +9,8 @@ where the second one lives.
|
||||
| [building-a-runner.md](building-a-runner.md) | The build, inside the VM: the Debian install, addressing, one bootstrap command, `install.sh`'s settings, and the runners. The same on any hypervisor and for both tiers. Start at your hypervisor's page and it sends you here |
|
||||
| [reference.md](reference.md) | The two settings named `docker.sock`, `CONFIG_FILE` and why an edit can appear to do nothing, the `/data` volume, compose networks against job networks, whether several runners should share a job network, why the privileged runner is separate, why public egress is not restricted on the node, why the resolver is set per-runner, the cache server |
|
||||
| [threat-model.md](threat-model.md) | The adversary, where each tier's boundary sits, blast radius by layer, and what was rejected |
|
||||
| [upgrading.md](upgrading.md) | Moving a runner that already exists to a newer version, why the two halves are counted separately, and the seven steps a guest-side change takes when a rollback is going to rebuild the machine |
|
||||
| [upgrading.md](upgrading.md) | Moving a runner that already exists to a newer version, why the two halves are counted separately, and the seven steps a guest-side change takes when a rollback is going to rebuild the machine. Start at [Updating](../README.md#updating-an-existing-runner) |
|
||||
| [../upgrades/](../upgrades/) | One note per version: what an operator does in the console and in the guest to reach it |
|
||||
| [../proxmox/README.md](../proxmox/README.md) | The node-side half on Proxmox: the VM, the datacenter firewall, the security group and the order its rules go in, and all three snapshot moments including the scheduled rollback |
|
||||
| [../proxmox/firewall.fw.template](../proxmox/firewall.fw.template) | The same node firewall as one annotated file, easier to review in one piece than as a table. Reference, not something to install |
|
||||
| [../examples/](../examples/) | The annotated templates `dejima-ci-runners` renders into `~/runners`. Every setting carries the reasoning for its value, which the generated copies do not - read them here rather than reading what lands on the host |
|
||||
|
||||
@@ -6,6 +6,11 @@ same on any hypervisor.
|
||||
|
||||
## The whole job
|
||||
|
||||
**Already have a runner?** This page builds one. To bring an existing VM
|
||||
forward, start at [Updating](../README.md#updating-an-existing-runner) — the
|
||||
upgrade notes name the rules to change rather than having you re-read the table
|
||||
below.
|
||||
|
||||
Four steps, and the two pages interleave — follow this order rather than either
|
||||
page top to bottom.
|
||||
|
||||
|
||||
@@ -276,6 +276,21 @@ stub_check() {
|
||||
|| { echo "bootstrap does not record tier=privileged"; false; }
|
||||
}
|
||||
|
||||
@test "the hand-install recipe writes where bootstrap writes" {
|
||||
# v2 gives a privileged operator three install commands, because install.sh
|
||||
# does not run on that tier and re-running bootstrap on a built machine
|
||||
# redoes runner setup. Those commands duplicate bootstrap's, so they can
|
||||
# drift from it - and a tool installed to the wrong path is a tool that is
|
||||
# not there, reported as "not installed here" three sections later.
|
||||
branch=$(awk '/elif \[ "\$ROLE" = privileged \]/,/^else$/' "$REPO/bin/dejima-ci-bootstrap")
|
||||
for dest in /usr/local/sbin/dejima-ci-upgrade /usr/local/share/dejima-ci/upgrades; do
|
||||
printf '%s' "$branch" | grep -qF "$dest" \
|
||||
|| { echo "bootstrap no longer installs to $dest"; false; }
|
||||
grep -qF "$dest" "$REPO/upgrades/0002-ipv6-lan-boundary.md" \
|
||||
|| { echo "v2's hand-install recipe no longer names $dest"; false; }
|
||||
done
|
||||
}
|
||||
|
||||
@test "every version from 1 to VERSION has an upgrade note" {
|
||||
newest=$(tr -d '[:space:]' < "$REPO/VERSION")
|
||||
for v in $(seq 1 "$newest"); do
|
||||
|
||||
@@ -114,21 +114,38 @@ So there are two honest ways to do this, and no third.
|
||||
If you want the record to survive, it takes the same sequence any guest-side
|
||||
change takes — [the seven steps](../docs/upgrading.md#a-guest-side-upgrade-is-not-one-step).
|
||||
|
||||
And if you are doing the cycle anyway, **run `install.sh` while you are in
|
||||
it** (confined tier only). It puts `dejima-ci-upgrade` and these notes on the
|
||||
box, so later versions can be read and recorded without a checkout. Doing the
|
||||
dance for `mark-node` alone and skipping the install would be the expensive half
|
||||
of the work for the smaller half of the benefit.
|
||||
**Install the tool in the same cycle.** A record that survives a rollback is
|
||||
worth little if the program that reads it does not, and the cycle is the
|
||||
expensive part either way.
|
||||
|
||||
**Confined tier**, where `install.sh` does it:
|
||||
|
||||
```sh
|
||||
# ... after rolling back to the clean snapshot, per the seven steps
|
||||
cd ~/dejima-ci-hardener && git pull
|
||||
sudo ./install.sh # confined tier only
|
||||
sudo ./install.sh
|
||||
sudo dejima-ci-upgrade set-tier confined
|
||||
sudo dejima-ci-upgrade mark-node 2
|
||||
# ... then re-snapshot and repoint the rollback job
|
||||
```
|
||||
|
||||
**Privileged tier**, where nothing does. `install.sh` does not run here — that
|
||||
is the grant the tier exists for — so the tool arrives only by
|
||||
`dejima-ci-bootstrap`, and re-running that on a built machine also redoes runner
|
||||
setup. The three commands it uses for this part are enough on their own:
|
||||
|
||||
```sh
|
||||
# ... after rolling back to the clean snapshot, per the seven steps
|
||||
cd ~/dejima-ci-hardener && git pull
|
||||
sudo install -m 0755 bin/dejima-ci-upgrade /usr/local/sbin/dejima-ci-upgrade
|
||||
sudo install -d -m 0755 /usr/local/share/dejima-ci/upgrades
|
||||
sudo install -m 0644 upgrades/*.md /usr/local/share/dejima-ci/upgrades/
|
||||
sudo dejima-ci-upgrade set-tier privileged
|
||||
sudo dejima-ci-upgrade mark-node 2
|
||||
# ... then re-snapshot and repoint the rollback job
|
||||
```
|
||||
|
||||
|
||||
### Or: skip it, and let the check answer for you
|
||||
|
||||
Perfectly reasonable, and probably right for v2 specifically. The console work
|
||||
|
||||
Reference in New Issue
Block a user