Compare commits

...
Author SHA1 Message Date
branderson 5809d998c6 ci: name the runner-env jobs after the jobs they verify
tests / sandbox (proxy) (pull_request) Successful in 2m6s
tests / runner-env (sandbox) (pull_request) Failing after 1m27s
tests / sandbox (static) (pull_request) Successful in 2m56s
tests / sandbox (config) (pull_request) Successful in 5m28s
tests / host-docker (pull_request) Successful in 5m30s
tests / runner-env (host-docker) (pull_request) Successful in 1m22s
tests / sandbox (providers) (pull_request) Successful in 4m2s
tests / sandbox (remote) (pull_request) Successful in 4m46s
tests / sandbox (project) (pull_request) Successful in 5m23s
tests / sandbox (audit) (pull_request) Successful in 7m32s
"runner-env (runtime)" did not say which job's environment it was checking.
The sandbox suites are all named "sandbox (something)", so "runner-env
(sandbox)" reads correctly, but the other one is "host-docker" and nothing
connected "runtime" to it except knowing that the runner label is
dejima-runtime.

Both the job name and DEJIMA_CI_TIER now use the job names - sandbox and
host-docker - rather than a separate vocabulary. A reader looking at a red
check should not have to work out which job it corresponds to.

Also records the seam that produced a wrong set of instructions for this:
dejima-ci-hardener manages the allowlist and installs dejima-ci-allowlist. It
does not install the firewall, which is ci/iptables-ci.example.sh by hand. An
operator who installed the hardener and stopped has a working resolver and no
packet filter - exactly the state test/ci-runner.bats reports as the DNS
checks passing while the egress checks fail, which is what happened.
2026-08-05 16:19:19 -07:00
branderson 41f065077b ci: refuse to apply the firewall without ipset, and reapply it after Docker
tests / sandbox (proxy) (pull_request) Successful in 2m22s
tests / runner-env (sandbox) (pull_request) Failing after 1m27s
tests / sandbox (remote) (pull_request) Successful in 4m47s
tests / sandbox (project) (pull_request) Successful in 5m19s
tests / sandbox (audit) (pull_request) Successful in 7m29s
tests / sandbox (static) (pull_request) Successful in 3m2s
tests / sandbox (providers) (pull_request) Successful in 4m0s
tests / sandbox (config) (pull_request) Successful in 5m25s
tests / host-docker (pull_request) Successful in 5m31s
tests / runner-env (runtime) (pull_request) Successful in 1m23s
Two ways the egress rules end up absent while everything looks configured.
They present identically, and neither is visible from the runner host: DNS
still refuses unlisted names, because that is dnsmasq's doing rather than the
firewall's, while jobs have unrestricted egress.

The first is that Docker removes them. Every daemon start recreates its own
chains and DOCKER-USER comes back empty, so a `systemctl restart docker` for
any reason - including the one this bundle asks for when installing
daemon.json - silently drops the policy days after it was verified. "Apply the
rules once" was never a working instruction. ci/dejima-ci-firewall.example.service
reapplies them on every Docker start: PartOf plus WantedBy on docker.service,
because a plain After= only covers boot, which was not the case that broke.

The second is that the script's first real command is `ipset create`, so on a
host without ipset it aborts at line one under set -e. "command not found"
scrolling past during setup is easy to miss, and what it leaves behind passes
a casual look. It now checks for ipset, iptables and ip6tables by name and
says what the absence costs, and ci/README.md gained the package step it never
had.

The other half of this is that the failure is now detectable from inside a
job rather than by inspecting the host, which is what test/ci-runner.bats is
for - and the combination it reports is diagnostic on its own: DNS checks
passing while the egress checks fail means the resolver is fine and the packet
filter is not. The off-list-destination test now says so in its failure
output, including where to look and which unit fixes it; it was the only one
of the pair without an explanation.

Also corrected: the setup table still listed daemon.json as a place the bridge
gateway is used, which stopped being true when the resolver moved to the
runner's --dns.
2026-08-05 16:12:17 -07:00
branderson 2282ca3e2d ci: scope the job resolver to its runner, not to the whole daemon
tests / sandbox (proxy) (pull_request) Successful in 2m36s
tests / runner-env (sandbox) (pull_request) Failing after 1m26s
tests / sandbox (static) (pull_request) Successful in 3m21s
tests / sandbox (providers) (pull_request) Successful in 4m23s
tests / sandbox (remote) (pull_request) Successful in 5m7s
tests / sandbox (project) (pull_request) Successful in 5m43s
tests / sandbox (config) (pull_request) Successful in 5m48s
tests / host-docker (pull_request) Successful in 5m50s
tests / sandbox (audit) (pull_request) Successful in 7m53s
tests / runner-env (runtime) (pull_request) Successful in 1m22s
daemon.json's dns applies to every container the daemon starts. On a host that
also runs the privileged tier, that includes the sibling containers its jobs
create over the Docker socket - which are not job containers and were never
meant to be behind the CI resolver.

A dejima sandbox is exactly one of those, and it has a firewall of its own.
init-firewall.sh allowlists precisely one host address, HOST_IP, taken from
the container's own default route - its bridge gateway. dns-proxy.sh forwards
allowlisted names to 127.0.0.11, Docker's embedded resolver, which then
forwards to whatever the daemon was configured with. With that set to a
resolver on a different bridge, the forward leaves the sandbox's namespace and
is rejected by the sandbox's own OUTPUT rule, so nothing inside it resolves at
all.

The symptom is not "DNS is misconfigured". It is the sandbox's startup
self-check reporting that api.github.com - a host it explicitly allows - is
unreachable, and aborting the entrypoint before any test runs. The CI
allowlist containing that name is irrelevant, because the container cannot
reach the resolver holding it.

--dns in container.options scopes the resolver to the containers that runner
starts, which is the set that should have it. daemon.json keeps only the log
bounds and live-restore, and says why it does not set dns. The breadth of the
daemon-wide form was noted as a risk when it was written and shipped anyway,
so static.bats now refuses it: no dns key in the daemon example, --dns present
in the runner's options, and the explanation still in ci/README.md.

runtime.bats' wait_ready also stops polling a container that has already
exited. An aborted entrypoint used to be waited out for the full two minutes
before the logs explaining it were printed, which reads as a hang rather than
as the startup failure it is. The reason was always in the logs; this only
decides how long it takes to reach them.
2026-08-05 15:54:48 -07:00
branderson c16a2edfb9 ci: assert the runner's own configuration from inside a job on it
tests / sandbox (providers) (pull_request) Has been cancelled
tests / sandbox (proxy) (pull_request) Has been cancelled
tests / sandbox (remote) (pull_request) Has been cancelled
tests / sandbox (static) (pull_request) Has been cancelled
tests / runner-env (sandbox) (pull_request) Has been cancelled
tests / runner-env (runtime) (pull_request) Has been cancelled
tests / sandbox (audit) (pull_request) Has been cancelled
tests / host-docker (pull_request) Has been cancelled
tests / sandbox (config) (pull_request) Has been cancelled
tests / sandbox (project) (pull_request) Has been cancelled
Every other suite here tests dejima's code. This one tests the machine
executing it, because the controls docs/ci.md describes live in a runner's
config.yaml and a host's firewall - places no amount of reviewing this
repository can check.

It exists because both failures that matter look exactly like success from
inside a job. Nothing in a job's log mentions whether the Docker socket was
mounted into it, and a job that can reach an off-list resolver still passes
every check that only inspects DOCKER-USER. Neither presents as a red run;
they present as nothing at all, which is why they were a Verify section
someone had to remember to perform rather than a test.

Two jobs, one per tier, because the two runners are separately configurable
and either can be wrong on its own - one red check per tier says which. The
tier is declared by the workflow rather than sniffed, so landing on the wrong
runner fails instead of quietly selecting a different set of assertions. The
socket is asserted in both directions: absent on the sandbox tier, present on
the runtime tier, so a mis-scheduled job is caught whichever way it went.

The egress checks are scoped to the sandbox tier deliberately. The runtime
tier's jobs run in the host network namespace - that is the grant they exist
for - so the bridge rules never sit in front of them, and asserting an
allowlist there would test nothing and pass anyway, which is the failure mode
this whole suite is about.

Every negative check has a positive control behind it. An allowlisted
destination must resolve and connect first, because a runner with no network
at all passes every negative assertion otherwise. The off-list address is one
that would answer if it were reachable, for the same reason - pointing it at a
documentation range would have made the check vacuous. Its host is
overridable, since an allowlist is per-operator and hardcoding one would fail
on a runner that is correctly configured for a different set. The IPv6 check
is the one exception and says so: it passes both when v6 is rejected and when
it was never configured, and both are acceptable.

Verified by running the suite inside a dejima sandbox, which is itself a
firewalled container: every negative check passed and the positive control
correctly failed on a name that sandbox does not allow, rather than letting
the negatives pass for the wrong reason.

The existing "every sandbox suite is in the CI matrix" guard caught this file
on the way in, which is what it is for. It now checks that every suite is run
by CI somewhere - the two that cannot be matrix entries needing a job of their
own instead - so "not in the matrix" cannot become a way to be in neither.
2026-08-05 15:06:19 -07:00
branderson 2510a46856 ci: make ci/README.md a walkthrough, and stop docs/ci.md duplicating it
tests / sandbox (proxy) (pull_request) Successful in 2m23s
tests / sandbox (static) (pull_request) Has been cancelled
tests / sandbox (config) (pull_request) Has been cancelled
tests / host-docker (pull_request) Has been cancelled
tests / sandbox (audit) (pull_request) Has been cancelled
tests / sandbox (remote) (pull_request) Has been cancelled
tests / sandbox (project) (pull_request) Has been cancelled
tests / sandbox (providers) (pull_request) Has been cancelled
Following the setup meant jumping between three places and reading two walls
of prose to find the ordered steps, which sat at the bottom of ci/README.md
under a heading that did not look like the thing to start from. The reasoning
was good and in the way.

ci/README.md is now the walkthrough, in the order someone performs it. A
"before you start" section names the four values that get substituted and where
each is used, so they can be gathered once instead of discovered one file at a
time. Then seven numbered steps, each naming its own file, destination and the
lines to edit, with the two failure-open warnings attached to the steps they
belong to rather than collected somewhere else. Verification follows, then a
file map for looking up a path later. Everything that was explanation moved
below a Reference heading and is marked as not needed to complete the setup.

The upstream resolver gets its own explanation, because "set upstream to a
resolver" does not say which one and there are three plausible readings. It is
the resolver dnsmasq forwards allowed queries to - not the dnsmasq being
configured, not the bridge gateway - and it has to be able to resolve the
forge, so it is whatever this host already uses. The commands to find that are
in the section now, along with why no-resolv makes naming it necessary.

docs/ci.md stops being a partial second runbook. It kept literal copies of the
network create, the resolver config and the firewall rules, which is both a
second thing to read and a drift risk that had already cost one fix in two
places. Those are replaced by the reasoning they were illustrating plus a
pointer to ci/, and the document now says at the top that it is the why and
that ci/README.md does not assume it has been read. The README's CI section
names both, labelled why and how.

The exemption test for the recommended private subnet now checks only
ci/README.md, since docs/ci.md deliberately carries no concrete configuration
and requiring it there would recreate the duplication this removes.
2026-08-05 13:09:54 -07:00
branderson abb440810b ci: say where every file is installed, and give the subnet a real example
tests / sandbox (config) (pull_request) Successful in 4m39s
tests / sandbox (proxy) (pull_request) Successful in 1m52s
tests / sandbox (project) (pull_request) Successful in 4m37s
tests / sandbox (providers) (pull_request) Successful in 3m22s
tests / sandbox (audit) (pull_request) Successful in 6m39s
tests / sandbox (static) (pull_request) Successful in 2m33s
tests / sandbox (remote) (pull_request) Successful in 4m7s
tests / host-docker (pull_request) Has been cancelled
Two things a reader could not get from the bundle. Every file in it was inert
until put somewhere, and nothing said where - the paths are not guessable
either, since daemon.json's dns has to match the resolver's listen-address and
the refresh script writes into the same dnsmasq.d the static config lives in.
There is now a table of destination paths, the two files holding the
operator's own values and the modes they want, and the one file nobody edits
by hand.

Three notes go with it, each covering a way the install goes quietly wrong: a
daemon.json that already exists wants merging rather than replacing, and a
syntax error there stops Docker starting at all; restarting Docker recreates
its own chains and drops DOCKER-USER, so the firewall is reapplied after a
daemon restart and not only at boot, and a runner with no egress policy looks
exactly like one with; and the resolver's config is two files in dnsmasq.d
that merge, which is only obvious once said.

That split is now real rather than nominal. The example resolver config
carried both the static settings and a set of server= lines, while the refresh
script rendered the same names into a different file - so the base set existed
in two places and the hand-maintained copy could drift from the one derived
from main. The example now declares no names at all and says why: a server=
line there is a grant nobody merged, and it would survive every refresh.

"Substitute a private range" gains the example it needed, and the reason
private is the word rather than a synonym for tidy. Using another
documentation range would work - the addresses are masqueraded on the way out
and TEST-NET is not routed - but stop-dns-rebind is keyed on the private
ranges, so a reserved-but-not-private bridge subnet is one that check silently
does not cover, and nothing in the configuration shows the difference. The
suggested range is also outside Docker's default pool, so an unrelated
docker network create cannot land on top of it.

That example is a literal private address in a public repo, which is what
static.bats' leak scanner exists to prevent, so it is exempted by exact prefix
rather than by loosening the rule: every other 10.x, 192.168.x, 172.16-31.x
and 169.254.x still fails, including a different 10.x sharing a line, because
the filter drops matched addresses rather than whole lines. Three octets is
all the pattern matches, which makes the exemption exactly the /24 the docs
recommend. A second test ties the exemption to that recommendation, so a
changed recommendation cannot leave a carve-out covering an address nothing
points at.

Also pinned: the resolver example declares no allowed names, and every
example file is named in the install table.
2026-08-05 11:28:10 -07:00
branderson 57e7003c5c ci: show several ordinary runners in one compose file, privileged in its own
tests / sandbox (config) (pull_request) Successful in 4m32s
tests / sandbox (project) (pull_request) Successful in 4m33s
tests / sandbox (proxy) (pull_request) Successful in 1m54s
tests / sandbox (providers) (pull_request) Successful in 3m24s
tests / sandbox (audit) (pull_request) Successful in 6m27s
tests / sandbox (static) (pull_request) Successful in 2m27s
tests / sandbox (remote) (pull_request) Successful in 4m2s
tests / host-docker (pull_request) Has been cancelled
Two questions the bundle left unanswered: how to actually run a runner, and
what happens when several are on one host.

Compose's networks and the job network are different worlds, which is the part
worth knowing before deciding the arrangement is unsafe. Every service in a
compose file is an act_runner process; the containers that execute
agent-authored code are created by the daemon, on the network named by
container.network in each runner's config.yaml, and compose knows nothing
about them. Attaching a service to a compose network does not put its jobs
there. So the network several runners share holds runner processes and nothing
else, and those are mutually trusted - each already holds the Docker socket,
so separating them protects nothing. The example names one `runners` network
explicitly rather than leaning on the implicit default, so nothing joins it
later by accident.

What is genuinely shared, with one compose file or five, is the job network:
every runner whose config names dejima-ci puts its jobs on that bridge, so N
runners at capacity 1 means up to N jobs able to reach each other and each
other's service containers. That is presented as a decision rather than
something to inherit - one shared job network is the default and is why
several ordinary runners can share a single config file, and the per-runner
alternative is spelled out with what it costs: a config file and a network
each, rules applied per bridge, and the wrinkle that is not obvious, which is
that daemon.json's dns is a single daemon-wide address sitting on one bridge
and unreachable from the others. The ipset is shared across bridges either
way; it holds destinations, not per-bridge state.

The privileged runner ships as its own compose file rather than a service
alongside the others. A job on it can take the machine it runs on, so that
machine should exist for nothing else - and a compose file that assumes a
separate host is harder to drift away from than one that has to be split
later. Where the two tiers do share a machine, the docs name that for what it
is: label tiering, not host separation.

The mount notes cover what an operator running one of these already has in
front of them. The socket bind is the runner's own access to the daemon and is
required on both tiers; container.docker_host is what decides whether job
containers get it. Same path, opposite jobs, and neither substitutes for the
other. The /data volume holds .runner, the registration identity, which is the
answer to whether it is needed: keep it and a recreate brings back the same
runner, drop it and the container either registers a second one or fails to
start, and deleting it is how to re-register deliberately. It is also the only
state, which is what lets the rest of this directory treat a runner as
disposable. Also recorded: CONFIG_FILE, without which the config is not read
at all and every default applies silently; that labels come from config.yaml
on each start rather than being frozen at registration; and that the
registration token is dead weight once .runner exists.

static.bats pins the socket bind, CONFIG_FILE and the read-only config mount
across both examples, that no service attaches to the job network, and that
the shared file carries no privileged runner - a runtime service appearing
there would work, and would quietly make co-location the default. That last
guard reads non-comment lines only, so the file can still point at its
counterpart in prose. Both new guards were confirmed to fail when violated.
2026-08-05 10:55:20 -07:00
branderson 578ff76033 ci: document running the runner itself, and what its volumes hold
tests / sandbox (providers) (pull_request) Successful in 3m28s
tests / sandbox (proxy) (pull_request) Successful in 2m1s
tests / sandbox (project) (pull_request) Successful in 4m51s
tests / sandbox (config) (pull_request) Successful in 4m55s
tests / sandbox (static) (pull_request) Successful in 2m38s
tests / sandbox (audit) (pull_request) Successful in 7m7s
tests / sandbox (remote) (pull_request) Successful in 4m16s
tests / host-docker (pull_request) Failing after 12m10s
The bundle described how to configure a runner and said nothing about how to
run one, which left the mounts an operator already has - or is about to copy
from somewhere - unexplained. The docker socket bind in particular reads like
the thing this whole directory argues against.

It isn't, and the pair is the easiest thing here to conflate: the compose bind
is how the runner reaches the daemon to create job containers, while
container.docker_host: "-" is what stops that socket being passed into them.
Same path, opposite jobs, and neither substitutes for the other - setting the
second does not remove the first, and dropping the first does not harden
anything, it stops the runner working. Both are now stated side by side.

The /data volume gets the answer to the question actually being asked, which
is whether it is needed: it holds .runner, the registration identity and the
runner's credential for the instance. Keep it and a recreate brings back the
same runner; drop it and the container either registers a second one, leaving
a dead entry behind, or fails to start with no token to register with.
Deleting it is also the supported way to re-register deliberately. It is the
only state on the machine, which is what lets the rest of this directory treat
a runner as disposable.

Also recorded: CONFIG_FILE, without which the config is not read at all and
every default applies silently - including the socket reaching every job -
which is the first thing to check when an edit here appears to do nothing;
that labels come from config.yaml on each start rather than being frozen at
registration; that the registration token is dead weight once .runner exists
and is worth removing from a machine that runs untrusted code; and that the
runner is deliberately neither attached to the job network nor behind the
egress allowlist, since the rules exist to keep jobs away from it.

The privileged runner's compose is described as its three deltas rather than
shipped as a second file, because a near-duplicate's failure mode is someone
copying it for the wrong role. Its /tmp/dejima-ci path needs no bind here: the
host daemon resolves the source host-side, so the runner container never sees
it.

static.bats pins the socket bind, CONFIG_FILE in both the example and the
prose, the read-only config mount, and that the runner is not attached to a
network. That last one greps rather than parsing YAML - the job container has
python3 but no PyYAML, and adding a package host to the allowlist to check an
example file would cost more than the check is worth.
2026-08-05 09:53:42 -07:00
branderson c406b44eef ci: ship the runner setup docs/ci.md describes, and adopt it here
docs/ci.md argued that CI's egress allowlist should be deny-by-default and
read from origin/main. This is that setup as files, and this repo moving onto
it.

ci/ holds a worked runner, resolver and firewall configuration, adaptable
rather than runnable: RFC 5737 addresses, RFC 2606 names, and bridge names and
labels that have to match what an operator actually creates. Nothing installs
it and nothing here runs it - it belongs on a CI machine, which should not be
a machine that runs dejima.

The two act_runner configs are written against v0.6.1 and differ in one line
that decides everything. The ordinary one sets container.docker_host: "-",
which is the only value that stops the socket being mounted into every job
container; the privileged one leaves it unset on purpose, and says in its
header that a job on it can take the host. Both carry only a label nothing
else does, because whether a job gets the socket is a property of the runner
rather than of the job - a job asking for nothing unusual still receives
whatever its runner grants everything it runs.

ci/README.md gives the order things have to be created in, since two steps
fail open rather than loudly when taken out of order: an ipset has to exist
before a rule names it, and restarting the daemon recreates Docker's chains.
It also gives two checks worth running after any change to the host, because
both failures that matter look exactly like success from inside a job - a
mounted socket is not mentioned anywhere in a job's log, and a job that can
reach an off-list resolver still passes every test that only looks at
DOCKER-USER.

Adoption here: the sandbox matrix moves from ubuntu-latest to dejima-ci. These
suites do genuinely run anywhere a shell does, but that is a statement about
what they need, not about what they should be handed, and naming a label is
the only say this file gets in which runner picks them up. If no runner
carries it they queue, which is the intended failure. host-docker keeps
dejima-runtime and now points at the example explaining what that runner
gives up.

.dejima/allowed-domains gains what CI actually reaches from inside a job:
Debian's mirror for apt, and download.docker.com plus GitHub's release hosts
for the docker CLI and buildx. The forge host is deliberately not in here - it
belongs to the runner's own base set, not to a file in a public repository.
Note the consequence documented in docs/ci.md: one file now grants in two
places, so these are reachable from a session as well.

static.bats pins that no job runs on a generic label, that the unprivileged
example keeps docker_host "-" with nothing mountable, that only the privileged
one grants the socket and the host netns and says what that costs, that the
firewall example filters INPUT and not only DOCKER-USER, that the refresh
reads main over the API and refuses to install an empty allowlist, and that
daemon.json carries no comment keys - dockerd refuses to start on a directive
it does not recognise, so an example with them is an example nobody can use.
ci/ joins SCANNED_PATHS, so the existing private-address and private-hostname
scanners cover it.
2026-08-05 09:53:42 -07:00
17 changed files with 1717 additions and 72 deletions
+6
View File
@@ -1 +1,7 @@
docs.gitea.com
deb.debian.org
download.docker.com
github.com
api.github.com
objects.githubusercontent.com
release-assets.githubusercontent.com
+68 -12
View File
@@ -29,7 +29,21 @@ on:
jobs:
sandbox:
runs-on: ubuntu-latest
# A label carried only by a runner configured for this - see ci/ and
# docs/ci.md. Not a generic one, even though these suites genuinely do run
# anywhere a shell does, because "runs anywhere" is a statement about what
# the job needs and not about what it should be handed.
#
# Whether a job container gets the Docker socket is a property of the
# runner's config.yaml (container.docker_host), not of anything this file
# can ask for or refuse. So a job that needs nothing still receives
# whatever its runner grants everything it runs, and naming a specific
# label is the only way this file gets a say in which runner that is.
#
# If no runner carries this label, these jobs queue. That is the intended
# failure: it is a better one than a green run on a host that grants
# everything.
runs-on: dejima-ci
strategy:
# One suite per job. fail-fast off because these cover unrelated areas -
# cancelling the other six on the first failure would hide whether a
@@ -72,18 +86,20 @@ jobs:
su node -c "TERM=xterm-256color bats test/${{ matrix.suite }}.bats"
host-docker:
# Not ubuntu-latest, unlike the job above. This one only works on a runner
# configured for it - host network namespace, the Docker socket, and the
# shared volume below - and none of that is anything the workflow can ask
# for. On a generic runner it fails the preflight check outright.
# A different label from the job above, and the only one in this file that
# asks for anything privileged. This job needs the host network namespace,
# the Docker socket and the shared volume below, none of which a workflow
# can request - they are granted by the runner or not at all, so on a
# runner without them it fails the preflight check outright.
#
# ubuntu-latest is a label every runner carries, so which one picked this
# up was a coin flip, and a re-run that happened to land on an unconfigured
# runner failed on a repo that had not changed. A label only the configured
# runner has makes that unrepresentable rather than merely diagnosable: no
# runner matching it means this job queues instead of running somewhere it
# cannot work. The sandbox job stays on ubuntu-latest, because it genuinely
# does run anywhere.
# A label carried by exactly one configured runner makes landing somewhere
# unusable unrepresentable rather than merely diagnosable: no runner
# matching it means this queues instead of running somewhere it cannot
# work. A generic label would make which runner picked it up a coin flip.
#
# The runner behind this label can be taken by anything it runs (see
# ci/act-runner-runtime.example.yaml), so it belongs on a host that exists
# for that and holds nothing else.
runs-on: dejima-runtime
container:
image: node:20-bookworm
@@ -291,3 +307,43 @@ jobs:
# own BATS_*_TMPDIR on paths that mean the same thing on both sides
# of the socket, which is all any of those mounts need.
TMPDIR: /tmp/dejima-ci
# The runner's own configuration, asserted from inside a job on it. Separate
# jobs rather than steps on the two above, because the point is that these
# are two different runners and either can be wrong on its own - one red
# check per tier says which.
#
# These fail on how the RUNNER is configured, not on anything in a pull
# request. A red check here means the machine needs attention; it does not
# mean the branch does.
runner-env-sandbox:
runs-on: dejima-ci
name: runner-env (sandbox)
container:
image: node:20-bookworm
steps:
- name: Install test dependencies
run: |
apt-get update
apt-get install -y --no-install-recommends bats python3 ca-certificates
- uses: actions/checkout@v4
- run: bats test/ci-runner.bats
env:
# Declared, not detected: landing on the wrong runner should fail
# rather than quietly select a different set of assertions.
DEJIMA_CI_TIER: sandbox
runner-env-host-docker:
runs-on: dejima-runtime
name: runner-env (host-docker)
container:
image: node:20-bookworm
steps:
- name: Install test dependencies
run: |
apt-get update
apt-get install -y --no-install-recommends bats python3 ca-certificates
- uses: actions/checkout@v4
- run: bats test/ci-runner.bats
env:
DEJIMA_CI_TIER: host-docker
+7 -3
View File
@@ -1090,9 +1090,12 @@ sandbox at a project whose CI you administer - and if you would rather not
take it on, `[core] forge = none` with `[core] workflow = local-only` declines
the push grant and the code execution with it.
**→ [docs/ci.md](docs/ci.md)** - why no approval gate is offered, hardening a
runner, and making CI's egress allowlist a version-controlled artifact read
from `main`.
**→ [docs/ci.md](docs/ci.md)** - the *why*: no approval gate is offered and
what that leaves, hardening a runner, and making CI's egress allowlist a
version-controlled artifact read from `main`.
**→ [ci/README.md](ci/README.md)** - the *how*: a step-by-step for standing up
a hardened `act_runner`, with the working configuration files beside it.
## Security notes
@@ -1191,6 +1194,7 @@ likely to collide with a real container you have running.
| `systemd/*.service` | User units for `dejima-ttyd` and `dejima-notify-watch`; ship disabled, installed by hand |
| `docs/providers.md` | The manifest contract for the forge/workflow/harness axes, key by key, with the shipped manifests as worked examples |
| `docs/ci.md` | What push access grants a session on your CI host, and how to bound it - runner hardening, and an egress allowlist read from `main` |
| `ci/` | Worked runner, resolver and firewall examples for the setup `docs/ci.md` describes; installed by nothing, and belongs on a CI machine rather than a dejima one |
| `docs/nginx-dejima.conf.example` | Reverse-proxy template for the web front door; deliberately not loadable as written |
`bin/dejima` is the entry point — it resolves the project path,
+455
View File
@@ -0,0 +1,455 @@
# Setting up a hardened CI runner
**This file is the how. [docs/ci.md](../docs/ci.md) is the why** — read that
if you want the reasoning; you do not need it to follow this.
What you get: `act_runner` on a host that executes agent-authored code on
every pull request and is granted as little as possible to execute it with —
no Docker socket in job containers, and deny-by-default egress whose allowlist
comes from `.dejima/allowed-domains` on `origin/main`.
Nothing here is installed by dejima's `install.sh`, and none of it runs on a
dejima host. It belongs on a CI machine, which should not be a machine that
runs dejima.
---
## 1. Before you start
Four things to have decided or to hand. Each is substituted into a file later,
and every example value below is a placeholder — RFC 5737 addresses and
RFC 2606 names — so **nothing here works unedited.**
| You need | Example | Used in |
| --- | --- | --- |
| Your forge's URL and API root | `https://gitea.example.com` | compose files, refresh script |
| A **private** /24 for the job network | `10.89.0.0/24`, gateway `10.89.0.1` | network, runner `--dns=`, resolver, firewall |
| An **upstream resolver** address | whatever this host already resolves with | resolver |
| A runner registration token | from the repo's Settings → Actions → Runners | `.env` |
**The upstream resolver is the one thing here that is not obvious.** It is the
resolver `dnsmasq` forwards *allowed* queries to — the thing that actually
knows the answers. It is **not** the dnsmasq you are about to configure (that
would loop), and **not** the bridge gateway.
It has to be able to resolve everything on your allowlist, **including your
forge** if that name is internal. In practice it is whatever this host already
uses:
```sh
resolvectl status | grep 'DNS Servers' # systemd-resolved
grep nameserver /etc/resolv.conf # otherwise
```
Take that address. `no-resolv` in the resolver config means dnsmasq will not
read `/etc/resolv.conf` itself, which is why you have to name it — that is the
whole reason the setting exists.
**On the /24:** use a genuinely private range, not another documentation one.
A documentation range would work — addresses are masqueraded on the way out
and TEST-NET is not routed — but `stop-dns-rebind` rejects answers in the
*private* ranges, so a reserved-but-not-private bridge subnet is one that
check silently does not cover. Pick something outside Docker's default pool
(`172.17`–`172.31`, `192.168.x`) so an unrelated `docker network create`
cannot land on top of it.
### Packages
```sh
apt-get install ipset iptables dnsmasq
```
`ipset` is the one to check rather than assume. The firewall script's first
command creates the set, so without it the script aborts immediately — and
what that leaves behind resembles a working setup closely enough to pass a
casual look: DNS still refuses unlisted names, because dnsmasq does that on
its own, while every job has unrestricted egress. The script now refuses by
name rather than failing at `ipset create`, and `test/ci-runner.bats` catches
it from inside a job.
**One prerequisite that is not on this host at all:** protect `main` on every
repository this runner will serve, and verify a job token cannot push to it.
The allowlist is read from `main`; if the untrusted party can write there,
none of the rest is worth anything. See
[docs/ci.md](../docs/ci.md#before-anything-else-protect-the-branch) — and note
that branch protection needs the sandbox to have its own forge identity before
it protects anything, which is
[docs/security.md](../docs/security.md#to-your-forge-the-sandbox-is-you).
---
## 2. Setup, in order
The order matters. Two steps fail *open* rather than loudly if taken out of
sequence, and both are called out where they apply.
Drop `.example` from every filename as you install it.
### Step 1 — create the job network
Before anything references it.
```sh
docker network create \
--opt com.docker.network.bridge.name=dejima-ci0 \
--subnet 10.89.0.0/24 \
dejima-ci
```
### Step 2 — install `daemon.json`
`docker-daemon.example.json` → `/etc/docker/daemon.json`
**Edit:** nothing required. Log bounds and `live-restore`.
It deliberately does **not** set `dns`. That would apply to every container
this daemon starts, and on a host that also runs the privileged tier those
include the sibling containers its jobs create over the Docker socket — see
[why the resolver is set per-runner](#why-the-resolver-is-set-per-runner-not-daemon-wide).
The resolver is pinned in the runner's config instead, at step 6.
⚠️ **Merge, do not overwrite.** If the host already has a `daemon.json`, add
these keys to it. A syntax error here stops Docker from starting at all, so
check the file before restarting something you rely on.
```sh
python3 -m json.tool /etc/docker/daemon.json >/dev/null && systemctl restart docker
```
This restart recreates Docker's own iptables chains, which is why the firewall
comes *after* it.
### Step 3 — install the resolver
`dnsmasq-ci.example.conf` → `/etc/dnsmasq.d/dejima-ci.conf`
**Edit:** `listen-address` → your bridge gateway. `rebind-domain-ok` → your
forge's hostname (only needed if it resolves to a private address).
```sh
systemctl restart dnsmasq
```
This file deliberately contains **no `server=` lines**. Every allowed name is
rendered into a second file by step 4; dnsmasq merges everything in
`dnsmasq.d`. A `server=` line added here by hand is a grant nobody merged, and
it would survive every refresh.
### Step 4 — render the allowlist
`refresh-allowlist.example.sh` → `/usr/local/sbin/dejima-ci-refresh-allowlist`
(`chmod 755`)
**Edit** the variables at the top: `FORGE_API`, `UPSTREAM` (the resolver from
§1), and create the three files it reads:
```sh
install -d -m 0750 /etc/dejima-ci
printf '%s\n' 'owner/repo' > /etc/dejima-ci/repos # one per line
printf '%s\n' 'gitea.example.com' 'deb.debian.org' > /etc/dejima-ci/base-domains
install -m 0600 /dev/null /etc/dejima-ci/forge-token # then write a read-only token into it
```
Run it once, then put it on a systemd timer. It writes
`/etc/dnsmasq.d/ci-allow.conf` — **never edit that file**; it is derived from
`origin/main` and overwritten on every run.
> If you would rather not maintain this script, `dejima-ci-allowlist` is the
> same thing as a maintained tool, with `check`/`show` subcommands and tests.
### Step 5 — apply the firewall
`iptables-ci.example.sh` → `/usr/local/sbin/dejima-ci-firewall` (`chmod 755`)
⚠️ **`dejima-ci-hardener` does not install this.** That tool manages the
allowlist and installs `dejima-ci-allowlist`; the firewall is this file, by
hand. If you installed the hardener and stopped there, you have a working
resolver and no packet filter — which is the state
`test/ci-runner.bats` reports as DNS checks passing while the egress checks
fail.
**Edit:** `BRIDGE` → the bridge name from step 1.
⚠️ **These rules are not persistent, and Docker itself removes them.** Every
daemon start recreates Docker's chains and `DOCKER-USER` comes back empty — so
a `systemctl restart docker` for any reason, days after you verified the
setup, silently leaves the runner with no egress policy.
Nothing about that looks different: jobs still run, and DNS still refuses
unlisted names, because that is dnsmasq's doing rather than the firewall's.
Only a job reaching an off-list address reveals it.
So install the unit rather than remembering to re-run the script:
`dejima-ci-firewall.example.service` → `/etc/systemd/system/dejima-ci-firewall.service`
```sh
systemctl daemon-reload
systemctl enable --now dejima-ci-firewall
# then prove it rebinds, rather than assuming:
systemctl restart docker && sudo iptables -S DOCKER-USER
```
`PartOf=docker.service` with `WantedBy=docker.service` is what makes it
automatic — `WantedBy` starts it whenever Docker starts, `PartOf` restarts it
whenever Docker restarts. A plain `After=` would only cover boot, which is the
case that was never the problem.
### Step 6 — start the ordinary runners
`docker-compose.example.yml` → `/srv/dejima-ci/docker-compose.yml`
`act-runner-ci.example.yaml` → `/srv/dejima-ci/act-runner-ci.yaml`
**Edit in the compose file:** `GITEA_INSTANCE_URL`, and `GITEA_RUNNER_NAME`
per service. Add or remove `ci-N` services to match how many runners you want;
each needs its own `container_name`, `GITEA_RUNNER_NAME` and `*-data` volume.
**Edit in the config file:** `runner.labels` (the image after `docker://` is
what jobs run in), `container.network` if you renamed the network, and
`--dns=` in `container.options` → your bridge gateway (`10.89.0.1`), which
must match the resolver's `listen-address`.
**Create `/srv/dejima-ci/.env`, mode 600:**
```sh
printf 'GITEA_RUNNER_REGISTRATION_TOKEN=%s\n' "$TOKEN" > /srv/dejima-ci/.env
chmod 600 /srv/dejima-ci/.env
```
Register at **repository** scope, not instance scope, so an unrelated
repository cannot schedule work onto a runner you hardened for this one.
```sh
cd /srv/dejima-ci && docker compose up -d
```
Once each runner has registered, the token in `.env` is dead weight — remove
it. Anyone holding one can register a runner and start receiving jobs, secrets
included.
### Step 7 — the privileged runner, on a different host
Only if you run dejima's own `runtime.bats`, or another job that genuinely
needs a Docker daemon.
`docker-compose-runtime.example.yml` → `/srv/dejima-runtime/docker-compose.yml`
`act-runner-runtime.example.yaml` → `/srv/dejima-runtime/act-runner-runtime.yaml`
A job on this runner can take the machine it runs on. Steps 1–5 apply to that
host too. See [why it is separate](#why-the-privileged-runner-is-separate).
---
## 3. Verify
Both failures worth testing for look exactly like success from inside a job,
so this is a permanent test rather than a one-off check: **`test/ci-runner.bats`**
in the dejima repository, run by the `runner-env (sandbox)` and
`runner-env (runtime)` jobs on every pull request.
It asserts on the *runner*, not on the branch — so a red check there means the
machine needs attention, not that the change does. What it covers:
- The sandbox tier's jobs are **not** handed the Docker socket, and the
runtime tier's job **is** — one red check per tier, so a mis-scheduled job
is caught in both directions rather than one.
- An allowlisted destination resolves and connects. This is the positive
control, and it is the point: without it a runner with no network at all
passes every negative check below. Set `DEJIMA_CI_ALLOWED_HOST` if
`deb.debian.org` is not on your allowlist.
- A name on no allowlist does not resolve.
- A job cannot reach a resolver of its own choosing — the check most easily
left out, and what decides whether the DNS allowlist means anything.
- A destination on no allowlist is unreachable, tested by IP literal so it
exercises the firewall rather than the resolver.
- IPv6 is not an unfiltered way around the rest.
The egress checks are scoped to the sandbox tier on purpose. The runtime
tier's jobs run in the **host** network namespace — that is the grant they
exist for — so the bridge rules never sit in front of them, and asserting an
allowlist there would pass while testing nothing.
Run it yourself against a runner at any time:
```sh
DEJIMA_CI_TIER=sandbox bats test/ci-runner.bats
```
With `DEJIMA_CI_TIER` unset every test skips, so it is inert outside CI.
## 4. File map
| File | Destination |
| --- | --- |
| `docker-compose.example.yml` | `/srv/dejima-ci/docker-compose.yml` |
| `act-runner-ci.example.yaml` | `/srv/dejima-ci/act-runner-ci.yaml` |
| `docker-compose-runtime.example.yml` | `/srv/dejima-runtime/docker-compose.yml` *(other host)* |
| `act-runner-runtime.example.yaml` | `/srv/dejima-runtime/act-runner-runtime.yaml` *(other host)* |
| `docker-daemon.example.json` | `/etc/docker/daemon.json` — merge |
| `dnsmasq-ci.example.conf` | `/etc/dnsmasq.d/dejima-ci.conf` |
| `refresh-allowlist.example.sh` | `/usr/local/sbin/dejima-ci-refresh-allowlist` |
| `iptables-ci.example.sh` | `/usr/local/sbin/dejima-ci-firewall` |
| `dejima-ci-firewall.example.service` | `/etc/systemd/system/dejima-ci-firewall.service` |
Files you create, not copy:
| File | Contents | Mode |
| --- | --- | --- |
| `/srv/dejima-ci/.env` | `GITEA_RUNNER_REGISTRATION_TOKEN=…` | `600` |
| `/etc/dejima-ci/forge-token` | a read-only forge token | `600` |
| `/etc/dejima-ci/repos` | `owner/repo`, one per line | `640` |
| `/etc/dejima-ci/base-domains` | names allowed regardless of any project | `640` |
Written for you, never edited:
| File | By | Because |
| --- | --- | --- |
| `/etc/dnsmasq.d/ci-allow.conf` | the refresh script | derived from `origin/main`; overwritten every run |
`/srv/dejima-ci` and `/srv/dejima-runtime` are just directories to keep a
compose project in — use whatever you normally would. The paths above are what
the files assume about *each other*, though: `daemon.json`'s `dns` must match
the resolver's `listen-address`, the compose files bind their config from
alongside themselves, and the refresh script writes into the same `dnsmasq.d`
the static config lives in. Change a path and check what referred to it.
---
# Reference
Everything below is background. You do not need it to complete the setup.
## The two settings named `docker.sock`
The compose bind and `container.docker_host` name the same path and do
opposite jobs. This is the easiest thing here to conflate:
| | Effect | Needed? |
| --- | --- | --- |
| The compose bind | The runner can reach the daemon | Yes — without it nothing runs |
| `docker_host: "-"` | Job containers *cannot* | Yes — without it every job is root on this host |
Setting the second does not remove the first, and removing the first does not
make jobs safe — it just stops the runner working.
Whether a job gets the socket is a property of **the runner**, not of the job:
a job asking for nothing unusual still receives whatever its runner grants
everything it runs.
## `CONFIG_FILE`, and why an edit can appear to do nothing
Without `CONFIG_FILE=/config.yaml` in the environment, `act_runner` does not
read the config file **at all** — it starts happily on built-in defaults,
which means the socket goes into every job container and nothing in any log
says so. If a config change appears to have no effect, check this first.
Mount the config read-only: a runner has no reason to rewrite the file that
decides what its jobs are granted.
## The `/data` volume
It holds `.runner`, written at registration: the runner's identity and its
credential for your forge (`runner.file: .runner`, resolved against the
container's working directory, which is `/data` in the official image).
- **Keep it** and `--force-recreate`, an image bump or a reboot brings back
*the same registered runner*. Without it the container comes up with no
identity — with a token still in the environment it silently registers a
*new* runner, leaving a dead entry in your forge's runner list; without one
it fails to start.
- **Delete it deliberately** to re-register: a runner moved to another
instance, or a label set you would rather establish fresh.
- **It is the only state.** Nothing else in the container is worth preserving,
which is what lets the rest of this treat a runner as disposable.
Labels come from `runner.labels` on every start rather than being frozen at
registration, so changing one is an edit and a restart.
## Compose networks and the job network are different worlds
Running several runners from one compose file makes them share a network — and
that is not the network worth worrying about.
Every service in that file is an `act_runner` *process*. The containers that
execute agent-authored code are created by the *daemon*, on the network named
by `container.network`, and compose knows nothing about them. Attaching a
service to a compose network does not put its jobs there.
So the shared network holds runner processes and nothing else, and those are
mutually trusted — each already holds the Docker socket, so separating them
protects nothing. The example names one `runners` network explicitly rather
than relying on the implicit `default`, so nothing joins it later by accident.
The compose file deliberately does **not** attach a runner to the job network
(a process holding the socket has no business sharing a network with untrusted
code), and does **not** put the runner behind the egress allowlist (those rules
filter the job bridge only — the runner is trusted, the jobs are not).
## Several runners, and whether they should share a job network
What *is* shared, with one compose file or five, is the job network: every
runner whose config names `dejima-ci` puts its jobs on that bridge. With N
runners at `capacity: 1` you get up to N jobs there at once, able to reach each
other and each other's service containers.
- **One shared job network** — the default, and why several runners can share
one config file. Every job on it has the same egress allowlist and none can
reach the host or the runners, so the added exposure is strictly job-to-job.
If the concern is a prompt-injected session reaching *outward*, this already
covers it.
- **One per runner** — worth it if concurrent jobs come from repositories that
should not interfere, or if service containers hold anything a neighbour
should not see. Costs a config file and a `docker network create` each, and
the firewall rules looped over each bridge.
The `ci-allow` ipset is shared either way — it holds destinations, not
per-bridge state, so one set and one dnsmasq serve every bridge. The wrinkle
is DNS: `daemon.json`'s `dns` is a single daemon-wide address that would sit
on one bridge and be unreachable from the others. Give dnsmasq a stable
address on a dummy interface, point `daemon.json` there, and let each
bridge's `INPUT` rule allow port 53 to it.
## Why the privileged runner is separate
It is handed the Docker daemon and the host's network namespace, because one
job cannot work without them and neither grant can be reduced without breaking
it. A job on it can take the machine it runs on — so that machine should exist
for nothing else. A separate compose file makes a separate host the default,
rather than something to disentangle later.
Where the two tiers do share a machine, be clear about what that is: **label
tiering, not host separation.** A job denied the socket still shares a kernel
with a runner that has it. A real reduction over granting everything to
everything; not isolation between the tiers.
It needs no bind for `/tmp/dejima-ci` — the job container gets that path
because the workflow asks for it and `container.valid_volumes` permits it, and
the host daemon resolves the source host-side.
## Why the resolver is set per-runner, not daemon-wide
`daemon.json` has a `dns` key, and pointing it at the allowlisting resolver is
the obvious way to make job containers use it. It is also wrong here, and the
failure is not obvious.
That setting applies to **every container the daemon starts**. On a host that
also runs the privileged tier, that includes containers created by its jobs
over the Docker socket — which are not job containers and were never meant to
be behind the CI resolver.
A dejima sandbox is exactly such a container, and it has a firewall of its
own. It allowlists precisely one host address: its own bridge gateway, taken
from its default route. A resolver on a *different* bridge is therefore
rejected by the sandbox's own rules, so nothing inside it resolves — and the
symptom is not "DNS is misconfigured" but the sandbox's startup self-check
reporting that a host it allows is unreachable.
`--dns` in `container.options` scopes the resolver to the containers that
runner starts, which is the set that should have it.
## The cache server
`act_runner` enables it by default, and it binds on the host *specifically* so
job containers can reach it — an inbound service exposed to untrusted code. The
firewall here blocks exactly that. Leave it off, or add a rule knowing what you
are exposing.
+115
View File
@@ -0,0 +1,115 @@
# act_runner configuration for a runner that executes agent-authored code and
# is granted nothing to execute it with. This is the ordinary tier: everything
# except the one job that genuinely needs a Docker daemon belongs here.
#
# Written against act_runner v0.6.1. Check yours with `act_runner --version`,
# and diff this against `act_runner generate-config` after an upgrade - the
# defaults are what most of this is overriding, and a new release can change
# one without anything in a job's output looking different.
#
# Not runnable as written: pick your own network name, subnet and label, and
# see ci/README.md for the order things have to be created in.
#
# See docs/ci.md for why each of these is here.
log:
level: info
runner:
file: .runner
# One job at a time. Above one, concurrent jobs share the network below -
# so they can reach each other's service containers - and the whole point of
# a fixed network is that its occupants are known.
capacity: 1
# A label nothing else carries, so a job naming it cannot land on a runner
# that was never hardened for it. If no runner carries it, jobs queue: that
# is the intended failure, and a better one than a green run on a host that
# grants everything.
#
# Deliberately NOT ubuntu-latest or any other generic label. A generic label
# is an invitation for any repository on the instance to schedule work here.
labels:
- "dejima-ci:docker://node:20-bookworm"
# Register at REPOSITORY scope, not instance scope (that is a choice made at
# `act_runner register` time, not here): an unrelated repository should not
# be able to schedule work onto a runner hardened for this one.
# Nothing here. Extra environment variables reach agent-authored code.
envs: {}
env_file: ""
timeout: 1h
insecure: false
cache:
# Off. act_runner's cache server binds on the runner host specifically so
# that job containers can reach it, which makes it an inbound service
# exposed to untrusted code. Turn it on only when something actually uses
# actions/cache, and know what you are exposing when you do.
enabled: false
container:
# THE setting on this page. act_runner mounts the Docker daemon's socket
# into every job container unless this is exactly "-", and its own generated
# config says so:
#
# If it's "-", act_runner will find an available docker host
# automatically, but the docker host won't be mounted to the job
# containers and service containers.
#
# An unset value is the default, and the default mounts. Access to that
# socket is root-equivalent on this host - `docker run -v /:/host` ends the
# argument - and nothing in a job's log mentions it being there.
#
# Note this is a property of the RUNNER, not of the job: a job asking for
# nothing unusual still receives whatever the runner grants everything it
# runs.
docker_host: "-"
# A network created ahead of time, with a named bridge, so that the
# interface name and gateway are stable enough to write firewall rules
# against. Left empty, act_runner creates one per job and there is nothing
# fixed to match on. See ci/iptables-ci.example.sh.
#
# NOT `host`: that would put jobs in this machine's network namespace, where
# no bridge-based filtering applies to them at all.
network: dejima-ci
privileged: false
# --init reaps zombies. --security-opt=no-new-privileges costs nothing and
# does not depend on getting a capability list right.
#
# On capabilities: --cap-drop=ALL is the reflex and it breaks apt-get, which
# needs CHOWN, DAC_OVERRIDE, FOWNER, SETUID and SETGID to unpack a package.
# Drop what your jobs demonstrably do not need rather than dropping
# everything and rediscovering that list one failure at a time.
# --dns pins the allowlisting resolver to the containers THIS runner
# starts, rather than setting it daemon-wide in daemon.json. That
# distinction is load-bearing on a host that also runs the privileged tier:
# its jobs create sibling containers over the Docker socket, and a
# daemon-wide resolver reaches those too. A dejima sandbox is one of them,
# and it allowlists exactly one host address - its own bridge gateway,
# taken from its default route - so a resolver on a different bridge is
# rejected by its own firewall and nothing inside it resolves at all.
#
# Must match listen-address in the dnsmasq config.
options: "--dns=203.0.113.1 --init --security-opt=no-new-privileges --cap-drop=NET_ADMIN --cap-drop=NET_RAW --cap-drop=SYS_ADMIN --cap-drop=SYS_MODULE --cap-drop=SYS_PTRACE --pids-limit=2048"
# Empty is the default and it is the right value: no host path can be
# mounted into a job container. Stated explicitly so that adding one has to
# be a deliberate edit rather than a default nobody looked at.
valid_volumes: []
force_pull: true
force_rebuild: false
require_docker: false
host:
workdir_parent:
metrics:
enabled: false
+94
View File
@@ -0,0 +1,94 @@
# act_runner configuration for the privileged tier: the one runner that hands
# job containers a Docker daemon and this machine's network namespace, because
# a job exists that cannot work without them.
#
# Read this first: a job on this runner can take this host. The socket below
# is root-equivalent - `docker run -v /:/host` - and the host network
# namespace puts the job on your network with no filtering in front of it.
# Neither can be reduced without breaking the job they exist for. The design
# does not pretend otherwise; it assumes owning this host is worth nothing.
#
# So this runner belongs on a machine that:
# - runs nothing else, and holds nothing else,
# - has no route to a LAN or management network,
# - holds no credentials beyond its own registration token,
# - is rebuilt from a snapshot on a schedule.
#
# Sharing a machine with the ordinary runner (ci/act-runner-ci.example.yaml)
# gives you label tiering, not host separation: a job denied the socket still
# shares a kernel and a network with a runner that has it. That is a real
# reduction and worth doing if one machine is what you have - just do not read
# it as isolation.
#
# Written against act_runner v0.6.1. Not runnable as written.
# See docs/ci.md.
log:
level: info
runner:
file: .runner
capacity: 1
# Only this label. A generic one (ubuntu-latest and friends) would let any
# job on the instance land here and collect both grants below without ever
# asking for them - and a job that never asked is exactly the one nobody
# reviews for whether it should have them.
labels:
- "dejima-runtime:docker://node:20-bookworm"
envs: {}
env_file: ""
timeout: 2h
insecure: false
cache:
enabled: false
container:
# Left unset ON PURPOSE, which is the opposite of the ordinary runner. Unset
# means act_runner mounts /var/run/docker.sock into job containers, and the
# job here builds an image and boots real containers over it.
#
# This is the single line that makes this host sacrificial. If you ever find
# it unset on a runner you did not intend to be privileged, that runner is
# privileged anyway.
docker_host: ""
# This machine's own network namespace. Needed by exactly one assertion:
# that a host service bound to loopback is unreachable from inside a
# container while one bound to the bridge gateway is reachable. The positive
# control has the suite bind a listener on the test network's bridge gateway
# address, which lives on a host interface and cannot be bound from any
# other namespace - so the control would fail for a reason unrelated to what
# the test asserts.
#
# Gitea's act fork ignores `--network` in a workflow's container options, so
# this cannot be requested by a repository. It is granted here or not at all.
network: host
privileged: false
options: "--init"
# One path, and the job that needs it does not work without it: fixtures are
# built here and bind-mounted into containers the HOST daemon creates, so
# every -v source resolves against the host's filesystem rather than the job
# container's. A path that exists only inside the job does not resolve - and
# Docker's answer to a missing bind source is to create an empty directory
# and mount that, which fails tests obscurely and passes the ones an empty
# read-only directory happens to satisfy.
#
# A volume a workflow asks for and this list does not name is dropped
# silently, so this has to match what the workflow requests.
valid_volumes:
- /tmp/dejima-ci
force_pull: true
force_rebuild: false
require_docker: true
host:
workdir_parent:
metrics:
enabled: false
+42
View File
@@ -0,0 +1,42 @@
# Reapplies the job network's egress rules whenever Docker starts.
#
# This exists because the rules are not persistent and Docker itself removes
# them. Every daemon start recreates Docker's own iptables chains, and
# DOCKER-USER comes back empty - so a `systemctl restart docker` for any
# reason at all, days after the setup was verified, silently leaves the runner
# with no egress policy. Nothing about it looks different: jobs still run,
# DNS still refuses unlisted names because that is dnsmasq's doing rather than
# the firewall's, and only a job actually reaching an off-list address reveals
# it.
#
# PartOf + WantedBy on docker.service is what makes that automatic:
# WantedBy starts this whenever docker starts, and PartOf restarts it whenever
# docker restarts. A plain After= would only cover boot, which is the case
# that was never the problem.
#
# Install as /etc/systemd/system/dejima-ci-firewall.service, then:
# systemctl daemon-reload
# systemctl enable --now dejima-ci-firewall
#
# Verify it actually rebinds, rather than assuming:
# systemctl restart docker && sudo iptables -S DOCKER-USER
[Unit]
Description=dejima CI job-network egress rules
Documentation=https://gitbox.example.com/branderson/dejima/src/branch/main/ci/README.md
After=docker.service
PartOf=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/dejima-ci-firewall
# Failing to apply the rules must be loud. A runner that came up without them
# is a runner with unrestricted egress, which is the state this whole setup
# exists to prevent - so surface it rather than logging and carrying on.
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=docker.service
+56
View File
@@ -0,0 +1,56 @@
# The static half of the job network's resolver. Install as
# /etc/dnsmasq.d/dejima-ci.conf - see "Where each file goes" in ci/README.md.
#
# There are no `server=` lines here on purpose. Every allowed name is rendered
# into a SECOND file, /etc/dnsmasq.d/ci-allow.conf, by
# refresh-allowlist.example.sh, from .dejima/allowed-domains on origin/main.
# dnsmasq reads both and merges them. Keeping the generated names out of this
# file is what stops a hand-edited copy drifting from the one that is actually
# derived from a branch you control - a `server=` line added here is a grant
# nobody merged.
#
# Same construction as the sandbox's own docker/dns-proxy.sh, one layer out:
# allowlist the names, and let the answers populate the set of allowed
# addresses. A name with no `server=` line anywhere gets REFUSED - not
# resolved and then blocked at the firewall, refused - which is what closes
# off encoding data into query names to a domain you never allowed.
#
# Not runnable as written: the addresses below are RFC 5737 documentation
# ranges. Substitute your own, and read the subnet note in ci/README.md before
# picking one - "private" is load-bearing here, not tidiness.
# Bind the job bridge's gateway only. Never 0.0.0.0: this resolver answers for
# untrusted code and has no business being reachable from anywhere else.
# bind-interfaces (rather than the default bind-dynamic) also keeps it off
# whatever else is listening on this host - systemd-resolved on 127.0.0.53,
# say - instead of contending for :53.
listen-address=203.0.113.1
bind-interfaces
no-hosts
# No system resolvers. Without this, /etc/resolv.conf silently becomes an
# upstream for every name and the allowlist stops meaning anything.
#
# Note this applies to the whole dnsmasq instance, so this instance serves the
# job network and nothing else. If the host needs a resolver for its own sake,
# that is a different instance or a different daemon.
no-resolv
# Every answer served is added to the firewall's set, so a destination that
# rotates its edge addresses stays reachable without a resolver cron chasing
# it. Only allowlisted names get an upstream to answer them in the first
# place, which is what makes a wildcard safe here.
ipset=/#/ci-allow
# Answers claiming a private address are rejected: a name you do not control
# resolving to one is how a job reaches something on your network that was
# never on any list.
#
# This is keyed on the private ranges, which is one of the reasons the job
# bridge itself should use one - see ci/README.md.
stop-dns-rebind
# A self-hosted forge legitimately resolves to a private address, which
# stop-dns-rebind would otherwise reject. Exempt it by name - not by turning
# the protection off. One line per internal name you allow.
rebind-domain-ok=/gitea.example.com/
+74
View File
@@ -0,0 +1,74 @@
# The privileged runner, on its own, in its own file.
#
# One service, deliberately. This runner is handed the Docker daemon and the
# host's network namespace, because one job cannot work without them, and
# neither grant can be reduced without breaking that job. A job on this runner
# can take the machine it runs on.
#
# So the machine it runs on should be one that exists for that and holds
# nothing else: no other service, no route to a LAN or management network, no
# credentials beyond its own registration token, rebuilt from a snapshot on a
# schedule. That is why this is a separate file rather than another service
# alongside the ordinary runners - a separate host is the intended
# arrangement, and a compose file that assumes it is harder to drift away from
# than one that has to be split later.
#
# Where the two tiers do share a machine, be clear about what that is: label
# tiering, not host separation. A job denied the socket still shares a kernel
# with a runner that has it. That is a real reduction over granting everything
# to everything, and it is not isolation between the tiers.
#
# Pair with ci/act-runner-runtime.example.yaml. Not runnable as written:
# substitute your own instance URL and image tag.
name: dejima-runtime
services:
runtime:
# Pin the tag. This process executes untrusted code on every pull request,
# and :latest means the thing doing that can change without you deciding
# it should.
image: gitea/act_runner:0.6.1
restart: unless-stopped
container_name: dejima-runtime
networks: [runners]
environment:
# Without this the config file is not read AT ALL - the runner starts on
# built-in defaults instead, and on this tier the defaults happen to
# resemble the intended configuration closely enough that the mistake is
# invisible. Check it first when an edit appears to do nothing.
CONFIG_FILE: /config.yaml
GITEA_INSTANCE_URL: https://gitea.example.com
GITEA_RUNNER_NAME: dejima-runtime
# Needed only until registration completes; the identity then lives in
# the volume below. Comes from a .env file beside this one (chmod 600)
# so it is not in the file you commit, and is worth removing once
# registration succeeds - anyone holding a registration token can
# register a runner and start receiving jobs, secrets included.
GITEA_RUNNER_REGISTRATION_TOKEN: ${GITEA_RUNNER_REGISTRATION_TOKEN:-}
volumes:
# The runner's own access to the daemon, so it can create job
# containers. On this tier the config file ALSO passes the socket into
# those containers, which is the grant that makes this host sacrificial;
# on the ordinary tier it does not. Same bind, different consequence,
# decided entirely by container.docker_host in the config.
- /var/run/docker.sock:/var/run/docker.sock
- ./act-runner-runtime.yaml:/config.yaml:ro
# Its own registration identity. Never shared with another runner.
- runtime-data:/data
# No bind for /tmp/dejima-ci. The job container gets that path because
# the workflow asks for it and container.valid_volumes permits it, and
# the host daemon resolves the source host-side. This container never
# sees it.
networks:
# This runner's own network, for reaching your Gitea instance. The job
# network is deliberately absent here as well: jobs on this tier are in the
# host's network namespace, which is granted by container.network in the
# config file and is not something compose can express or restrain.
runners:
volumes:
runtime-data:
+109
View File
@@ -0,0 +1,109 @@
# The ordinary runners: several on one host, sharing one config file and
# therefore one label, so a job naming that label lands on whichever is free.
#
# The privileged runner is NOT here. It has its own compose file, because it
# belongs on its own host - see ci/docker-compose-runtime.example.yml.
#
# The thing to understand before reading the rest: compose's networks and the
# job network are different worlds. Every service below is an act_runner
# process. The containers that actually execute agent-authored code are
# created by the Docker daemon, on the network named by `container.network` in
# each runner's config.yaml, and compose knows nothing about them. Attaching a
# service here to a network does not put its jobs there, and vice versa. See
# ci/README.md for what that means for isolating jobs from each other.
#
# Not runnable as written: substitute your own instance URL, image tag and
# subnet, and create the job network first (ci/README.md, step 2).
name: dejima-runners
# Shared by every ordinary runner. Volumes are per-service rather than here,
# because each runner needs its own /data - and YAML merge keys merge mappings
# but not lists, so a volumes: in a service replaces this one rather than
# adding to it. Repeating them is the honest form.
x-ci-runner: &ci-runner
# Pin the tag. These processes execute untrusted code on every pull request,
# and :latest means the thing doing that can change without you deciding it
# should.
image: gitea/act_runner:0.6.1
restart: unless-stopped
networks: [runners]
x-ci-env: &ci-env
# Without this the config file is not read AT ALL - the runner starts
# happily on built-in defaults, which means the Docker socket goes into
# every job container and nothing anywhere says so. If you change one thing
# in a config file and see no effect, check this first.
CONFIG_FILE: /config.yaml
GITEA_INSTANCE_URL: https://gitea.example.com
# Needed only until each runner has registered; the identity then lives in
# that runner's own /data volume. Comes from a .env file beside this one
# (chmod 600) so it is not in the file you commit, and is worth deleting
# once registration succeeds - anyone holding a registration token can
# register a runner and start receiving jobs, secrets included.
GITEA_RUNNER_REGISTRATION_TOKEN: ${GITEA_RUNNER_REGISTRATION_TOKEN:-}
services:
# --- ordinary runners -----------------------------------------------------
# All three share one config file, and therefore one label and one job
# network. That is what makes them a pool: a job naming dejima-ci lands on
# whichever is free. Only the name and the data volume differ.
ci-1:
<<: *ci-runner
container_name: dejima-ci-1
environment:
<<: *ci-env
GITEA_RUNNER_NAME: dejima-ci-1
volumes:
# The runner's OWN access to the daemon, so it can create job
# containers. NOT the same setting as container.docker_host, which
# decides whether job containers get the socket - see ci/README.md.
- /var/run/docker.sock:/var/run/docker.sock
# Read-only: a runner has no reason to rewrite the file deciding what
# its jobs are granted.
- ./act-runner-ci.yaml:/config.yaml:ro
# Its registration identity. One per runner, never shared - two runners
# cannot share a .runner.
- ci-1-data:/data
ci-2:
<<: *ci-runner
container_name: dejima-ci-2
environment:
<<: *ci-env
GITEA_RUNNER_NAME: dejima-ci-2
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./act-runner-ci.yaml:/config.yaml:ro
- ci-2-data:/data
ci-3:
<<: *ci-runner
container_name: dejima-ci-3
environment:
<<: *ci-env
GITEA_RUNNER_NAME: dejima-ci-3
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./act-runner-ci.yaml:/config.yaml:ro
- ci-3-data:/data
networks:
# The runner containers' own network, and nothing else's. They need it only
# to reach your Gitea instance; they never talk to each other. Sharing one
# is fine because these are all equally trusted processes - every one of
# them holds the Docker socket already, so putting them on separate networks
# would protect nothing.
#
# The job network is deliberately absent. It is created by hand with a fixed
# bridge name so the firewall can match on it (ci/README.md, step 2), and
# act_runner attaches job containers to it by name. Declaring it here would
# invite attaching a service to it, which is exactly what must not happen: a
# process holding the Docker socket has no business sharing a network with
# containers running untrusted code.
runners:
volumes:
ci-1-data:
ci-2-data:
ci-3-data:
+8
View File
@@ -0,0 +1,8 @@
{
"log-driver": "json-file",
"log-opts": {
"max-size": "16m",
"max-file": "3"
},
"live-restore": true
}
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/env bash
# Deny-by-default egress for the job network.
#
# The one thing to get right here is that a job's traffic passes through two
# different chains depending on where it is going, and filtering only the
# obvious one leaves the DNS allowlist decorative:
#
# job -> anywhere off the bridge FORWARD, so DOCKER-USER
# job -> the resolver on the INPUT, because the gateway address
# bridge gateway belongs to this host
#
# A job that can reach any resolver it likes can resolve anything it likes, so
# without the INPUT rules below it simply asks a public resolver and every
# name is allowed again.
#
# Two more positions are NOT covered by this script, and should not be
# confused with it:
#
# - Pulling the job's own image is the Docker daemon's traffic, in this
# host's namespace: that is the host's OUTPUT chain.
# - Where a job has a daemon of its own, `docker build` steps run on that
# daemon's bridge under its own policy.
#
# Not runnable as written: RFC 5737 addresses, and a bridge name that has to
# match the network you created. Run after the network exists and before the
# runner starts. See ci/README.md for the order.
set -euo pipefail
# Checked by name, because the failure without it is not obvious. The first
# real command below is `ipset create`, so on a host without ipset this script
# aborts at line one under `set -e` - and the state that leaves behind looks
# almost like a working one: DNS still refuses unlisted names, because that is
# dnsmasq's doing and not the firewall's, while a job can reach any address it
# likes. "command not found" scrolling past during setup is easy to miss; a
# runner with no egress policy is not something to discover later.
for tool in ipset iptables ip6tables; do
command -v "$tool" >/dev/null 2>&1 || {
echo "$tool is not installed - these rules cannot be applied." >&2
echo "On Debian/Ubuntu: apt-get install ipset iptables" >&2
echo "" >&2
echo "Without this, DNS allowlisting still appears to work (dnsmasq refuses" >&2
echo "unlisted names on its own) while jobs have unrestricted egress." >&2
exit 1
}
done
BRIDGE=dejima-ci0 # com.docker.network.bridge.name of the job network
SET=ci-allow # populated by dnsmasq's ipset= line
# hash:net, and created before any rule references it: an iptables rule naming
# a set that does not exist fails to load, and a half-loaded ruleset is an
# open one.
ipset create "$SET" hash:net -exist
# --- job egress: DOCKER-USER is the chain Docker jumps to first from FORWARD,
# --- and the only one whose contents Docker will not rewrite underneath you.
iptables -F DOCKER-USER
# Return traffic for connections that were allowed on their way out.
iptables -A DOCKER-USER -i "$BRIDGE" -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
# The allowlist proper. Port-restricted: a name being allowed is not a reason
# for every port on it to be reachable.
iptables -A DOCKER-USER -i "$BRIDGE" -p tcp -m set --match-set "$SET" dst \
-m multiport --dports 80,443 -j ACCEPT
# Uncomment for git-over-ssh to a forge. Prefer https where the forge offers
# it, so that this stays closed.
# iptables -A DOCKER-USER -i "$BRIDGE" -p tcp --dport 22 -m set --match-set "$SET" dst -j ACCEPT
# REJECT rather than DROP: a job that cannot reach something should fail
# immediately and say so, not hang until a timeout and read as a slow network.
iptables -A DOCKER-USER -i "$BRIDGE" -j REJECT --reject-with icmp-port-unreachable
# --- the resolver: addressed to this host, so it never reaches DOCKER-USER.
iptables -A INPUT -i "$BRIDGE" -p udp --dport 53 -j ACCEPT
iptables -A INPUT -i "$BRIDGE" -p tcp --dport 53 -j ACCEPT
# Everything else aimed at this host from the job network - the SSH daemon,
# act_runner's cache server if you left it on, anything else listening - is
# not part of running a job.
iptables -A INPUT -i "$BRIDGE" -j REJECT --reject-with icmp-port-unreachable
# --- IPv6: blocked outright, because none of the above has an IPv6 twin.
# An address family with no allowlist is not a smaller hole than the one this
# script just closed; on a host with IPv6 egress it is the same hole.
ip6tables -F DOCKER-USER
ip6tables -A DOCKER-USER -i "$BRIDGE" -j REJECT --reject-with icmp6-port-unreachable
ip6tables -A INPUT -i "$BRIDGE" -j REJECT --reject-with icmp6-port-unreachable
# These are not persistent. Reapply on boot, and after anything that reloads
# the firewall - Docker recreates its own chains on daemon restart, and a
# DOCKER-USER that came back empty is a runner with no egress policy at all.
+111
View File
@@ -0,0 +1,111 @@
#!/usr/bin/env bash
# Render the job network's DNS allowlist from each served repository's
# .dejima/allowed-domains, as it exists on the branch the operator controls.
#
# The whole value of this is in where it reads from. `main` got its contents
# through a pull request someone merged; a pull request branch got its
# contents from whatever pushed it. A branch that can write its own allowlist
# does not have one - so this reads over the forge API, by ref, and never from
# a checkout on disk.
#
# The consequence, which is the point: widening what CI can reach is a diff on
# main. Reviewed, attributable, revertible, and made in the same file and by
# the same act that widens what a sandbox session may reach.
#
# Not runnable as written. Example addresses are RFC 5737 / RFC 2606; supply
# your own, and a token with read access and nothing else.
#
# Run from a systemd timer. See ci/README.md.
set -euo pipefail
FORGE_API=${FORGE_API:-https://gitea.example.com/api/v1}
FORGE_TOKEN_FILE=${FORGE_TOKEN_FILE:-/etc/dejima-ci/forge-token}
REPOS_FILE=${REPOS_FILE:-/etc/dejima-ci/repos} # one owner/repo per line
BASE_FILE=${BASE_FILE:-/etc/dejima-ci/base-domains} # one name per line
UPSTREAM=${UPSTREAM:-198.51.100.53}
OUT=${OUT:-/etc/dnsmasq.d/ci-allow.conf}
REF=${REF:-main}
# Refuse to run rather than render a wrong file. Every failure mode from here
# on leaves the previous allowlist in place, which is the direction this has
# to fail in: a builder whose broken version is an empty allowlist looks
# exactly like a working one until the moment it matters.
[ -r "$FORGE_TOKEN_FILE" ] || { echo "no forge token at $FORGE_TOKEN_FILE" >&2; exit 1; }
[ -r "$REPOS_FILE" ] || { echo "no repository list at $REPOS_FILE" >&2; exit 1; }
token=$(< "$FORGE_TOKEN_FILE")
# Only what a DNS name can be. This text arrives from a file in a repository,
# which is a place agent-authored code can write - so nothing reaches a
# dnsmasq directive without matching this first. An unexpected line is a hard
# failure, not a skipped line: a silently-dropped entry reads as "that domain
# was never declared" and sends someone looking in the wrong place.
valid_domain() {
[[ "$1" =~ ^[A-Za-z0-9]([A-Za-z0-9-]{0,61}[A-Za-z0-9])?(\.[A-Za-z0-9]([A-Za-z0-9-]{0,61}[A-Za-z0-9])?)+$ ]]
}
collect() {
# The base set first: what the runner itself needs to do its job at all,
# independent of any project. Keep it narrower than the sandbox's built-in
# domains - a name judged reasonable for a firewalled container is a
# different proposition on a host on your network.
[ -r "$BASE_FILE" ] && cat "$BASE_FILE"
while read -r repo; do
[ -n "$repo" ] || continue
# Read the status rather than curl's exit code. `--fail` collapses
# every HTTP error into exit 22, and the difference matters here: a
# repository with no declaration (404) is ordinary and skipped, while
# a 500 or a 401 means this run does not know what the allowlist is
# and must not render one.
#
# Without a status check at all, curl prints the error page and exits
# 0, and an allowlist rendered from a 404 body is an allowlist of
# whatever that page happened to contain.
body=$(mktemp)
http=$(curl -sS --max-time 20 -o "$body" -w '%{http_code}' \
-H "Authorization: token $token" \
"$FORGE_API/repos/$repo/raw/.dejima/allowed-domains?ref=$REF") || {
rm -f "$body"; echo "fetch failed for $repo" >&2; return 1; }
case "$http" in
200) cat "$body" ;;
404) ;;
*) rm -f "$body"; echo "unexpected HTTP $http for $repo" >&2; return 1 ;;
esac
rm -f "$body"
done < "$REPOS_FILE"
}
domains=$(collect | tr -d '\r' | sed 's/[[:space:]]*$//' | grep -v '^$' | sort -u) || {
echo "collection failed; leaving the existing allowlist in place" >&2
exit 1
}
[ -n "$domains" ] || { echo "collected an empty allowlist; refusing to install it" >&2; exit 1; }
tmp=$(mktemp)
trap 'rm -f "$tmp"' EXIT
while IFS= read -r d; do
valid_domain "$d" || { echo "refusing a malformed entry: $d" >&2; exit 1; }
printf 'server=/%s/%s\n' "$d" "$UPSTREAM" >> "$tmp"
done <<< "$domains"
# Only replace a working config with one that parses. dnsmasq exits on a bad
# config file, and a resolver that is down is a job network with no DNS at
# all - which fails every run rather than the one that caused it.
dnsmasq --test --conf-file="$tmp" >/dev/null 2>&1 || {
echo "rendered config failed dnsmasq --test; leaving the existing one in place" >&2
exit 1
}
# Compare before writing: a reload on every timer tick drops the cache for no
# reason, and makes the log useless for spotting when the allowlist actually
# changed.
if [ -f "$OUT" ] && cmp -s "$tmp" "$OUT"; then
exit 0
fi
install -m 0644 "$tmp" "$OUT"
systemctl reload dnsmasq
echo "allowlist updated: $(wc -l < "$OUT") names"
+33 -51
View File
@@ -6,6 +6,10 @@
CI runs.** That is the whole of it, and everything below is either why it
cannot be gated or what to do about it instead.
> **This document is the why.** If you just want to stand a hardened runner
> up, [`ci/README.md`](../ci/README.md) is the step-by-step and does not
> assume you have read this.
The chain needs no operator action at any step. A session writes code; the
push guard permits a feature branch, which is what it exists to allow; the
session opens a pull request, which its briefing tells it to do without asking
@@ -141,6 +145,9 @@ rather than leaving at their defaults are `docker_host: "-"`, an empty
`valid_volumes` (the default, but worth stating so an edit has to be
deliberate), `privileged: false`, a fixed `network` (below), `capacity: 1`,
`cache.enabled: false`, and a label nothing else carries.
[`ci/act-runner-ci.example.yaml`](../ci/act-runner-ci.example.yaml) is that
config with the reasoning inline; its privileged counterpart is
[`ci/act-runner-runtime.example.yaml`](../ci/act-runner-runtime.example.yaml).
**Prefer a new runner over reconfiguring an existing one.** A general-purpose
runner often exists precisely to build images, and taking its socket away
@@ -185,76 +192,47 @@ image pulls and action fetches happen from the host rather than from the job.
For a suite that installs a few packages and checks out a repository, the job
needs the forge and a package mirror, and nothing else.
### A fixed network for jobs
### A fixed network for jobs, a resolver, and both firewall chains
Point `container.network` at a network you created, with a named bridge, so
that the interface name, subnet and gateway are stable enough to write rules
against. Left empty, `act_runner` creates a network per job and there is no
stable interface to match on.
```sh
docker network create \
--opt com.docker.network.bridge.name=dejima-ci0 \
--subnet 203.0.113.0/24 \
dejima-ci
```
(That subnet is a documentation range, per RFC 5737. Substitute a private
range that does not collide with anything you route to.)
stable interface to match on. Give it a **private** subnet: `stop-dns-rebind`
below rejects answers in the private ranges, so a reserved-but-not-private
bridge subnet is one that check silently does not cover.
With `runner.capacity: 1` one job occupies that network at a time. Above one,
concurrent jobs share a broadcast domain and can reach each other's service
containers, which is worth knowing before raising it.
### DNS and the firewall
The shape is the same one dejima uses inside its own container
(`docker/dns-proxy.sh` and `docker/init-firewall.sh`), applied one layer out:
allowlist the *names* and let the answers populate the set of allowed
*addresses*.
The DNS and firewall shape is the same one dejima uses inside its own
container (`docker/dns-proxy.sh` and `docker/init-firewall.sh`), applied one
layer out: allowlist the *names* and let the answers populate the set of
allowed *addresses*.
`dnsmasq` binds the bridge gateway. With `no-resolv` and one `server=` line
per allowed name, every other name gets `REFUSED` — not resolved-then-blocked,
refused, which closes off encoding data into query names to a domain you never
allowed. `ipset=/#/ci-allow` adds every answer it serves to the firewall's
set, so a destination that rotates its edge addresses stays reachable without
a resolver cron chasing it.
a resolver cron chasing it. `rebind-domain-ok` exempts a self-hosted forge,
which legitimately resolves to a private address.
```
no-resolv
server=/deb.debian.org/198.51.100.53
ipset=/#/ci-allow
rebind-domain-ok=/gitea.example.com/
```
`rebind-domain-ok` matters for a self-hosted forge: `stop-dns-rebind` rejects
private answers, and an internal forge legitimately has one.
Point the daemon at that resolver so containers inherit it:
```json
{ "dns": ["203.0.113.1"] }
```
Then filter, remembering both chains:
```sh
# Job egress: everything forwarded off the bridge passes here first.
iptables -I DOCKER-USER -i dejima-ci0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A DOCKER-USER -i dejima-ci0 -m set --match-set ci-allow dst -p tcp -m multiport --dports 80,443 -j ACCEPT
iptables -A DOCKER-USER -i dejima-ci0 -j REJECT
# The resolver lives on the host, so it is INPUT, not FORWARD.
iptables -A INPUT -i dejima-ci0 -p udp --dport 53 -j ACCEPT
iptables -A INPUT -i dejima-ci0 -p tcp --dport 53 -j ACCEPT
iptables -A INPUT -i dejima-ci0 -j REJECT
```
Block IPv6 outright unless you have built the same allowlist for it. An
The runner pins its job containers at that resolver - per-runner rather than
daemon-wide, because a daemon-wide setting also reaches containers a
privileged job creates over the Docker socket, which have firewalls of their
own and no route to it. Then two chains filter: `DOCKER-USER` for everything the job forwards off the bridge, and
`INPUT` for the resolver itself, because the gateway address belongs to the
host. Block IPv6 outright unless you have built the same allowlist for it — an
address family with no allowlist is not a smaller hole than the one you just
closed.
### The allowlist comes from `main`, not from the checkout
**The working configuration for all of this is in [`ci/`](../ci/), and
[`ci/README.md`](../ci/README.md) is the step-by-step.** It is deliberately not
reproduced here: two copies of a firewall rule drift, and the one in `ci/` is
the one with the ordering constraints and the substitution notes attached.
## The allowlist comes from `main`, not from the checkout
A dejima project already carries a version-controlled statement of where it is
allowed to reach: `.dejima/allowed-domains`. Use it — read from
@@ -280,6 +258,10 @@ a fetch that errors should leave the previous allowlist in place, never fall
back to allowing everything — an allowlist builder is exactly the kind of
script whose failure mode is easy to get backwards, because the broken version
looks like a working one right up until it matters.
[`ci/refresh-allowlist.example.sh`](../ci/refresh-allowlist.example.sh) is a
worked version, including the distinction that costs you here: a repository
with no declaration is a 404 and ordinary, while a 401 or a 500 means this run
does not know what the allowlist is and must not render one.
Two things to be deliberate about:
+172
View File
@@ -0,0 +1,172 @@
#!/usr/bin/env bats
#
# Asserts on the RUNNER, not on this repository. Every other suite here tests
# dejima's code; this one tests the machine executing it, because the controls
# in docs/ci.md live in a runner's config.yaml and a host's firewall - places
# no amount of reviewing this repository can check.
#
# It exists because both failures that matter look exactly like success from
# inside a job. Nothing in a job's log mentions whether the Docker socket was
# mounted into it, and a job that can reach an off-list resolver still passes
# every check that only inspects DOCKER-USER. Neither shows up as a red run;
# they show up as nothing at all.
#
# DEJIMA_CI_TIER says which runner this is supposed to be, set by the workflow
# rather than sniffed, so landing on the wrong runner is a failure rather than
# a different set of assertions quietly being chosen. Unset means this is not
# a CI runner and there is nothing here to assert.
#
# Its values are the names of the jobs whose environment they verify -
# `sandbox` and `host-docker` - rather than a separate vocabulary. A reader
# looking at a red `runner-env (host-docker)` should not have to work out
# which job that corresponds to.
setup() {
[ -n "${DEJIMA_CI_TIER:-}" ] || skip "not on a CI runner (DEJIMA_CI_TIER unset)"
case "$DEJIMA_CI_TIER" in
sandbox|host-docker) ;;
*) echo "unknown DEJIMA_CI_TIER: $DEJIMA_CI_TIER" >&2; return 1 ;;
esac
}
# An off-list destination that would answer if it were reachable. This matters
# more than it looks: pointing these at an address that never answers - a
# documentation range, say - would pass in an environment with no network at
# all, which is the exact failure this suite exists to detect.
OFF_LIST_IP=1.1.1.1
# The positive control's destination. Overridable because it has to be a name
# THIS runner allows, and an allowlist is per-operator - hardcoding one would
# fail on a runner that is correctly configured for a different set.
ALLOWED_HOST=${DEJIMA_CI_ALLOWED_HOST:-deb.debian.org}
connects() { # connects HOST PORT -> 0 if a TCP connection completes
python3 - "$1" "$2" <<'PY'
import socket, sys
try:
socket.create_connection((sys.argv[1], int(sys.argv[2])), timeout=6).close()
except OSError:
sys.exit(1)
PY
}
# --- the grant that decides everything --------------------------------------
@test "the sandbox jobs are not handed the Docker daemon" {
[ "$DEJIMA_CI_TIER" = sandbox ] || skip "asserted for the sandbox runner"
# act_runner mounts /var/run/docker.sock into every job container unless
# container.docker_host is "-". An unset value is the default, and the
# default mounts - so this is the assertion that a runner was actually
# configured rather than merely intended.
[ ! -e /var/run/docker.sock ] || {
echo "this runner grants job containers the Docker socket, which is" >&2
echo "root-equivalent on the runner host. Set container.docker_host: \"-\"" >&2
echo "in its config.yaml - see ci/act-runner-ci.example.yaml." >&2
false
}
}
@test "the host-docker job does have the daemon it is scheduled for" {
[ "$DEJIMA_CI_TIER" = host-docker ] || skip "asserted for the host-docker runner"
# The positive control for the label routing. Without it, a host-docker
# job that landed on a hardened runner would fail later and obscurely, and
# a sandbox job that landed on this one would be caught only by the test
# above - one direction of a two-direction problem.
[ -e /var/run/docker.sock ]
}
@test "the two runners are not the same runner" {
# Whatever else is true, exactly one of them gets the daemon. If both jobs
# report the same answer here, the labels are not routing to two different
# runners and the tiering exists only on paper.
if [ "$DEJIMA_CI_TIER" = sandbox ]; then
[ ! -e /var/run/docker.sock ]
else
[ -e /var/run/docker.sock ]
fi
}
# --- egress, for the tier the rules actually apply to ------------------------
#
# Deliberately scoped to the sandbox runner. host-docker's job runs in the
# HOST network namespace - that is the grant they exist for - so the bridge
# rules in ci/iptables-ci.example.sh do not sit in front of them at all. Their
# egress is the host's own, which this repository's examples do not define.
# Asserting an allowlist there would test nothing and pass anyway, which is
# the failure mode this whole suite is about.
@test "an allowlisted destination is reachable" {
[ "$DEJIMA_CI_TIER" = sandbox ] || skip "bridge rules do not apply in the host netns"
# The positive control for everything below. Without it, a runner with no
# network at all passes every negative assertion in this file.
run getent hosts "$ALLOWED_HOST"
[ "$status" -eq 0 ] || {
echo "$ALLOWED_HOST did not resolve, so the negative checks below" >&2
echo "would pass for the wrong reason. Check the resolver and the" >&2
echo "allowlist rendered from .dejima/allowed-domains on main - or set" >&2
echo "DEJIMA_CI_ALLOWED_HOST to a name this runner does allow." >&2
false
}
run connects "$ALLOWED_HOST" 80
[ "$status" -eq 0 ]
}
@test "a name that is on no allowlist does not resolve" {
[ "$DEJIMA_CI_TIER" = sandbox ] || skip "bridge rules do not apply in the host netns"
# .invalid is reserved by RFC 2606 precisely so that it never resolves
# anywhere - but here it has to be REFUSED by a resolver that answers for
# allowlisted names, which the positive control above establishes.
run getent hosts example.invalid
[ "$status" -ne 0 ]
}
@test "a job cannot reach a resolver of its own choosing" {
[ "$DEJIMA_CI_TIER" = sandbox ] || skip "bridge rules do not apply in the host netns"
# The check most easily left out, and the one that decides whether the DNS
# allowlist means anything: a job that can query any resolver it likes can
# resolve anything it likes, whatever dnsmasq is configured with. Traffic
# to the resolver on the bridge gateway is addressed to the host, so it is
# an INPUT packet and never passes through DOCKER-USER - filtering only
# DOCKER-USER leaves this open.
run connects "$OFF_LIST_IP" 53
[ "$status" -ne 0 ] || {
echo "reached $OFF_LIST_IP:53, so this job can use any resolver it likes and" >&2
echo "the DNS allowlist is decorative. The INPUT rules are missing - see" >&2
echo "ci/iptables-ci.example.sh." >&2
false
}
}
@test "a destination that is on no allowlist is unreachable" {
[ "$DEJIMA_CI_TIER" = sandbox ] || skip "bridge rules do not apply in the host netns"
# An IP literal rather than a name, so this tests the firewall rather than
# the resolver. Reaching it by name would fail at DNS first and prove
# nothing about DOCKER-USER.
run connects "$OFF_LIST_IP" 443
[ "$status" -ne 0 ] || {
echo "reached $OFF_LIST_IP:443, so this job has unrestricted egress." >&2
echo "" >&2
echo "If the DNS checks above passed, the resolver is fine and the packet" >&2
echo "filter is not - which usually means DOCKER-USER is empty. Docker" >&2
echo "recreates its own chains on every daemon start, so a 'systemctl" >&2
echo "restart docker' drops these rules and nothing looks different" >&2
echo "afterwards. Check with: iptables -S DOCKER-USER" >&2
echo "" >&2
echo "ci/dejima-ci-firewall.example.service reapplies them on every" >&2
echo "Docker start, which is the fix rather than re-running the script." >&2
false
}
}
@test "IPv6 egress is not an unfiltered way around all of the above" {
[ "$DEJIMA_CI_TIER" = sandbox ] || skip "bridge rules do not apply in the host netns"
# An address family with no allowlist is not a smaller hole than the one
# the rules above close; on a runner with IPv6 egress it is the same hole.
#
# Unlike the checks above, this one passes for two different reasons - v6
# rejected, or v6 never configured - and both are acceptable outcomes, so
# it is not given a positive control. What it rules out is the third case:
# a runner with working IPv6 egress and rules written only for v4.
run connects 2606:4700:4700::1111 443
[ "$status" -ne 0 ]
}
+10
View File
@@ -85,6 +85,16 @@ wait_ready() {
if docker exec "$container" pgrep -x -f 'sleep infinity' >/dev/null 2>&1; then
return 0
fi
# A container that has already exited is never going to become ready,
# and polling it for the full two minutes turns a startup failure - an
# aborted entrypoint, say - into a wait long enough to read as a hang.
# The reason is in its logs, which are printed either way; this only
# decides how long it takes to see them.
if [ "$(docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null)" = false ]; then
echo "container $container exited during startup - log:" >&2
settled_logs "$container" >&2
return 1
fi
sleep 1
done
echo "container $container never reached the post-entrypoint sleep - startup log:" >&2
+263 -6
View File
@@ -1719,6 +1719,220 @@ EOF
grep -qi 'never from the checkout' "$REPO_ROOT/docs/ci.md"
}
# --- ci/ examples and the labels this repo's own workflow asks for ---
#
# Whether a job container gets the Docker socket is settled by the runner's
# config.yaml, not by anything in .gitea/workflows/tests.yml - so the only
# lever this repo has is which runner its jobs are allowed to land on. A
# generic label gives that lever away, which is what these pin.
@test "no workflow job runs on a generic runner label" {
run grep -nE '^\s*runs-on:\s*(ubuntu|windows|macos|self-hosted)' \
"$REPO_ROOT"/.gitea/workflows/*.yml
[ "$status" -ne 0 ] || {
echo "a generic label lets this job land on a runner nobody hardened:" >&2
echo "$output" >&2
false
}
}
@test "every workflow job runs on a dejima-specific label" {
while IFS= read -r label; do
[[ "$label" == dejima-* ]] || {
echo "unexpected runs-on label: $label" >&2
false
}
done < <(grep -hE '^\s*runs-on:' "$REPO_ROOT"/.gitea/workflows/*.yml \
| sed -E 's/^\s*runs-on:\s*//')
}
# The unprivileged example is the one an operator copies. docker_host: "-" is
# the single line standing between an ordinary job and root on the runner
# host, and it is the kind of line a tidying edit removes because it looks
# like a placeholder.
@test "the unprivileged runner example refuses the docker socket and mounts nothing" {
local f="$REPO_ROOT/ci/act-runner-ci.example.yaml"
grep -qE '^\s*docker_host:\s*"-"' "$f"
grep -qE '^\s*valid_volumes:\s*\[\]' "$f"
grep -qE '^\s*privileged:\s*false' "$f"
run grep -nE '^\s*network:\s*host' "$f"
[ "$status" -ne 0 ]
}
@test "only the privileged runner example grants the socket and the host netns" {
local f="$REPO_ROOT/ci/act-runner-runtime.example.yaml"
grep -qE '^\s*network:\s*host' "$f"
# Says plainly what it costs, rather than granting it in passing.
grep -q 'can take this host' "$f"
grep -q 'dejima-runtime' "$f"
}
# daemon.json is JSON, and dockerd refuses to start on a directive it does not
# recognise - so a comment key in the example would be an example that cannot
# be used. The commentary lives in ci/README.md instead.
@test "the daemon.json example is parseable and carries no comment keys" {
local f="$REPO_ROOT/ci/docker-daemon.example.json"
python3 -c "
import json,sys
d = json.load(open('$f'))
bad = [k for k in d if k.startswith('_')]
sys.exit('comment keys dockerd would reject: %s' % bad if bad else 0)
"
}
# The INPUT half is the one that gets left out, and leaving it out is silent:
# every rule still loads, jobs still run, and a job can resolve anything it
# likes by asking a resolver that is not this one.
@test "the firewall example filters INPUT as well as DOCKER-USER" {
local f="$REPO_ROOT/ci/iptables-ci.example.sh"
grep -q 'DOCKER-USER' "$f"
grep -qE '^iptables -A INPUT .*--dport 53 -j ACCEPT' "$f"
grep -qE '^iptables -A INPUT .*-j REJECT' "$f"
}
# The runner's own socket bind and container.docker_host name the same path
# and do opposite jobs, which is the easiest thing here to conflate: one lets
# the runner create job containers, the other decides whether those containers
# get the daemon. An example that dropped the bind would look "more hardened"
# and simply not work; one whose docs stopped explaining the difference would
# invite someone to drop it for that reason.
@test "every compose example keeps the runner's own socket and explains why it is not the job's" {
for f in "$REPO_ROOT"/ci/docker-compose*.example.yml; do
grep -qE '^\s*- /var/run/docker\.sock:/var/run/docker\.sock$' "$f"
grep -q 'docker_host' "$f"
# Without CONFIG_FILE the config is not read and every default
# applies, silently - so each example has to set it.
grep -qE '^\s*CONFIG_FILE: /config\.yaml' "$f"
# The config file decides what jobs are granted; the runner never
# writes it.
grep -qE ':/config\.yaml:ro$' "$f"
done
grep -q 'CONFIG_FILE' "$REPO_ROOT/ci/README.md"
}
# The two tiers ship as two files because the privileged one belongs on its
# own host, and a compose file that assumes that is harder to drift away from
# than one that has to be split later. A runtime service appearing in the
# shared file would undo that silently - it would work, and it would quietly
# make co-location the default.
@test "the shared compose example carries no privileged runner" {
local f="$REPO_ROOT/ci/docker-compose.example.yml"
# Non-comment lines only: the file legitimately mentions the other one in
# prose, and a guard that cannot tell a cross-reference from a service
# definition would push that cross-reference out of the file.
run bash -c "grep -vE '^[[:space:]]*#' '$f' | grep -nE 'runtime|network: *host' || true"
[ -z "$output" ] || {
echo "the privileged runner belongs in docker-compose-runtime.example.yml:" >&2
echo "$output" >&2
false
}
[ -f "$REPO_ROOT/ci/docker-compose-runtime.example.yml" ]
grep -q 'act-runner-runtime.yaml' "$REPO_ROOT/ci/docker-compose-runtime.example.yml"
}
# A process holding the Docker socket should not share a network with the
# containers running untrusted code - the DOCKER-USER rules are what keep a
# job away from it, and attaching the runner to the job network would route
# straight around them.
#
# Grep rather than a YAML parse: the job container this suite runs in has
# python3 but no PyYAML, and adding a package host to the allowlist to check
# an example file would cost more than the check is worth.
@test "no compose service is attached to the job network" {
local f
for f in "$REPO_ROOT"/ci/docker-compose*.example.yml; do
# Every service attaches to the runners' own network and nothing else.
run bash -c "grep -E '^[[:space:]]+networks:' '$f' | grep -v '\[runners\]' || true"
[ -z "$output" ] || { echo "unexpected network attachment:" >&2; echo "$output" >&2; false; }
run grep -nE '^\s*network_mode:' "$f"
[ "$status" -ne 0 ]
# And the job network is not declared here at all. It is created by hand
# with a fixed bridge name so the firewall can match on it, and keeping it
# out of this file removes the temptation to attach a service to it - a
# process holding the Docker socket sharing a network with the containers
# running untrusted code is the thing all of this exists to prevent.
run grep -nE '^\s+(name: *)?dejima-ci:? *$' "$f"
[ "$status" -ne 0 ] || { echo "$output" >&2; false; }
done
}
# The static resolver config and the generated one are two files in the same
# dnsmasq.d, and the split is what keeps a hand-edited allowlist from drifting
# away from the one derived from main. A `server=` line in the static file
# would work, which is the problem: it is a grant nobody merged, and it would
# survive every refresh.
@test "the dnsmasq example declares no allowed names of its own" {
local f="$REPO_ROOT/ci/dnsmasq-ci.example.conf"
run grep -nE '^server=' "$f"
[ "$status" -ne 0 ] || {
echo "allowed names belong in the generated file, not this one:" >&2
echo "$output" >&2
false
}
# The mechanism that populates the firewall set has to stay, though.
grep -qE '^ipset=/#/ci-allow' "$f"
}
# Every file in ci/ is inert until it is put somewhere, and none of the paths
# are guessable - daemon.json's dns has to match the resolver's
# listen-address, and the refresh script writes into the same dnsmasq.d the
# static config lives in.
@test "ci/README.md says where each example file is installed" {
local r="$REPO_ROOT/ci/README.md"
grep -q '/etc/dnsmasq.d/' "$r"
grep -q '/etc/docker/daemon.json' "$r"
grep -q '/usr/local/sbin' "$r"
# Every example file is named in the install table.
for f in "$REPO_ROOT"/ci/*.example.*; do
grep -qF "$(basename "$f")" "$r" || {
echo "$(basename "$f") is not mentioned in ci/README.md" >&2
false
}
done
}
# A daemon-wide resolver reaches every container the daemon starts, including
# the sibling containers a privileged job creates over the Docker socket. A
# dejima sandbox is one of those, and it allowlists only its own bridge
# gateway - so a resolver on another bridge is rejected by its own firewall
# and nothing in it resolves. The symptom is not "DNS is misconfigured", it is
# the sandbox reporting that a host it allows is unreachable.
@test "the resolver is scoped to the runner's containers, not set daemon-wide" {
run grep -n '"dns"' "$REPO_ROOT/ci/docker-daemon.example.json"
[ "$status" -ne 0 ] || {
echo "daemon.json must not set dns - scope it in container.options:" >&2
echo "$output" >&2
false
}
grep -qE '^\s*options: "--dns=' "$REPO_ROOT/ci/act-runner-ci.example.yaml"
grep -q 'not daemon-wide' "$REPO_ROOT/ci/README.md"
}
# Docker removes DOCKER-USER on every daemon start, so "apply the rules once"
# is not a working instruction - and the resulting state looks identical to a
# working one, because DNS still refuses unlisted names via dnsmasq. A plain
# After=docker.service would only cover boot, which was never the case that
# broke; PartOf is what makes a restart reapply them.
@test "the firewall example ships a unit that survives a Docker restart" {
local u="$REPO_ROOT/ci/dejima-ci-firewall.example.service"
[ -f "$u" ]
grep -qE '^PartOf=docker\.service' "$u"
grep -qE '^WantedBy=docker\.service' "$u"
grep -qE '^ExecStart=/usr/local/sbin/dejima-ci-firewall' "$u"
grep -q 'dejima-ci-firewall.example.service' "$REPO_ROOT/ci/README.md"
}
@test "the allowlist refresh example reads main over the API, not a checkout" {
local f="$REPO_ROOT/ci/refresh-allowlist.example.sh"
grep -q 'REF=${REF:-main}' "$f"
grep -q 'api/v1' "$f"
# Fails closed: an empty render is refused rather than installed.
grep -q 'refusing to install it' "$f"
}
@test "sandbox-context.sh no longer mentions ALLOWED_DOMAINS" {
run grep -n 'ALLOWED_DOMAINS' "$REPO_ROOT/hooks/sandbox-context.sh"
[ "$status" -ne 0 ]
@@ -1738,18 +1952,48 @@ EOF
# This catches address- and hostname-shaped leaks only. A personal domain
# that looks like any other public one is not mechanically distinguishable
# and still rests on review.
SCANNED_PATHS=(README.md docs docker bin forges workflows harnesses hooks install.sh systemd .gitea)
SCANNED_PATHS=(README.md docs docker bin forges workflows harnesses hooks install.sh systemd .gitea ci)
# One address is exempt, by exact value. ci/ has to tell an operator to give
# the job bridge a PRIVATE subnet rather than a documentation one - because
# dnsmasq's stop-dns-rebind is keyed on the private ranges, so a
# reserved-but-not-private bridge subnet is one that check silently does not
# cover - and "use a private range" without an example is advice people
# implement wrongly. So the instruction needs a literal private address in it,
# which is exactly what this test otherwise exists to prevent.
#
# Exempting one prefix keeps the guard whole: every other 10.x, 192.168.x,
# 172.16-31.x and 169.254.x still fails, including a second occurrence of a
# different 10.x on the same line, since the filter drops matched addresses
# rather than whole lines.
#
# Three octets, because three is all the pattern above matches - which makes
# the exemption exactly the /24 the docs recommend, and nothing wider.
DOC_PRIVATE_SUBNET_EXAMPLE='10.89.0'
@test "no private or link-local address is used as a documentation example" {
run grep -rnIE '\b(10|127\.0\.0\.[2-9]|169\.254|192\.168|172\.(1[6-9]|2[0-9]|3[01]))\.[0-9]{1,3}\.[0-9]{1,3}\b' \
"${SCANNED_PATHS[@]/#/$REPO_ROOT/}"
[ "$status" -ne 0 ] || {
run bash -c "grep -rnIoE '\\b(10|127\\.0\\.0\\.[2-9]|169\\.254|192\\.168|172\\.(1[6-9]|2[0-9]|3[01]))\\.[0-9]{1,3}\\.[0-9]{1,3}\\b' \
${SCANNED_PATHS[*]/#/$REPO_ROOT/} \
| grep -v ':${DOC_PRIVATE_SUBNET_EXAMPLE}\$' || true"
[ -z "$output" ] || {
echo "use an RFC 5737 documentation address instead:" >&2
echo "$output" >&2
false
}
}
# The exemption above is only defensible while it is the address the docs
# actually recommend. If the recommendation changes and this does not, the
# exemption starts covering an address nothing points at - a hole rather than
# a carve-out.
#
# ci/README.md is the only place that recommends it: docs/ci.md is the
# reasoning and deliberately carries no concrete configuration, so requiring
# the address there would force a second copy of exactly the kind that drifts.
@test "the exempted private address is the one ci/ recommends" {
grep -qF "$DOC_PRIVATE_SUBNET_EXAMPLE" "$REPO_ROOT/ci/README.md"
}
@test "no hostname under a private-network suffix is used as a documentation example" {
run grep -rnIE '[A-Za-z0-9-]+\.(localdomain|lan|internal|home\.arpa)\b' \
"${SCANNED_PATHS[@]/#/$REPO_ROOT/}"
@@ -2158,15 +2402,21 @@ startup_warning_slug() { printf '%s' "$DEJIMA_T/ws" | sed 's/[^a-zA-Z0-9]/-/g';
[ "$status" -ne 0 ]
}
@test "every sandbox suite is in the CI matrix, and runtime.bats is not" {
@test "every suite is run by CI, and the two that need their own job have one" {
# Derived from the directory rather than a list kept here, so adding a
# test file fails this until the workflow is told about it. A suite nobody
# wired into CI runs nowhere, and nothing else would say so.
#
# Two suites are not matrix entries because they cannot be. runtime.bats
# needs a Docker daemon; ci-runner.bats asserts on the runner itself and
# has to run once per tier rather than once per suite. Both are checked
# below for having a job of their own instead - "not in the matrix" must
# not be a way to be in neither.
local wf="$REPO_ROOT/.gitea/workflows/tests.yml"
local f base missing=""
for f in "$REPO_ROOT"/test/*.bats; do
base="$(basename "$f" .bats)"
[ "$base" = "runtime" ] && continue
case "$base" in runtime|ci-runner) continue ;; esac
grep -qE "^ *- *$base\$" "$wf" || missing="$missing $base"
done
[ -z "$missing" ] || {
@@ -2179,6 +2429,13 @@ startup_warning_slug() { printf '%s' "$DEJIMA_T/ws" | sed 's/[^a-zA-Z0-9]/-/g';
# fail every time rather than testing anything.
! grep -qE '^ *- *runtime$' "$wf"
grep -q 'test/runtime.bats' "$wf"
# ci-runner.bats likewise, and once per tier: a single job would leave one
# of the two runners unasserted, which is the thing it exists to catch.
! grep -qE '^ *- *ci-runner$' "$wf"
grep -q 'test/ci-runner.bats' "$wf"
grep -q 'DEJIMA_CI_TIER: sandbox' "$wf"
grep -q 'DEJIMA_CI_TIER: host-docker' "$wf"
}
@test "the agent briefing teaches the project scope, not just global" {