providers: ship the github and none forges, the branch-push and local-only workflows, and the validation Phase 2 deferred (Phase 3 + Phase 4 remnant) #69

Merged

branderson merged 20 commits from new-providers into main 2026-08-04 14:18:55 -07:00

Conversation 0 Commits 20 Files Changed 22

+1423 -170

branderson commented 2026-08-04 12:14:15 -07:00

Owner

Copy Link

Copy Source

Summary generated by LLM agent

Phase 3 of the provider-axes work, plus the one item Phase 4 left behind.

Phase 2 built the machinery; this ships the providers that exercise it, and
turns on the validation Phase 2 deliberately deferred until there was a
correct alternative to point a refused operator at.

What ships

Forges. github — pushes and opens PRs via gh, using its own
write-capable push_token_file. That is deliberately NOT [github] token_file, which is a read-only fine-grained PAT: promoting it would turn a
credential chosen for being unable to write into the one this workflow pushes
with. dejima warns at startup when both are set, since gh reads one variable
and the write-capable one wins.

none — the honest way to say this sandbox pushes nowhere, as distinct from a
forge whose keys merely aren't filled in. The deferred rules needed it to
exist before they could refuse anything.

Workflows. branch-push (worktree, push, stop — no PR attempted) and
local-only (commit locally, push nothing). Both require no forge capability,
so both work against none.

Validation now enforced. A pushing workflow needs a forge resolving to a
host. A partly-configured forge is refused rather than warned about — but only
when the workflow pushes, and a key with a working schema default never counts
as missing. A forge with required settings and none of them configured is
refused under a pushing workflow, which is the only rule that catches a forge
whose single required key makes "partly configured" unreachable. And an
autonomy its forge cannot support is refused at launch rather than
discovered when the agent tries.

The briefing tells the truth about the network. It described the allowlist
as "GitHub read-only, the configured host". With none that rendered
"the configured none host"; with github as the forge, "read-only" was false
in the direction that matters. One resolver now states the real posture for
each case.

Phase 4 remnant. The exit-time audit names the writable mounts a session
could reach. Named, never hashed — writing to one is the point of configuring
it, so a finding per write would fire constantly and train the reader to skim.
It lands in both the terminal summary and the on-disk report, which are
separate code paths; only the first would have left dejima-audit show silent
about it.

The shipped defaults changed

[core] forge now defaults to none and [core] workflow to local-only.

They were gitea and worktree-pr — a pushing pair, which the new rule
correctly refuses when no forge is configured. That meant a fresh install
could not launch. The defaults were the false claim, not the rule: a default
of gitea + worktree-pr asserts a pushing setup an install has no credentials
for. Out of the box dejima now commits locally, and you opt into a forge.

Two lines of schema change took 26 failing launcher tests to zero with no
fixture edits, which is the clearest evidence the tests were right.

This is user-visible for anyone relying on implicit defaults. A config
with no explicit [core] forge line was getting gitea + worktree-pr and now
gets none + local-only — that sandbox stops pushing. install.sh reports it
on every run until both lines are written out, and an operator with
GITEA_SSH_HOST/GITEA_API exported still gets forge = gitea seeded
explicitly, since they are demonstrably already pushing to Gitea.

Two findings worth knowing about

forge = github could not boot a container at all. Its fixed_hosts
reached the port-unrestricted allowed-domains ipset, opening port 22, which
init-firewall.sh's startup self-check treats as fatal without the push
grant — so the container died before the agent ran. The quieter half: routing
three public GitHub names down the internal-host path also stripped their
stop-dns-rebind protection, which fixed-domains.sh deliberately preserves
for public names. Fixing the boot failure by relaxing the self-check would
have shipped that widening silently.

Such a forge now declares hosts_allowlisted_by_core, and core reads it
generically: those hosts stay out of DEJIMA_FORGE_HOSTS, still reach
DEJIMA_PUSH_HOSTS for the push guard, and port 22 opens through the
documented, logged, self-checked ALLOW_GITHUB_PUSH path — only when the
selected workflow actually pushes.

The documented gitea setup was briefly refused. Turning the
partly-configured rule from a warning into a refusal exposed that it asked
"did the operator write this line", which is the right question for a warning
and the wrong one for a refusal: [forge.gitea] token_file has a schema
default, so it always has a working value. Following the README produced
exit 1. Fixed, and the test that would have caught it is restored to assert
the documented setup.

Testing

static 160, project 71, remote 104, audit 141, config 94,
providers 64, plus proxy — full sandbox set green.

test/runtime.bats needs a Docker daemon and could not run in the sandbox
this was built in. It gained a case for the github-forge boot path — precisely
the failure above — which has NOT been executed. CI's host-docker job is the
first thing to check on this PR.

Refs #56

**Summary generated by LLM agent** Phase 3 of the provider-axes work, plus the one item Phase 4 left behind. Phase 2 built the machinery; this ships the providers that exercise it, and turns on the validation Phase 2 deliberately deferred until there was a correct alternative to point a refused operator at. ## What ships **Forges.** `github` — pushes and opens PRs via `gh`, using its own write-capable `push_token_file`. That is deliberately NOT `[github] token_file`, which is a read-only fine-grained PAT: promoting it would turn a credential chosen for being unable to write into the one this workflow pushes with. dejima warns at startup when both are set, since `gh` reads one variable and the write-capable one wins. `none` — the honest way to say this sandbox pushes nowhere, as distinct from a forge whose keys merely aren't filled in. The deferred rules needed it to exist before they could refuse anything. **Workflows.** `branch-push` (worktree, push, stop — no PR attempted) and `local-only` (commit locally, push nothing). Both require no forge capability, so both work against `none`. **Validation now enforced.** A pushing workflow needs a forge resolving to a host. A partly-configured forge is refused rather than warned about — but only when the workflow pushes, and a key with a working schema default never counts as missing. A forge with required settings and none of them configured is refused under a pushing workflow, which is the only rule that catches a forge whose single required key makes "partly configured" unreachable. And an `autonomy` its forge cannot support is refused at launch rather than discovered when the agent tries. **The briefing tells the truth about the network.** It described the allowlist as "GitHub read-only, the configured <forge> host". With `none` that rendered "the configured none host"; with `github` as the forge, "read-only" was false in the direction that matters. One resolver now states the real posture for each case. **Phase 4 remnant.** The exit-time audit names the writable mounts a session could reach. Named, never hashed — writing to one is the point of configuring it, so a finding per write would fire constantly and train the reader to skim. It lands in both the terminal summary and the on-disk report, which are separate code paths; only the first would have left `dejima-audit show` silent about it. ## The shipped defaults changed `[core] forge` now defaults to `none` and `[core] workflow` to `local-only`. They were `gitea` and `worktree-pr` — a pushing pair, which the new rule correctly refuses when no forge is configured. That meant a fresh install could not launch. The defaults were the false claim, not the rule: a default of gitea + worktree-pr asserts a pushing setup an install has no credentials for. Out of the box dejima now commits locally, and you opt into a forge. Two lines of schema change took 26 failing launcher tests to zero with no fixture edits, which is the clearest evidence the tests were right. **This is user-visible for anyone relying on implicit defaults.** A config with no explicit `[core] forge` line was getting gitea + worktree-pr and now gets none + local-only — that sandbox stops pushing. `install.sh` reports it on every run until both lines are written out, and an operator with `GITEA_SSH_HOST`/`GITEA_API` exported still gets `forge = gitea` seeded explicitly, since they are demonstrably already pushing to Gitea. ## Two findings worth knowing about **`forge = github` could not boot a container at all.** Its `fixed_hosts` reached the port-unrestricted `allowed-domains` ipset, opening port 22, which `init-firewall.sh`'s startup self-check treats as fatal without the push grant — so the container died before the agent ran. The quieter half: routing three *public* GitHub names down the internal-host path also stripped their `stop-dns-rebind` protection, which `fixed-domains.sh` deliberately preserves for public names. Fixing the boot failure by relaxing the self-check would have shipped that widening silently. Such a forge now declares `hosts_allowlisted_by_core`, and core reads it generically: those hosts stay out of `DEJIMA_FORGE_HOSTS`, still reach `DEJIMA_PUSH_HOSTS` for the push guard, and port 22 opens through the documented, logged, self-checked `ALLOW_GITHUB_PUSH` path — only when the selected workflow actually pushes. **The documented gitea setup was briefly refused.** Turning the partly-configured rule from a warning into a refusal exposed that it asked "did the operator write this line", which is the right question for a warning and the wrong one for a refusal: `[forge.gitea] token_file` has a schema default, so it always has a working value. Following the README produced `exit 1`. Fixed, and the test that would have caught it is restored to assert the documented setup. ## Testing `static` 160, `project` 71, `remote` 104, `audit` 141, `config` 94, `providers` 64, plus `proxy` — full sandbox set green. `test/runtime.bats` needs a Docker daemon and could not run in the sandbox this was built in. It gained a case for the github-forge boot path — precisely the failure above — which has NOT been executed. CI's `host-docker` job is the first thing to check on this PR. Refs #56

branderson added 17 commits 2026-08-04 12:14:16 -07:00

forges: add none, for a sandbox that pushes nowhere 13c04748f0

forges: trim none's manifest comment to the one surprising fact e216708cc8

briefing: state the network posture the selected forge actually gives f4bfc78631

workflows: add branch-push, which stops at the branch f48e5827bb

workflows: add local-only, which pushes nothing 2f2722a9f2

forges: add github, pushing with its own write-capable token de38870e1f

forges: stop github's collision warning lying, stop it clobbering the read-only token ... ea49f6a0db

Fix round 1 on code review of the github forge (task 5):

- forge_env_exports now skips an env_export pair whose resolved value is
  empty instead of emitting NAME= - selecting forge = github with
  push_token_file unset was overriding docker-compose.yml's own read-only
  GH_TOKEN with an explicit empty one via -e, silently voiding the baseline
  GitHub read access every session gets regardless of forge.
- The collision warning now also checks that push_token_file is actually
  set before claiming GH_TOKEN comes from it - previously it only checked
  the read-only token_file, so it could fire (with a false claim) even when
  push_token_file was never configured.

providers: refuse an autonomy the selected forge cannot support 623d565ebb

providers: refuse an unconfigured or half-configured pushing forge ... 6d0f603b95

Turn provider_validate_selection's two deferred rules into refusals now
that forge = none (Task 1) gives a refused operator a correct alternative
to point at: rule 3 (a pushing workflow needs a forge that resolves to a
host) and rule 5 (a partly configured forge is a typo, not a choice).

Closes a gap rule 3 alone leaves open: github pins fixed_hosts
independent of config, so forge_hosts is never empty regardless of
push_token_file, and github's single-key requires can never look
"partly" configured either - a wholly unconfigured github pushing forge
would sail past both rules otherwise. Added a third check, scoped like
rule 3 to a pushing workflow, that refuses when none of a forge's
requires keys are set.

Updated bin/dejima's now-stale comment describing the two rules as
deferred, and every test (in providers.bats and config.bats) that
launched with the default gitea/worktree-pr pair unconfigured, or a
gitea forge only partly configured - switched to forge = none +
workflow = local-only where the test wasn't actually about forge
selection, or to a fully configured gitea/github where it was.

providers: fix round 1 - default-valued keys, push-gated rule 5, honest none message ... 9802328321

Three operator rulings on Task 7's review:

- A requires key with a non-empty schema default (gitea's token_file,
  defaulting to ~/.config/gitea/token) can never be genuinely missing, so
  it no longer counts toward set_n/unset_n. Without this, the README's
  documented gitea setup (ssh_host + api only, token saved at the default
  path) was refused as "partly configured" at every launch.
- Rule 5 is now gated on push = on, matching the single-required-key check
  it sits next to. A non-pushing workflow never invokes the forge, so
  setting only ssh_host (e.g. so the firewall allows fetches) is a
  legitimate configuration, not a typo.
- Rule 3's message for a forge with no keys at all (forge = none) no
  longer tells the operator to "Configure [forge.none]" or "set forge =
  none" when the forge already is none - it names the only real fix
  (a non-pushing workflow) instead.

Also: reuse the already-computed $push instead of re-deriving it in a
subshell, and fix a stale "warning" comment in test/config.bats.

docs: document the shipped providers and the rules now enforced d27bec6643

docs: fix two more stale forge-count references (review round 1) ... cd6b911114

The requires-key table row and docs/security.md still described a single
shipped forge and a warning-not-refusal, both stale since the earlier
providers.md rewrite in this task.

providers: fix round 2 - drop forge names from core comments, fix audit-startup fixture ... 7d93b33f06

Two static.bats regressions from fix round 1:

- "core names no forge": my two fix-round comments used gitea as a worked
  example, violating the rule this branch exists to establish (core names
  no forge, in code or prose). Rewritten to make the same point generically
  (a required key with a non-empty schema default already has a working
  value; a non-pushing workflow never invokes the forge at all) without
  naming one.
- The four bin/dejima audit-startup tests share a fixture
  (startup_warning_setup) that wrote a config with no forge/workflow
  override - the default gitea/worktree-pr pair, unconfigured, which rule 3
  now refuses before bin/dejima ever reaches the audit-startup path those
  tests are about. Added forge = none / workflow = local-only to the shared
  fixture, the same fix already applied across test/config.bats.

audit: name the writable mounts a session could reach 982be2e391

providers: fix the final review's five must-fix findings ... 9d82c7b569

C1: `forge = github` could not boot a container with the firewall on. Its
fixed_hosts reached DEJIMA_FORGE_HOSTS, which entrypoint.sh folds into
EXTRA_ALLOWED_DOMAINS - the path that resolves a name into the
port-unrestricted allowed-domains ipset and exempts it from
stop-dns-rebind. Three public GitHub names therefore got all-port egress
and lost rebind protection, and init-firewall.sh's own self-check then
found github.com:22 reachable without ALLOW_GITHUB_PUSH=1 and killed the
container before the agent started.

New forge manifest key hosts_allowlisted_by_core (bool, default off), set
by forges/github/forge.conf and read generically by core: its hosts stay
out of DEJIMA_FORGE_HOSTS, still fill DEJIMA_PUSH_HOSTS so the push guard
permits the push, and when the workflow pushes, port 22 opens through
core's own grant (ALLOW_GITHUB_PUSH) - the documented, logged,
self-checked path. Core branches on the manifest value, never on a name.

Also:
- the "core names no forge" test now builds its pattern from
  provider_list forge, so every shipped forge name is covered, matching a
  name in literal position (quoted, bracketed, forge.-prefixed, case arm)
  alongside the original gitea/github.com prose scan. The two core sites
  it was blind to are gone: the GH_TOKEN collision warning is derived
  from the selected forge's env_export, and the briefing's
  network-posture sentence from hosts_allowlisted_by_core plus the
  workflow's push.
- workflows/worktree-pr/briefing.md no longer tells a github-forge
  session to push to Gitea and never to GitHub: "the remote",
  {{blocked_branches}} instead of a hardcoded main/master, and "the
  forge" instead of Gitea.
- rules 5 and 6 require a required key to resolve to a non-empty value,
  not merely to be written. install.sh writes every key commented out, so
  uncommenting one yields `api =` - set, and worth nothing, with every
  forge call failing inside the container instead.
- the README no longer says a fresh install is enough to run dejima (it
  is refused: gitea + worktree-pr with no host), nor that
  --allow-github-push replaces selecting a forge; rule 3's remedy offers
  forge = github; the network-egress and security.md passages the C1 fix
  falsified are corrected.

Regression tests for each, including a runtime.bats case that boots with
the github forge's resolved environment and asserts the container comes
up with port 22 in the documented posture (needs Docker; not run here).

providers: fix residual findings from the final review ... 0f5727e206

- README.md: the --allow-github-push section still claimed no config-file
  path could grant a push path, contradicted since 9d82c7b let
  [core] forge = github do exactly that when the workflow pushes; rewrite
  to state both truths and align with docs/security.md.
- docs/providers.md: qualify the shipped-providers table's "opens
  ssh.github.com:22" with "when the selected workflow pushes", matching
  the prose below it; also document hosts_allowlisted_by_core's
  GitHub-specific trap for future forge authors.
- bin/dejima-provider-lib.sh: comment the same trap where the manifest key
  is declared.
- test/providers.bats: assert the no-host remedy names a forge that
  actually exists (provider_list forge) instead of pinning the literal
  text "forge = github", which would go stale silently if that forge were
  renamed or removed.
- bin/dejima: skip _warn_forge_env_collisions when config_get core forge
  already failed, so an invalid forge value reports one error instead of
  two; add a regression test.
- test/static.bats: declare the missing local `body`.

providers: ship the non-pushing pair as the default, and say so on upgrade ... 47832effc2

The shipped defaults were core.forge = gitea and core.workflow = worktree-pr
- a PUSHING pair. Rule 3 in provider_validate_selection refuses a workflow
that pushes when its forge resolves to no host, so every config without a
filled-in [forge.gitea] was refused at launch, including the one install.sh
writes. A fresh install could not start; 26 launcher tests in project.bats
and remote.bats failed for exactly that reason.

The rule is right. The defaults were the thing asserting a setup no fresh
install has, so the defaults change: forge = none, workflow = local-only.
Pushing is opted into. All 26 failures resolve with no launcher fixture
touched.

This is a behaviour change for an existing operator with no explicit
[core] forge / workflow line - their sandbox stops pushing - so it is
reported rather than silent:

- install.sh prints a notice when an existing config parses but does not set
  both keys explicitly, naming the old and new values and the two lines to
  write to keep pushing. Same stance as the remote.conf "no longer read"
  notice: reported, never rewritten.
- The starter config now writes forge and workflow uncommented, so a config
  states the selection it makes rather than inheriting a future default.
- install.sh's legacy seed path is repaired: _config_extra_keys enumerates
  only the selected forge's keys, so with `none` selected [forge.gitea]
  vanished from the starter config and GITEA_SSH_HOST/GITEA_API had nowhere
  to land. An operator with those exported was pushing to Gitea, so
  install.sh selects gitea + worktree-pr for them explicitly.
- README and docs/providers.md restate the defaults, with the reasoning and
  a migration callout.

providers.bats pins the property that broke: the shipped defaults must be a
non-pushing pair that validates with no forge configured, driven off the
schema rather than literal names. config.bats' embedded-newline token test
was relying on the old default to select the forge whose token_file it
poisons, and now names gitea explicitly - its subject genuinely needs one.

branderson added 2 commits 2026-08-04 12:52:50 -07:00

test: wait for the startup line this asserts on, instead of reading the log once ... 03c2d3abd9

The json-file log driver's writes lag the process that produced them - the
same short read settled_logs() was written for. It landed here as a CI
failure in which the container booted, port 22 was reachable, and the
rebind assertion held: everything the test guards passed, and only the
grep for the line announcing the branch that had demonstrably been taken
came back empty.

Merge remote-tracking branch 'origin/main' into new-providers 63084ccd87

branderson referenced this pull request 2026-08-04 13:26:06 -07:00

gitea: add a CLI for reading CI runs and job logs #71

branderson added 1 commit 2026-08-04 14:00:48 -07:00

test: wait for the client before reading the window an attach selects ... 16104760dd

dejima-attach creates the grouped session detached, checks it holds the
pane, selects the window, and only then attaches. await_group waited on
has-session, which goes true at the first of those four steps - so a caller
that went on to assert the window index was reading it two steps early. It
held locally and failed on a loaded CI runner, with the group assertion on
the line before it passing: the group is set at creation, the window is not.

Waits for the client instead, the same way the earlier attach test does, and
for the same reason. 12 consecutive runs of both affected tests, no failures.

Also retries the api.github.com self-check in init-firewall.sh. Not from this
branch - it fails on main the same way, and did here in one derived-image
container while a dozen others in the same run came up fine. A transient DNS
answer or a CDN edge rotating mid-boot answers "can the allowlist reach
anything" wrongly in the direction that kills the container. Bounded and
reported like the ranges fetch above it. Its opposite number, the
example.com check, is deliberately left un-retried: a single success there is
the real answer, and retrying would be asking until it says what we want.

branderson merged commit b95a742b8d into main 2026-08-04 14:18:55 -07:00

branderson referenced this issue from a commit 2026-08-04 14:18:56 -07:00

Merge pull request 'providers: ship the github and none forges, the branch-push and local-only workflows, and the validation Phase 2 deferred (Phase 3 + Phase 4 remnant)' (#69) from new-providers into main

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

Compat/Breaking

Breaking change that won't be backward compatible

Kind/Bug

Something is not working

Kind/Documentation

Documentation changes

Kind/Enhancement

Improve existing functionality

Kind/Feature

New functionality

Kind/Security

This is security issue

Kind/Testing

Issue or pull request related to testing

Priority

Critical

1

The priority is critical

Priority

High

2

The priority is high

Priority

Low

4

The priority is low

Priority

Medium

3

The priority is medium

Reviewed

Confirmed

1

Issue has been confirmed

Reviewed

Duplicate

2

This issue or pull request already exists

Reviewed

Invalid

3

Invalid issue

Reviewed

Won't Fix

3

This issue won't be fixed

Status

Abandoned

3

Somebody has started to work on this but abandoned work

Status

Blocked

1

Something is blocking this issue or pull request

Status

Need More Info

2

Feedback is required to reproduce issue or to continue work

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

branderson dejima-agent

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dejima/dejima#69