Say when the audit is describing the wrong machine #6

2026-08-10T18:35:37-07:00

dejima-agent commented

2026-08-10 18:35:37 -07:00

Summary generated by LLM agent

The audit reports on the machine it runs on. Inside a container it reports on the container, and says nothing about the difference.

What goes wrong

Every reading is real and none of them is about the devbox. The gateway probed is the container's bridge, not the VM's first hop. Any credential found is whatever is mounted there — in a dejima session that is the placeholder the proxy substitutes, not the real token. The group memberships are the image's. The verdict at the end is about the container's egress.

The output looks exactly like a reading of the devbox. That is the problem: it is plausible, and wrong about the thing it appears to answer.

This is not hypothetical. I ran it that way repeatedly while building the tool and read container answers each time.

Detected, not declared, and that is deliberate

test/devbox.bats is gated on DEJIMA_DEVBOX=1 because a detector that guesses wrong makes a test skip silently and assert nothing. This does the opposite thing on purpose.

The difference is which way each fails. A mis-detecting test reports success having checked nothing. A mis-detecting banner prints one extra paragraph. Loud and wrong costs a paragraph; quiet and wrong is the failure the declared variable exists to prevent. So the same repository uses opposite mechanisms for the two, and the reason is written where each one lives.

Detection is /.dockerenv, /run/.containerenv, and /proc/1/cgroup. The first fires in a dejima session; the second covers Podman; the third catches runtimes that set neither.

It warns rather than refusing

Someone reading the output while working on the tool has a good reason to be there, and refusing would teach them to route around the check. The banner goes first, before any finding, because after the report it is a footnote on conclusions already drawn.

On testing this

You are right that an automated test of the real thing would say nothing — the credential and the gateway only exist as themselves on an actual devbox host, and CI has neither.

So the tests are about the warning, not about the environment: it fires when detection says so, it appears before the findings rather than after, it warns without blocking, and it stays quiet otherwise. All four run anywhere and none pretends to know what machine it is on.

One of them is narrower than it first looks. The "stays quiet" test matches the banner's own wording rather than the word "container", because the report says that legitimately in several places and a test keyed to the bare word would fail for reasons unrelated to this. I wrote it the broad way first and it failed immediately, which is the version of that mistake I would rather have.

Verified

bats test/ passes, 68 ok and 0 failed. The banner fires correctly in this container, which is the one environment available to check it in.

**Summary generated by LLM agent** The audit reports on the machine it runs on. Inside a container it reports on the container, and says nothing about the difference. ## What goes wrong Every reading is real and none of them is about the devbox. The gateway probed is the container's bridge, not the VM's first hop. Any credential found is whatever is mounted there — in a dejima session that is the *placeholder* the proxy substitutes, not the real token. The group memberships are the image's. The verdict at the end is about the container's egress. The output looks exactly like a reading of the devbox. That is the problem: it is plausible, and wrong about the thing it appears to answer. This is not hypothetical. I ran it that way repeatedly while building the tool and read container answers each time. ## Detected, not declared, and that is deliberate `test/devbox.bats` is gated on `DEJIMA_DEVBOX=1` because a detector that guesses wrong makes a test skip silently and assert nothing. This does the opposite thing on purpose. The difference is which way each fails. A mis-detecting test reports success having checked nothing. A mis-detecting banner prints one extra paragraph. Loud and wrong costs a paragraph; quiet and wrong is the failure the declared variable exists to prevent. So the same repository uses opposite mechanisms for the two, and the reason is written where each one lives. Detection is `/.dockerenv`, `/run/.containerenv`, and `/proc/1/cgroup`. The first fires in a dejima session; the second covers Podman; the third catches runtimes that set neither. ## It warns rather than refusing Someone reading the output while working on the tool has a good reason to be there, and refusing would teach them to route around the check. The banner goes first, before any finding, because after the report it is a footnote on conclusions already drawn. ## On testing this You are right that an automated test of the real thing would say nothing — the credential and the gateway only exist as themselves on an actual devbox host, and CI has neither. So the tests are about the *warning*, not about the environment: it fires when detection says so, it appears before the findings rather than after, it warns without blocking, and it stays quiet otherwise. All four run anywhere and none pretends to know what machine it is on. One of them is narrower than it first looks. The "stays quiet" test matches the banner's own wording rather than the word "container", because the report says that legitimately in several places and a test keyed to the bare word would fail for reasons unrelated to this. I wrote it the broad way first and it failed immediately, which is the version of that mistake I would rather have. ## Verified `bats test/` passes, 68 ok and 0 failed. The banner fires correctly in this container, which is the one environment available to check it in.

dejima-agent referenced this pull request

2026-08-10 20:27:57 -07:00

Carry dejima's canonical statements: the disclaimer, and the push rule #8

dejima-agent force-pushed container-banner from 66c4676997 to 29a662b24c

2026-08-11 18:27:26 -07:00

Compare

dejima-agent added 1 commit 2026-08-11 18:35:03 -07:00

Say when the audit is describing the wrong machine

tests / runner-env (pull_request_target) Successful in 31s

Details

tests / tests (pull_request_target) Successful in 47s

Details

67db46f3c6

The program reports on the machine it runs on. Inside a container it reports on the container, and every reading is real: the gateway probed is the bridge, the credential found is whatever is mounted there, the group memberships are the image's. None of it is about the devbox and nothing in the output said so.

Detected rather than declared, which is the opposite of test/devbox.bats, and the reason is the direction each fails in. A test that mis-detects skips and asserts nothing, so it takes a declared variable and refuses to guess. This one mis-detects and prints an extra paragraph. Loud and wrong costs a paragraph; quiet and wrong is what the declared variable exists to prevent.

It warns rather than refusing. Someone reading the output while developing the tool has a reason to be there, and refusing would teach them to work around the check.

The tests are about the warning and not about the environment: whether it fires when detection says so, whether it comes before the findings rather than after them, whether it warns without blocking. Those run anywhere. Asserting anything about a real devbox from CI would be asserting nothing, which is the trap this repository keeps finding in its own tests.

dejima-agent force-pushed container-banner from 29a662b24c to 67db46f3c6

2026-08-11 18:35:03 -07:00

Compare

branderson approved these changes 2026-08-11 18:36:22 -07:00

branderson merged commit 5b06aeccb2 into main

2026-08-11 18:36:29 -07:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dejima/dejima-devbox-hardener#6