Say what changed, hold alerts during expected changes, and shorten the commands
Alerts now name the kind of change and the watched area it falls under, from a fixed list, and report lists each path as added, modified or removed. ignore_paths keeps a change in the report without alerting on it. dejima-guard expect opens a window that holds change and unreachable alerts, then accepts at its end. Critical paths and containment still alert, and a window never accepts a critical path. accept now runs a check afterwards so the heartbeat resumes, status summarises the guard's state, and bin/dejima-guard-docker runs any command in the container from any directory.
This commit is contained in:
@@ -236,6 +236,7 @@ other end of the connection, not the machine the guard runs on.
|
||||
| `ssh_known_hosts` | Where the devbox's host key is remembered. Must persist: in a container, on the state volume. Populated by `dejima-guard trust-host`, never automatically |
|
||||
| `ssh_timeout` | Seconds to wait. The line between a slow devbox and a gone one |
|
||||
| `critical_paths` | Paths whose change is its own max-priority alert, one per line. Defaults to the audit binary, the forced command, the sudoers rule, `sshd_config` and `authorized_keys` |
|
||||
| `ignore_paths` | Paths whose change is reported but never alerted on, space-separated substrings. None by default. See [Changes not worth waking for](#changes-not-worth-waking-for) |
|
||||
|
||||
**Where alerts go.**
|
||||
|
||||
@@ -245,6 +246,7 @@ other end of the connection, not the machine the guard runs on.
|
||||
| `ntfy_token` | Write access to that topic and nothing else. The reason the config file is `0640` |
|
||||
| `heartbeat_url` | GET after a cycle that completed clean. An Uptime Kuma push monitor. **A credential**: the token is in the path, so it belongs in the config file rather than the environment |
|
||||
| `alert_command` | An alternative to ntfy: gets the title and body on stdin |
|
||||
| `guard_name` | This machine's name, for alerts to say where to run `dejima-guard report`. Optional; inside a container the hostname is a container ID, so it is never guessed |
|
||||
|
||||
`etc/config.example` carries the same reference with the reasoning attached.
|
||||
|
||||
@@ -406,25 +408,62 @@ minutes to notice the monitoring and scrub, and the resulting gap is evidence.
|
||||
fails, so a broken channel surfaces as a dead guard rather than as silence:
|
||||
and `test-alert` exists so you can settle it deliberately rather than find
|
||||
out.
|
||||
- **It sends no detail.** Alert bodies carry counts and categories, never a
|
||||
filename or a unit name. A compromised devbox controls those strings, and
|
||||
interpolating one into a push hands it a channel to your phone. The detail is
|
||||
in `dejima-guard report`, on this machine.
|
||||
- **It sends no paths.** Alert bodies say what kind of change and under which
|
||||
watched area ("1 file(s) modified in ~/.ssh/known_hosts"), worded by the
|
||||
guard from a fixed list, never a filename or a unit name. A compromised devbox
|
||||
controls those strings, and interpolating one into a push hands it a channel
|
||||
to your phone. The paths are in `dejima-guard report`, on this machine.
|
||||
|
||||
## Use
|
||||
|
||||
```sh
|
||||
dejima-guard check # what the timer runs
|
||||
dejima-guard report # the last cycle's detail
|
||||
dejima-guard accept # record the devbox's current state as the baseline
|
||||
dejima-guard status # baseline, last check, any expected-changes window
|
||||
dejima-guard report # the last cycle's detail: each path added, modified or removed
|
||||
dejima-guard accept # record the devbox's current state as the baseline, then check
|
||||
dejima-guard expect 30m # updating the devbox: hold alerts, then accept at the end
|
||||
dejima-guard test-alert # publish a test alert, then go and look
|
||||
dejima-guard trust-host # record the devbox's host key; run once, before check
|
||||
|
||||
dejima-guard-install-key # put the guard's key on a devbox, restricted
|
||||
```
|
||||
|
||||
`accept` is a decision, which is why a check never makes it. After an `apt
|
||||
upgrade` on the devbox, read the report, and if the changes are yours, accept.
|
||||
`accept` is a decision, which is why a check never makes it. After an
|
||||
unannounced change, read the report, and if the changes are yours, accept. The
|
||||
check it runs afterwards gets the heartbeat going again.
|
||||
|
||||
### Updating the devbox without being paged
|
||||
|
||||
Say so first, on the guard:
|
||||
|
||||
```sh
|
||||
dejima-guard expect 30m # or 2h, 90s; 24h at most
|
||||
```
|
||||
|
||||
Until then, changes and the devbox being unreachable (a reboot) are held
|
||||
rather than sent. When the window ends the guard accepts the devbox's state and
|
||||
sends a low-priority note saying what it accepted; `report --finding` has the
|
||||
paths. Finished early? `dejima-guard accept`. Called it off? `dejima-guard
|
||||
expect --cancel`.
|
||||
|
||||
**Critical paths and containment still alert**, and a window never accepts a
|
||||
critical path: those need an `accept` by hand. **It is opened on the guard and
|
||||
only there.** A devbox that could announce its own changes could silence its
|
||||
own guard.
|
||||
|
||||
### Changes not worth waking for
|
||||
|
||||
`ignore_paths` in the config, space-separated, each matched as a substring of
|
||||
the path like `critical_paths`:
|
||||
|
||||
```
|
||||
ignore_paths = /.ssh/known_hosts
|
||||
```
|
||||
|
||||
`~/.ssh/known_hosts` is the usual candidate. It changes whenever the devbox
|
||||
first connects to an SSH host, such as cloning from a new git server. Ignored
|
||||
changes still show in `report`, under their own heading, and a critical path
|
||||
cannot be ignored.
|
||||
|
||||
### In a container, every one of those needs a prefix
|
||||
|
||||
@@ -454,16 +493,19 @@ Either works; `exec` is cheaper and is what to reach for once the thing is up.
|
||||
the guard, so `run --rm dejima-guard accept` is the whole of it. Naming the
|
||||
binary again passes it as an argument to itself.
|
||||
|
||||
If the prefix gets tiresome, this belongs in your profile, pointed at wherever
|
||||
your compose file lives:
|
||||
**Or skip the prefix.** `bin/dejima-guard-docker` runs the guard in its
|
||||
container by name, from any directory. Install it on the container host as
|
||||
`dejima-guard` and every command above works as written:
|
||||
|
||||
```sh
|
||||
dejima-guard() {
|
||||
docker compose -f /srv/dejima-guard/docker-compose.yml \
|
||||
exec -T dejima-guard dejima-guard "$@"
|
||||
}
|
||||
sudo install -m 0755 bin/dejima-guard-docker /usr/local/bin/dejima-guard
|
||||
dejima-guard status
|
||||
```
|
||||
|
||||
It assumes the `container_name` the example compose file sets; set
|
||||
`DEJIMA_GUARD_CONTAINER` if yours differs. Do not install it beside a host
|
||||
install of the guard, which would be a second deployment with its own baseline.
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
|
||||
+7
-3
@@ -88,9 +88,13 @@ than discovering:
|
||||
- **It does not catch an intruder who anticipated it.** The devbox-side audit
|
||||
can be lied to by root on the devbox. This catches a payload that did not
|
||||
think to look, which is most of them.
|
||||
- **It sends no detail off this machine.** Alert bodies carry counts and
|
||||
categories, never a filename or a unit name, because the devbox chooses those
|
||||
strings. The detail stays in `dejima-guard report`.
|
||||
- **It sends no paths off this machine.** Alert bodies carry counts and the
|
||||
watched area a change falls under, printed from a fixed list in the guard,
|
||||
never a filename or a unit name, because the devbox chooses those strings.
|
||||
The detail stays in `dejima-guard report`.
|
||||
- **An expected-changes window is opened on the guard, never by the devbox.**
|
||||
One the devbox could open would let it silence its own guard. A window never
|
||||
holds or accepts a change to a critical path, or a containment failure.
|
||||
- **It treats what the devbox says as untrusted input**, including in that
|
||||
report: control characters are replaced as the text is collected, since a
|
||||
filename can carry an escape sequence and the report is read on the machine
|
||||
|
||||
+289
-31
@@ -369,21 +369,98 @@ LAST="$STATE_DIR/last-report"
|
||||
# is most likely to catch a clean cycle and see "no changes against the
|
||||
# baseline", which reads as the alert having been imaginary.
|
||||
FINDING="$STATE_DIR/last-finding"
|
||||
# The end of an expected-changes window, in epoch seconds. See cmd_expect.
|
||||
WINDOW="$STATE_DIR/expect-until"
|
||||
|
||||
# Categories rather than paths, because the body of an alert must carry nothing
|
||||
# the watched machine chose. A count of changed unit files is safe to send; the
|
||||
# name of one is not.
|
||||
summarise() { # summarise DIFF_FILE
|
||||
local f=$1 n_file n_unit n_port n_link
|
||||
n_file=$(grep -c '^[+-] FILE ' "$f" || true)
|
||||
n_unit=$(grep -c '^[+-] UNIT ' "$f" || true)
|
||||
n_port=$(grep -c '^[+-] PORT ' "$f" || true)
|
||||
n_link=$(grep -c '^[+-] LINK ' "$f" || true)
|
||||
[ "$n_file" -gt 0 ] && printf '%s file(s)\n' "$n_file"
|
||||
[ "$n_link" -gt 0 ] && printf '%s symlink(s)\n' "$n_link"
|
||||
[ "$n_unit" -gt 0 ] && printf '%s systemd unit(s)\n' "$n_unit"
|
||||
[ "$n_port" -gt 0 ] && printf '%s listening port(s)\n' "$n_port"
|
||||
return 0
|
||||
# Sorted here rather than trusted to arrive sorted, so a baseline this program
|
||||
# rewrote compares line for line with what the devbox sends next.
|
||||
statediff() { # statediff OLD NEW
|
||||
diff --unchanged-line-format= --old-line-format='- %L' --new-line-format='+ %L' \
|
||||
<(LC_ALL=C sort "$1") <(LC_ALL=C sort "$2") || true
|
||||
}
|
||||
|
||||
# One line per thing that changed: STATUS<TAB>KIND<TAB>KEY.
|
||||
#
|
||||
# An edited file is two diff lines, its old hash removed and its new one added,
|
||||
# which read as a deletion and a creation. Paired by what the line is about, an
|
||||
# edit says "modified".
|
||||
changes() { # changes DIFF_FILE
|
||||
awk '
|
||||
{
|
||||
sign = substr($0, 1, 1); kind = $2; key = $0
|
||||
sub(/^[+-] [A-Z]+ /, "", key)
|
||||
if (kind == "FILE") sub(/^[^ ]+ +/, "", key)
|
||||
else if (kind == "LINK") sub(/ -> .*/, "", key)
|
||||
id = kind SUBSEP key
|
||||
if (!(id in seen)) order[++n] = id
|
||||
seen[id] = seen[id] sign
|
||||
}
|
||||
END {
|
||||
for (i = 1; i <= n; i++) {
|
||||
split(order[i], p, SUBSEP); s = seen[order[i]]
|
||||
st = (s ~ /-/ && s ~ /\+/) ? "modified" : (s ~ /\+/ ? "added" : "removed")
|
||||
printf "%s\t%s\t%s\n", st, p[1], p[2]
|
||||
}
|
||||
}' "$1"
|
||||
}
|
||||
|
||||
changes_table() { # changes_table DIFF_FILE
|
||||
changes "$1" | awk -F '\t' '{ printf " %-9s %-5s %s\n", $1, $2, $3 }'
|
||||
}
|
||||
|
||||
# What changed, in words this program chose, for the body of an alert.
|
||||
#
|
||||
# The devbox names every path, so no path goes in a push. What goes instead is
|
||||
# the watched area it falls under, printed from the fixed list below and never
|
||||
# from the path itself: a home directory or filename an intruder chose cannot
|
||||
# ride along. A count of changed unit files is safe to send; the name of one is
|
||||
# not.
|
||||
describe() { # describe DIFF_FILE
|
||||
changes "$1" | awk -F '\t' '
|
||||
function area(p) {
|
||||
if (p ~ /^\/home\/[^\/]+\/\.ssh\/known_hosts/) return "~/.ssh/known_hosts"
|
||||
if (p ~ /^\/home\/[^\/]+\/\.ssh\//) return "~/.ssh"
|
||||
if (p ~ /^\/home\/[^\/]+\/\.config\/systemd\/user\//) return "~/.config/systemd/user"
|
||||
if (p ~ /^\/home\/[^\/]+\/\.local\/bin\//) return "~/.local/bin"
|
||||
if (p ~ /^\/home\/[^\/]+\/\.[^\/]+$/) return "shell startup files"
|
||||
if (p ~ /^\/root\/\.ssh\//) return "/root/.ssh"
|
||||
if (p ~ /^\/var\/spool\/cron\//) return "crontabs"
|
||||
if (p ~ /^\/etc\/apt\//) return "/etc/apt"
|
||||
if (p ~ /^\/etc\/systemd\//) return "/etc/systemd"
|
||||
if (p ~ /^\/etc\/ssh\//) return "/etc/ssh"
|
||||
if (p ~ /^\/etc\//) return "/etc"
|
||||
if (p ~ /^\/usr\/local\/bin\//) return "/usr/local/bin"
|
||||
if (p ~ /^\/usr\/local\/sbin\//) return "/usr/local/sbin"
|
||||
return "other paths"
|
||||
}
|
||||
{
|
||||
if ($2 == "UNIT") g = "unit(s) " ($1 == "added" ? "enabled" : "disabled")
|
||||
else if ($2 == "PORT") g = "listening port(s) " ($1 == "added" ? "opened" : "closed")
|
||||
else g = ($2 == "LINK" ? "symlink(s) " : "file(s) ") $1 " in " area($3)
|
||||
if (!(g in c)) order[++n] = g
|
||||
c[g]++
|
||||
}
|
||||
END {
|
||||
for (i = 1; i <= n && i <= 8; i++) printf "%s %s\n", c[order[i]], order[i]
|
||||
if (n > 8) printf "%s more kind(s) of change\n", n - 8
|
||||
}' | paste -sd ';' - | sed 's/;/, /g'
|
||||
}
|
||||
|
||||
# Said beside a change to known_hosts, which is the one most often harmless and
|
||||
# least obviously so.
|
||||
known_hosts_note() { # known_hosts_note DESCRIPTION
|
||||
case "$1" in
|
||||
*"~/.ssh/known_hosts"*)
|
||||
printf ' known_hosts changes when the devbox first connects to an SSH host it has not seen, such as cloning from a new git server.' ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Where to read the rest. guard_name comes from this machine's config, so it is
|
||||
# as safe to send as the devbox's own configured name.
|
||||
details_hint() {
|
||||
local g
|
||||
g=$(cfg guard_name)
|
||||
printf ' Details: %s report%s.' "$PROG" "${g:+, on $g}"
|
||||
}
|
||||
|
||||
# Paths whose change is a different kind of event from a config edit.
|
||||
@@ -413,6 +490,44 @@ critical_hits() { # critical_hits DIFF_FILE -> matching lines
|
||||
done <<< "$(cfg critical_paths "$CRITICAL_DEFAULT")"
|
||||
}
|
||||
|
||||
# Moves the lines an operator has said are not worth waking for out of the
|
||||
# diff. They stay in the report, under their own heading.
|
||||
#
|
||||
# A critical path is never ignored. Ignoring the file that keeps the guard's
|
||||
# access read-only would be a way to stop watching the guard's own footing.
|
||||
ignore_split() { # ignore_split DIFF_FILE IGNORED_FILE
|
||||
local pats=$2.pats crit=$2.crit
|
||||
cfg ignore_paths | tr -s '[:space:]' '\n' | grep . > "$pats" || true
|
||||
critical_hits "$1" > "$crit"
|
||||
grep -F -f "$pats" "$1" | grep -vxF -f "$crit" > "$2" || true
|
||||
grep -vxF -f "$2" "$1" > "$1.keep" || true
|
||||
mv "$1.keep" "$1"
|
||||
rm -f "$pats" "$crit"
|
||||
}
|
||||
|
||||
# The baseline an expected-changes window ends with: everything collected now,
|
||||
# except lines under a critical path, which keep their old value so the check
|
||||
# still alerts on them. Those are accepted by hand or not at all.
|
||||
accept_window() { # accept_window NOW_FILE
|
||||
local pats=$1.crit
|
||||
cfg critical_paths "$CRITICAL_DEFAULT" | grep . > "$pats" || true
|
||||
{ grep -vF -f "$pats" "$1"; grep -F -f "$pats" "$BASE"; } \
|
||||
| LC_ALL=C sort > "$BASE.new" || true
|
||||
rm -f "$pats"
|
||||
grep -q . "$BASE.new" || { rm -f "$BASE.new"; return 1; }
|
||||
mv "$BASE.new" "$BASE"
|
||||
}
|
||||
|
||||
window_end() {
|
||||
local e
|
||||
[ -r "$WINDOW" ] || return 0
|
||||
e=$(head -1 "$WINDOW")
|
||||
case "$e" in ''|*[!0-9]*) return 0 ;; esac
|
||||
printf '%s\n' "$e"
|
||||
}
|
||||
|
||||
clock() { date -d "@$1" +'%H:%M %Z'; }
|
||||
|
||||
# A listener or an enabled unit that was not there before is not a config edit.
|
||||
#
|
||||
# It is the shape of a thing arranging to run again, or to be reached. Both were
|
||||
@@ -487,11 +602,18 @@ require_config() {
|
||||
}
|
||||
|
||||
cmd_check() {
|
||||
local rc=0 now reach_rc reach_out changed=0 body=""
|
||||
local rc=0 now reach_rc reach_out changed=0 body="" held=0 wend window=none
|
||||
require_config
|
||||
install -d -m 0750 "$STATE_DIR"
|
||||
now=$(mktemp) || die "no temp space"
|
||||
|
||||
# An expected-changes window is open until its end, and closing once it has
|
||||
# passed. The check that finds it closing is the one that accepts.
|
||||
wend=$(window_end)
|
||||
if [ -n "$wend" ]; then
|
||||
if [ "$(date +%s)" -lt "$wend" ]; then window=open; else window=closing; fi
|
||||
fi
|
||||
|
||||
# 1. Can it be reached at all?
|
||||
#
|
||||
# Deliberately the first question. A devbox that does not answer is the
|
||||
@@ -499,7 +621,7 @@ cmd_check() {
|
||||
local ssh_err
|
||||
ssh_err=$(mktemp) || die "no temp space"
|
||||
# shellcheck disable=SC2064
|
||||
trap "rm -f '$now' '$now.diff' '$ssh_err'" EXIT
|
||||
trap "rm -f '$now' '$now.raw' '$now.diff' '$now.ignored' '$now.old' '$now.accepted' '$ssh_err'" EXIT
|
||||
|
||||
# ssh's own words are kept, not sent to /dev/null. They were, and the alert
|
||||
# offered three guesses instead - so the one failure that stops this program
|
||||
@@ -522,6 +644,20 @@ cmd_check() {
|
||||
|
||||
if [ "$crc" -ne 0 ]; then
|
||||
sanitise < "$ssh_err" > "$ssh_err.clean" && mv "$ssh_err.clean" "$ssh_err"
|
||||
# A reboot is part of most updates, so inside a window this is held
|
||||
# rather than sent. No heartbeat either way: nothing was checked.
|
||||
if [ "$window" = open ]; then
|
||||
say "HELD: could not reach $(target), inside the expected-changes window"
|
||||
say " until $(clock "$wend"). ssh said:"
|
||||
sed 's/^/ /' "$ssh_err" >&2
|
||||
return 1
|
||||
fi
|
||||
# A window ends at its end, reachable or not. Accepting whatever the box
|
||||
# says when it next answers, hours later, is not what was expected.
|
||||
if [ "$window" = closing ]; then
|
||||
rm -f "$WINDOW"
|
||||
say "The expected-changes window has ended with nothing to accept."
|
||||
fi
|
||||
alert max "$(target_short): unreachable" \
|
||||
"dejima-guard could not collect from $(target). It may be down, off the network, or refusing this key." || true
|
||||
say "FAIL: could not reach $(target). ssh said:"
|
||||
@@ -583,13 +719,27 @@ cmd_check() {
|
||||
return 1
|
||||
fi
|
||||
|
||||
diff --unchanged-line-format= --old-line-format='- %L' --new-line-format='+ %L' \
|
||||
"$BASE" "$now" > "$now.diff" || true
|
||||
statediff "$BASE" "$now" > "$now.diff"
|
||||
|
||||
local accepted=""
|
||||
if [ "$window" = closing ]; then
|
||||
cp "$BASE" "$now.old"
|
||||
accept_window "$now" || say "Nothing usable to accept as the window closed."
|
||||
rm -f "$WINDOW"
|
||||
statediff "$now.old" "$BASE" > "$now.accepted"
|
||||
statediff "$BASE" "$now" > "$now.diff"
|
||||
accepted=$(describe "$now.accepted")
|
||||
alert low "$(target_short): expected changes accepted" \
|
||||
"The expected-changes window on $(target) has closed. Accepted: ${accepted:-nothing had changed}.$(known_hosts_note "$accepted")" || true
|
||||
fi
|
||||
|
||||
ignore_split "$now.diff" "$now.ignored"
|
||||
|
||||
if [ -s "$now.diff" ]; then
|
||||
changed=1
|
||||
body=$(summarise "$now.diff" | paste -sd', ' -)
|
||||
body=$(describe "$now.diff")
|
||||
fi
|
||||
[ "$window" = open ] && [ "$changed" = 1 ] && held=1
|
||||
|
||||
# 4. One alert per cycle, batching every finding.
|
||||
#
|
||||
@@ -606,19 +756,21 @@ cmd_check() {
|
||||
# No path names in the body. The devbox chose those strings, and the
|
||||
# count plus the category is enough to say go and read the report.
|
||||
alert max "$(target_short): watched path changed" \
|
||||
"$crit_n line(s) changed under paths this guard treats as its own footing on $(target), including possibly the audit it collects with. Treat what it has reported since as unverified." || true
|
||||
"$crit_n line(s) changed under paths this guard treats as its own footing on $(target), including possibly the audit it collects with. Treat what it has reported since as unverified.$(details_hint)" || true
|
||||
rc=1
|
||||
elif [ "$changed" = 1 ] && [ "$reach_rc" -ne 0 ]; then
|
||||
alert max "$(target_short): changes and containment" \
|
||||
"On $(target): $body changed, and the containment check did not pass." || true
|
||||
"On $(target): $body, and the containment check did not pass.$(details_hint)" || true
|
||||
rc=1
|
||||
elif [ "$held" = 1 ]; then
|
||||
say "HELD: changes on $(target), inside the expected-changes window until $(clock "$wend")."
|
||||
elif [ "$newp" -gt 0 ]; then
|
||||
alert max "$(target_short): new listener or unit" \
|
||||
"On $(target): $body changed, and $newp of them is something newly arranged to run or to be reached." || true
|
||||
"On $(target): $body. $newp of them is something newly arranged to run or to be reached.$(details_hint)" || true
|
||||
rc=1
|
||||
elif [ "$changed" = 1 ]; then
|
||||
alert high "$(target_short): unexplained changes" \
|
||||
"On $(target): $body changed since the last accepted state." || true
|
||||
"On $(target), since the last accepted state: $body.$(known_hosts_note "$body")$(details_hint)" || true
|
||||
rc=1
|
||||
elif [ "$reach_rc" -ne 0 ]; then
|
||||
alert high "$(target_short): containment" \
|
||||
@@ -626,19 +778,32 @@ cmd_check() {
|
||||
rc=1
|
||||
fi
|
||||
|
||||
local result=clean
|
||||
[ "$rc" -eq 0 ] || result=alerted
|
||||
[ "$held" = 0 ] || result="held, expected-changes window open until $(clock "$wend")"
|
||||
{
|
||||
printf '%s\n' "--- collected $(date -u +'%Y-%m-%dT%H:%M:%SZ')"
|
||||
printf 'result: %s\n' "$result"
|
||||
printf 'reach exit: %s\n%s\n\n' "$reach_rc" "$reach_out"
|
||||
if [ -s "$now.accepted" ]; then
|
||||
printf 'accepted as the expected-changes window closed:\n'
|
||||
changes_table "$now.accepted"; printf '\n'
|
||||
fi
|
||||
if [ "$changed" = 1 ]; then
|
||||
printf 'changes against the baseline:\n'; cat "$now.diff"
|
||||
printf 'changes against the baseline:\n'; changes_table "$now.diff"
|
||||
printf '\nas collected:\n'; cat "$now.diff"
|
||||
else
|
||||
printf 'no changes against the baseline\n'
|
||||
fi
|
||||
if [ -s "$now.ignored" ]; then
|
||||
printf '\nignored (ignore_paths):\n'; changes_table "$now.ignored"
|
||||
fi
|
||||
} > "$LAST"
|
||||
|
||||
# Kept only when there was something to keep, so it is not overwritten by
|
||||
# the next quiet cycle. This is the copy an alert is telling you to read.
|
||||
[ "$rc" -eq 0 ] || cp "$LAST" "$FINDING"
|
||||
# the next quiet cycle. This is the copy an alert is telling you to read,
|
||||
# and a closing window's is the record of what it let through.
|
||||
if [ "$rc" -ne 0 ] || [ -s "$now.accepted" ]; then cp "$LAST" "$FINDING"; fi
|
||||
|
||||
if [ "$ALERT_FAILED" = 1 ]; then
|
||||
# No heartbeat, deliberately.
|
||||
@@ -655,7 +820,7 @@ cmd_check() {
|
||||
fi
|
||||
|
||||
if [ "$rc" -eq 0 ]; then
|
||||
say "CLEAN: no changes, containment passed."
|
||||
[ "$held" = 1 ] || say "CLEAN: no changes, containment passed."
|
||||
# Only on a cycle that completed and found nothing wrong. Pinging in a
|
||||
# failure path would tell the dead-man's switch that all is well at the
|
||||
# exact moment it is not.
|
||||
@@ -685,9 +850,87 @@ cmd_accept() {
|
||||
[ "$crc" -eq 0 ] || die "could not collect from the devbox"
|
||||
grep -q . "$raw" || die "the devbox returned nothing - not accepting that as a baseline"
|
||||
sanitise < "$raw" > "$BASE"
|
||||
rm -f "$raw" "$err" "$WINDOW"
|
||||
trap - EXIT
|
||||
say "Baseline accepted: $(grep -c . "$BASE") entries in $BASE"
|
||||
say " Take it on a box you believe is clean. One taken on a compromised"
|
||||
say " machine records the compromise as normal and reports clean forever."
|
||||
|
||||
# A check straight after, so the heartbeat resumes now rather than at the
|
||||
# next cycle, and the report reflects what was just accepted.
|
||||
say ""
|
||||
cmd_check
|
||||
}
|
||||
|
||||
# Says changes are coming, so an update does not page anyone.
|
||||
#
|
||||
# Opened HERE, on the guard, and never from the devbox. A devbox that could say
|
||||
# "expect changes" is a devbox that could silence its own guard.
|
||||
#
|
||||
# Until the window ends, changes and the devbox being unreachable are held
|
||||
# rather than sent. Critical paths and containment still alert, since a change
|
||||
# there is worth a push even during maintenance, and a window never accepts a
|
||||
# critical path. At the end the rest of the devbox's state is accepted, by a
|
||||
# check started now to run then rather than by whichever cycle of the timer
|
||||
# comes next. If that check is lost, to a container restart say, the first
|
||||
# check after the end does the same.
|
||||
cmd_expect() { # cmd_expect [DURATION|--cancel]
|
||||
local d=${1:-30m} n secs wend
|
||||
[ -f "$BASE" ] || die "no baseline yet - run $PROG accept first"
|
||||
if [ "$d" = --cancel ]; then
|
||||
rm -f "$WINDOW"
|
||||
say "Expected-changes window closed, nothing accepted. Changes will alert."
|
||||
return 0
|
||||
fi
|
||||
n=${d%[smh]}
|
||||
case "$n" in ''|*[!0-9]*) die "expect takes a duration like 30m, 2h or 90s - got '$d'" ;; esac
|
||||
case "$d" in
|
||||
*h) secs=$((n * 3600)) ;;
|
||||
*s) secs=$n ;;
|
||||
*) secs=$((n * 60)) ;;
|
||||
esac
|
||||
[ "$secs" -le 86400 ] || die "refusing a window over 24h: that is not watching"
|
||||
wend=$(( $(date +%s) + secs ))
|
||||
printf '%s\n' "$wend" > "$WINDOW"
|
||||
|
||||
# fd 9 is the lock, closed so the waiting check does not hold it while it
|
||||
# sleeps.
|
||||
setsid -f sh -c 'sleep "$1"; exec "$2" check' sh "$secs" "$0" \
|
||||
</dev/null >/dev/null 2>&1 9>&-
|
||||
|
||||
say "Expecting changes on $(target) until $(clock "$wend")."
|
||||
say " Changes and the devbox being unreachable are held until then. Critical"
|
||||
say " paths and containment still alert. At $(clock "$wend") the state is accepted"
|
||||
say " and a low-priority note says what changed."
|
||||
say ""
|
||||
say " Done early: $PROG accept"
|
||||
say " Call it off: $PROG expect --cancel"
|
||||
}
|
||||
|
||||
cmd_status() {
|
||||
local wend
|
||||
if [ -f "$BASE" ]; then
|
||||
printf 'baseline: accepted %s, %s entries\n' \
|
||||
"$(date -u -r "$BASE" +'%Y-%m-%dT%H:%M:%SZ')" "$(grep -c . "$BASE")"
|
||||
else
|
||||
printf 'baseline: none - run %s accept\n' "$PROG"
|
||||
fi
|
||||
if [ -r "$LAST" ]; then
|
||||
printf 'last check: %s, %s\n' "$(sed -n 's/^--- collected //p' "$LAST" | head -1)" \
|
||||
"$(sed -n 's/^result: //p' "$LAST" | head -1)"
|
||||
else
|
||||
printf 'last check: none\n'
|
||||
fi
|
||||
wend=$(window_end)
|
||||
if [ -n "$wend" ]; then
|
||||
printf 'window: expecting changes until %s\n' "$(clock "$wend")"
|
||||
else
|
||||
printf 'window: none\n'
|
||||
fi
|
||||
if [ -r "$FINDING" ]; then
|
||||
printf 'last finding: %s (%s report --finding)\n' \
|
||||
"$(sed -n 's/^--- collected //p' "$FINDING" | head -1)" "$PROG"
|
||||
fi
|
||||
}
|
||||
|
||||
# The alert path is the one thing here nothing else exercises.
|
||||
@@ -874,12 +1117,16 @@ cmd_trust_host() {
|
||||
|
||||
usage() {
|
||||
cat >&2 <<USAGE
|
||||
usage: $PROG [check|accept|report [--finding]|test-alert|trust-host]
|
||||
usage: $PROG [check|status|report [--finding]|accept|expect [30m]|test-alert|trust-host]
|
||||
|
||||
check collect from the devbox, diff against the baseline held here,
|
||||
alert on any change or on containment failing, and send the
|
||||
heartbeat only if the cycle completed clean and could alert
|
||||
accept record the devbox's current state as the baseline
|
||||
status the baseline, the last check, and any expected-changes window
|
||||
accept record the devbox's current state as the baseline, then check
|
||||
expect hold alerts for changes and for the devbox being unreachable,
|
||||
then accept: 30m by default, or 2h, 90s. --cancel ends it
|
||||
without accepting
|
||||
report the last cycle's detail, which is deliberately not in the alert.
|
||||
--finding shows the most recent cycle that found something, which
|
||||
a later clean cycle would otherwise have overwritten
|
||||
@@ -891,9 +1138,20 @@ usage: $PROG [check|accept|report [--finding]|test-alert|trust-host]
|
||||
USAGE
|
||||
}
|
||||
|
||||
# One run at a time on the state directory. The loop, an operator's accept and
|
||||
# the check a window left waiting would otherwise each read and write the
|
||||
# baseline under the others.
|
||||
lock() {
|
||||
install -d -m 0750 "$STATE_DIR"
|
||||
exec 9>"$STATE_DIR/lock"
|
||||
flock -w 300 9 || die "another $PROG run has held the lock for five minutes"
|
||||
}
|
||||
|
||||
case "${1:-check}" in
|
||||
check) cmd_check ;;
|
||||
accept) cmd_accept ;;
|
||||
check) lock; cmd_check ;;
|
||||
accept) lock; cmd_accept ;;
|
||||
expect) shift; lock; cmd_expect "$@" ;;
|
||||
status) cmd_status ;;
|
||||
report) shift; cmd_report "$@" ;;
|
||||
test-alert) cmd_test_alert ;;
|
||||
trust-host) shift; cmd_trust_host "$@" ;;
|
||||
|
||||
Executable
+19
@@ -0,0 +1,19 @@
|
||||
#!/bin/sh
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
# Copyright (C) 2026 Brad Anderson
|
||||
#
|
||||
# dejima-guard, run inside its container, from any directory.
|
||||
#
|
||||
# Install on the container host as `dejima-guard`, so the commands read the same
|
||||
# there as everywhere else:
|
||||
#
|
||||
# sudo install -m 0755 bin/dejima-guard-docker /usr/local/bin/dejima-guard
|
||||
#
|
||||
# `docker exec` by container name rather than `docker compose exec`, which only
|
||||
# works from the directory holding the compose file. The name is the
|
||||
# container_name the example compose file sets.
|
||||
|
||||
container=${DEJIMA_GUARD_CONTAINER:-dejima-guard}
|
||||
|
||||
if [ -t 0 ] && [ -t 1 ]; then tty=-it; else tty=-i; fi
|
||||
exec docker exec "$tty" "$container" dejima-guard "$@"
|
||||
@@ -20,6 +20,10 @@ services:
|
||||
build: .
|
||||
# image: gitbox.example.invalid/dejima/dejima-guard:latest
|
||||
container_name: dejima-guard
|
||||
# bin/dejima-guard-docker finds the container by this name. And an init,
|
||||
# because `dejima-guard expect` leaves a process waiting for the window's
|
||||
# end, which needs reaping once it is done.
|
||||
init: true
|
||||
restart: unless-stopped
|
||||
|
||||
environment:
|
||||
@@ -70,6 +74,9 @@ services:
|
||||
# Seconds between collections. Sets the dead-man's switch granularity
|
||||
# and how long an intruder has to notice the monitoring and scrub.
|
||||
DEJIMA_GUARD_INTERVAL: "900"
|
||||
# The name of the machine this container runs on, so an alert says
|
||||
# where to go and run `dejima-guard report`. Optional.
|
||||
# DEJIMA_GUARD_GUARD_NAME: your-docker-host
|
||||
# Anything secret goes in the config file rather than here. Everything
|
||||
# above is visible in `docker inspect` and to anything that can read
|
||||
# /proc; a mounted secret is not. The guard reads the environment first
|
||||
|
||||
@@ -69,6 +69,15 @@ ssh_known_hosts = /var/lib/dejima-guard/known_hosts
|
||||
# critical_paths = /usr/local/sbin/dejima-devbox-audit
|
||||
# /etc/sudoers.d/dejima-guard
|
||||
|
||||
# Paths whose change is reported but not alerted on, space-separated, each
|
||||
# matched as a substring of the path. None by default.
|
||||
#
|
||||
# ~/.ssh/known_hosts is the usual one: it changes whenever the devbox first
|
||||
# connects to an SSH host, such as cloning from a new git server. Ignored
|
||||
# changes still appear in `dejima-guard report`, and a critical path above is
|
||||
# never ignored, whatever this says.
|
||||
# ignore_paths = /.ssh/known_hosts
|
||||
|
||||
# Seconds to wait for the connection. A devbox that is slow is not a devbox
|
||||
# that is gone, and this is the line between those two readings.
|
||||
ssh_timeout = 10
|
||||
@@ -144,3 +153,11 @@ heartbeat_url = https://kuma.example.invalid/api/push/abcdef?status=up&msg=OK&pi
|
||||
# Set it and ntfy_url is ignored.
|
||||
#
|
||||
# alert_command = /usr/local/bin/my-alerter
|
||||
|
||||
# ===========================================================================
|
||||
# THIS MACHINE
|
||||
# ===========================================================================
|
||||
|
||||
# This machine's name, so an alert can say where to run `dejima-guard report`.
|
||||
# Optional. Never guessed: inside a container the hostname is a container ID.
|
||||
# guard_name = nas
|
||||
|
||||
+214
@@ -1382,3 +1382,217 @@ PORT tcp 0.0.0.0:4444" 0
|
||||
run_watch report
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
# --- what changed, in words the guard chose ------------------------------------
|
||||
|
||||
@test "the alert says where a change is and what kind, never the path" {
|
||||
stub_ssh "FILE aaa /home/intruder-chosen/.ssh/known_hosts" 0
|
||||
run_watch accept
|
||||
curl_reset
|
||||
stub_ssh "FILE bbb /home/intruder-chosen/.ssh/known_hosts
|
||||
FILE ccc /etc/apt/sources.list.d/new.list" 0
|
||||
run_watch check
|
||||
[[ "$(curl_log)" == *"1 file(s) modified in ~/.ssh/known_hosts"* ]]
|
||||
[[ "$(curl_log)" == *"1 file(s) added in /etc/apt"* ]]
|
||||
[[ "$(curl_log)" == *"first connects to an SSH host"* ]]
|
||||
[[ "$(curl_log)" != *intruder-chosen* ]]
|
||||
[[ "$(curl_log)" != *new.list* ]]
|
||||
}
|
||||
|
||||
@test "the report pairs an edit into one modified line" {
|
||||
stub_ssh "FILE aaa /etc/one
|
||||
FILE ddd /etc/gone" 0
|
||||
run_watch accept
|
||||
stub_ssh "FILE bbb /etc/one
|
||||
FILE ccc /etc/two" 0
|
||||
run_watch check
|
||||
run_watch report
|
||||
[[ "$output" == *"modified FILE /etc/one"* ]]
|
||||
[[ "$output" == *"added FILE /etc/two"* ]]
|
||||
[[ "$output" == *"removed FILE /etc/gone"* ]]
|
||||
}
|
||||
|
||||
@test "the alert names the guard and the command to run, when configured" {
|
||||
export DEJIMA_GUARD_GUARD_NAME=nas
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
stub_ssh "FILE bbb /etc/one" 0
|
||||
run_watch check
|
||||
[[ "$(curl_log)" == *"Details: dejima-guard report, on nas."* ]]
|
||||
}
|
||||
|
||||
# --- ignore_paths ----------------------------------------------------------------
|
||||
|
||||
@test "an ignored path does not alert, and the cycle is clean" {
|
||||
export DEJIMA_GUARD_IGNORE_PATHS='/.ssh/known_hosts'
|
||||
stub_ssh "FILE aaa /home/u/.ssh/known_hosts" 0
|
||||
run_watch accept
|
||||
curl_reset
|
||||
stub_ssh "FILE bbb /home/u/.ssh/known_hosts" 0
|
||||
run_watch check
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(alerts)" = 0 ]
|
||||
[ "$(heartbeats)" = 1 ]
|
||||
run_watch report
|
||||
[[ "$output" == *"ignored (ignore_paths):"* ]]
|
||||
[[ "$output" == *"/home/u/.ssh/known_hosts"* ]]
|
||||
}
|
||||
|
||||
@test "ignore_paths cannot hide a critical path" {
|
||||
export DEJIMA_GUARD_IGNORE_PATHS='/.ssh/'
|
||||
stub_ssh "FILE aaa /home/u/.ssh/authorized_keys" 0
|
||||
run_watch accept
|
||||
stub_ssh "FILE bbb /home/u/.ssh/authorized_keys" 0
|
||||
run_watch check
|
||||
[ "$status" -ne 0 ]
|
||||
[[ "$(curl_log)" == *"watched path changed"* ]]
|
||||
}
|
||||
|
||||
# --- expected-changes windows -------------------------------------------------
|
||||
|
||||
open_window() { printf '%s\n' "$(( $(date +%s) + $1 ))" > "$DEJIMA_GUARD_STATE/expect-until"; }
|
||||
|
||||
@test "inside a window a change is held, and the heartbeat still goes" {
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
curl_reset
|
||||
open_window 600
|
||||
stub_ssh "FILE bbb /etc/one" 0
|
||||
run_watch check
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(alerts)" = 0 ]
|
||||
[ "$(heartbeats)" = 1 ]
|
||||
run_watch report
|
||||
[[ "$output" == *"result: held"* ]]
|
||||
}
|
||||
|
||||
@test "inside a window an unreachable devbox is held, without a heartbeat" {
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
curl_reset
|
||||
open_window 600
|
||||
stub_ssh "" 0 255
|
||||
run_watch check
|
||||
[ "$status" -ne 0 ]
|
||||
[ "$(alerts)" = 0 ]
|
||||
[ "$(heartbeats)" = 0 ]
|
||||
}
|
||||
|
||||
@test "inside a window a critical path and containment still alert" {
|
||||
stub_ssh "FILE aaa /etc/ssh/sshd_config" 0
|
||||
run_watch accept
|
||||
curl_reset
|
||||
open_window 600
|
||||
stub_ssh "FILE bbb /etc/ssh/sshd_config" 0
|
||||
run_watch check
|
||||
[[ "$(curl_log)" == *"watched path changed"* ]]
|
||||
curl_reset
|
||||
stub_ssh "FILE aaa /etc/ssh/sshd_config" 1
|
||||
run_watch check
|
||||
[[ "$(curl_log)" == *"containment"* ]]
|
||||
}
|
||||
|
||||
@test "a window that has ended accepts, except critical paths" {
|
||||
stub_ssh "FILE aaa /etc/one
|
||||
FILE aaa /etc/ssh/sshd_config" 0
|
||||
run_watch accept
|
||||
curl_reset
|
||||
open_window -1
|
||||
stub_ssh "FILE bbb /etc/one
|
||||
FILE bbb /etc/ssh/sshd_config" 0
|
||||
run_watch check
|
||||
[ ! -f "$DEJIMA_GUARD_STATE/expect-until" ]
|
||||
grep -q 'FILE bbb /etc/one' "$DEJIMA_GUARD_STATE/baseline"
|
||||
grep -q 'FILE aaa /etc/ssh/sshd_config' "$DEJIMA_GUARD_STATE/baseline"
|
||||
[[ "$(curl_log)" == *"expected changes accepted"* ]]
|
||||
[[ "$(curl_log)" == *"Priority: low"* ]]
|
||||
[[ "$(curl_log)" == *"watched path changed"* ]]
|
||||
run_watch report --finding
|
||||
[[ "$output" == *"accepted as the expected-changes window closed"* ]]
|
||||
}
|
||||
|
||||
@test "a window that ends with the devbox unreachable accepts nothing" {
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
curl_reset
|
||||
open_window -1
|
||||
stub_ssh "" 0 255
|
||||
run_watch check
|
||||
[ ! -f "$DEJIMA_GUARD_STATE/expect-until" ]
|
||||
[[ "$(curl_log)" == *"unreachable"* ]]
|
||||
stub_ssh "FILE bbb /etc/one" 0
|
||||
run_watch check
|
||||
[[ "$(curl_log)" == *"unexplained changes"* ]]
|
||||
}
|
||||
|
||||
@test "expect accepts at the end of the window, not at the next cycle" {
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
run_watch expect 1s
|
||||
[ "$status" -eq 0 ]
|
||||
stub_ssh "FILE bbb /etc/one" 0
|
||||
for _ in 1 2 3 4 5 6 7 8 9 10; do
|
||||
grep -q 'FILE bbb' "$DEJIMA_GUARD_STATE/baseline" && break
|
||||
sleep 1
|
||||
done
|
||||
grep -q 'FILE bbb /etc/one' "$DEJIMA_GUARD_STATE/baseline"
|
||||
[ ! -f "$DEJIMA_GUARD_STATE/expect-until" ]
|
||||
}
|
||||
|
||||
@test "expect refuses a nonsense duration, an endless one, and no baseline" {
|
||||
run_watch expect 30m
|
||||
[ "$status" -ne 0 ]
|
||||
[[ "$output" == *"no baseline"* ]]
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
run_watch expect soon
|
||||
[ "$status" -ne 0 ]
|
||||
run_watch expect 25h
|
||||
[ "$status" -ne 0 ]
|
||||
[ ! -f "$DEJIMA_GUARD_STATE/expect-until" ]
|
||||
}
|
||||
|
||||
@test "accept closes a window and checks, so the heartbeat resumes" {
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
open_window 600
|
||||
curl_reset
|
||||
run_watch accept
|
||||
[ "$status" -eq 0 ]
|
||||
[ ! -f "$DEJIMA_GUARD_STATE/expect-until" ]
|
||||
[ "$(heartbeats)" = 1 ]
|
||||
}
|
||||
|
||||
@test "expect --cancel closes a window without accepting" {
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
open_window 600
|
||||
run_watch expect --cancel
|
||||
[ ! -f "$DEJIMA_GUARD_STATE/expect-until" ]
|
||||
stub_ssh "FILE bbb /etc/one" 0
|
||||
run_watch check
|
||||
[[ "$(curl_log)" == *"unexplained changes"* ]]
|
||||
}
|
||||
|
||||
@test "status shows the baseline, the last check and the window" {
|
||||
stub_ssh "FILE aaa /etc/one" 0
|
||||
run_watch accept
|
||||
open_window 600
|
||||
run_watch status
|
||||
[[ "$output" == *"1 entries"* ]]
|
||||
[[ "$output" == *"last check:"*"clean"* ]]
|
||||
[[ "$output" == *"expecting changes until"* ]]
|
||||
}
|
||||
|
||||
# --- the container host's wrapper --------------------------------------------
|
||||
|
||||
@test "the docker wrapper runs the guard in its container, from anywhere" {
|
||||
cat > "$STUB/docker" <<EOF
|
||||
#!/bin/sh
|
||||
printf '%s\n' "\$*" > "$BATS_TEST_TMPDIR/docker.args"
|
||||
EOF
|
||||
chmod +x "$STUB/docker"
|
||||
cd "$BATS_TEST_TMPDIR"
|
||||
PATH="$STUB:$PATH" run "$REPO/bin/dejima-guard-docker" accept < /dev/null
|
||||
[ "$(cat "$BATS_TEST_TMPDIR/docker.args")" = "exec -i dejima-guard dejima-guard accept" ]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user