Merge pull request 'Add upstream sync process and agent notes' (#13) from upstream-sync-process into main

Reviewed-on: #13
Reviewed-by: branderson <brad@branderson.io>

.dejima.md

@@ -0,0 +1,43 @@
+# Dotcom Arcade: notes for agents
+
+Dotcom Arcade is a heavily modified fork of
+[flashpoint-web](https://github.com/darkraise/flashpoint-web) by darkraise
+(GPL-3.0), with game streaming added under `streaming/`. Coding standards and
+architecture notes are in `CLAUDE.md`; feature and deployment docs are in
+`docs/`.
+
+## Upstream
+
+- **Upstream** is https://github.com/darkraise/flashpoint-web, branch `master`.
+  It's read-only from here: don't push to it or open pull requests there unless
+  asked.
+- **Most of upstream's code is still used:** the backend, the frontend's data
+  layer, the player, and the account and admin pages. Fixes to these should come
+  in from upstream rather than being rewritten here.
+- **To see what upstream has changed** since it was last merged, and how each
+  touched file stands here, run `scripts/upstream-status.sh`.
+- **To pull changes in**, follow `docs/08-development/upstream-sync.md`. In
+  short:
+  - merge upstream's `master` on a branch (don't cherry-pick);
+  - keep files removed here removed;
+  - keep the Dotcom Arcade branding;
+  - keep upstream's migration file names;
+  - have the pull request merged with a merge commit, never squashed, so the
+    next sync starts from the right place.
+- **Parts that are only ours:**
+  - `streaming/`;
+  - `scripts/`;
+  - `frontend/src/components/portal/`;
+  - `views/GamePage.tsx`;
+  - `lib/saves/flashSaves.ts`;
+  - the discovery backend;
+  - `.gitea/`;
+  - migrations upstream doesn't have.
+
+## CI
+
+Workflows are in `.gitea/workflows`. Jobs that run docker builds use the
+`dejima-privileged` runner label; anything else uses `dejima-confined`. The
+privileged job image has no Docker client, so install `docker-ce-cli` first.
+Never hardcode the forge's host name: read it at run time, or take it from the
+`REGISTRY_HOST` repository variable.

docs/08-development/README.md

@@ -15,6 +15,7 @@ the Dotcom Arcade application.
 5. **[Testing Guide](./testing-guide.md)** - Testing approach and best practices
 6. **[Debugging Guide](./debugging.md)** - Debugging tips and troubleshooting
 7. **[Common Pitfalls](./common-pitfalls.md)** - Known issues and solutions
+8. **[Upstream Sync](./upstream-sync.md)** - Merging changes from flashpoint-web
 
 ## Quick Start
 

docs/08-development/upstream-sync.md

@@ -0,0 +1,99 @@
+# Syncing with upstream flashpoint-web
+
+Dotcom Arcade is a heavily modified fork of
+[flashpoint-web](https://github.com/darkraise/flashpoint-web) by darkraise
+(branch `master`). Most of upstream's code is still used here: the backend, the
+frontend's data layer, the player, and the account and admin pages. Upstream's
+fixes and features to those parts should keep coming in, so the two projects
+don't drift apart.
+
+## What's whose
+
+Comparing this repository with the last upstream commit merged into it:
+
+- **Unchanged here:** most of the backend, the API client, hooks, stores and
+  types, the player, the account and admin views, and most docs. Upstream
+  changes to these merge as they are.
+- **Changed here:** files upstream also owns that this project has modified.
+  The main ones:
+  - `GameService` (mature filter, release ranges, shuffle, related games);
+  - `server.ts` and `config.ts` (stream proxy, update check);
+  - `App.tsx` (portal routes);
+  - `GamePlayer` (streaming, saves);
+  - the browse layout and filters;
+  - the rebrand across names and docs.
+
+  Upstream changes to these need merging by hand.
+- **Removed here:**
+  - the Flash and HTML5 sections;
+  - the GitHub stars endpoint;
+  - the old homepage pieces;
+  - the dev compose file;
+  - upstream's CI workflow and design notes.
+
+  These stay removed.
+- **Only here:**
+  - `streaming/`;
+  - `scripts/`;
+  - `frontend/src/components/portal/`;
+  - `views/GamePage.tsx`;
+  - `lib/saves/flashSaves.ts`;
+  - `.gitea/`;
+  - the discovery backend;
+  - migrations upstream doesn't have.
+
+  Upstream never touches these, but a change elsewhere can still need matching
+  work here.
+
+`scripts/upstream-status.sh` lists upstream's new commits and puts every file
+they touch in one of these groups, so this list doesn't need keeping up to date
+by hand.
+
+## When
+
+Run `scripts/upstream-status.sh` when upstream publishes a release, and at
+least monthly. Sync when it shows new commits.
+
+## How
+
+1. On a branch from `main`, merge upstream:
+
+   ```sh
+   scripts/upstream-status.sh
+   git checkout -b upstream-sync-$(date +%Y-%m-%d) origin/main
+   git merge refs/remotes/upstream/master
+   ```
+
+2. Resolve conflicts by group:
+   - **Changed here:** keep this project's behavior and take upstream's fix or
+     feature with it. Where they truly conflict, prefer this project's
+     behavior, and note what was left out in the pull request.
+   - **Removed here:** keep the file deleted
+     (`git rm <file>`). If upstream's change matters elsewhere, such as a bug
+     fix in shared logic, port that part to where the code lives now.
+   - **New upstream:** take it, unless it belongs to something removed here.
+     New UI usually lands in the old application layout. Decide whether it
+     belongs in the portal, and if so, follow up with a portal version.
+   - **Branding:** keep "Dotcom Arcade", our image and container names, and our
+     update check (`APP_UPDATE_URL`); don't take back upstream's.
+3. Migrations: keep upstream's migration files under their own names, even
+   when a number clashes with ours. The migrations table records full names,
+   so `009_app_tagline` and an upstream `009_something` both run, in name
+   order. Renaming an upstream migration would run it a second time on
+   databases that already applied it. Check that ours and theirs don't depend
+   on each other's order.
+4. Check the result:
+   - backend and frontend typecheck, tests and build;
+   - the session manager's tests if `streaming/` was affected;
+   - `docker compose up -d --build` with and without the `streaming` profile;
+   - the docs, for anything the merge made inaccurate.
+5. Open a pull request listing upstream's commits and how each conflict was
+   resolved. **Merge it with a merge commit, not squash or rebase.** The merge
+   records how far upstream has been synced, so the next run of
+   `scripts/upstream-status.sh` starts from the right place.
+
+## Sending changes upstream
+
+Fixes made here to code upstream still shares, such as backend bugs, may be
+worth offering upstream. Do that separately, from a branch based on upstream's
+`master`, and only when the project's maintainer decides to.

docs/README.md

@@ -125,6 +125,7 @@ for Flashpoint Ultimate.
 - [Testing Guide](08-development/testing-guide.md) - Testing approach
 - [Debugging](08-development/debugging.md) - Debugging tips
 - [Common Pitfalls](08-development/common-pitfalls.md) - Known issues
+- [Upstream Sync](08-development/upstream-sync.md) - Merging changes from flashpoint-web
 
 ### 09. Deployment
 

scripts/upstream-status.sh

@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+# Shows what upstream flashpoint-web has changed since it was last merged, and
+# how this project stands on each file it touches. Changes nothing but the
+# refs/remotes/upstream/master ref. See docs/08-development/upstream-sync.md.
+set -euo pipefail
+
+URL=${UPSTREAM_URL:-https://github.com/darkraise/flashpoint-web.git}
+BRANCH=master
+UPSTREAM=refs/remotes/upstream/$BRANCH
+
+cd "$(git rev-parse --show-toplevel)"
+git fetch -q "$URL" "+$BRANCH:$UPSTREAM"
+base=$(git merge-base HEAD "$UPSTREAM")
+count=$(git rev-list --count "$base..$UPSTREAM")
+
+echo "Last merged from upstream: $(git log -1 --format='%h %ad %s' --date=short "$base")"
+if ((count == 0)); then
+  echo "Up to date with upstream $BRANCH."
+  exit 0
+fi
+
+echo "$count new upstream commit(s):"
+git log --format='  %h %ad %s' --date=short "$base..$UPSTREAM"
+echo
+echo "Files they change, by where this project stands on each:"
+echo "  unchanged here: merges as is   changed here: merge both sides"
+echo "  removed here: keep it removed   new upstream: take it, unless it belongs to a removed area"
+echo
+git diff --name-only "$base" "$UPSTREAM" | while IFS= read -r file; do
+  if ! git cat-file -e "HEAD:$file" 2>/dev/null; then
+    if git cat-file -e "$base:$file" 2>/dev/null; then
+      echo "removed here    $file"
+    else
+      echo "new upstream    $file"
+    fi
+  elif git diff --quiet "$base" HEAD -- "$file"; then
+    echo "unchanged here  $file"
+  else
+    echo "changed here    $file"
+  fi
+done | sort