docs: don't pin our migration numbers in sync notes

Adds scripts/upstream-status.sh to report upstream flashpoint-web changes
since the last merge, docs/08-development/upstream-sync.md describing the
merge process, and .dejima.md for agents working in dejima.

.dejima.md

@@ -0,0 +1,43 @@
+# Dotcom Arcade: notes for agents
+
+Dotcom Arcade is a heavily modified fork of
+[flashpoint-web](https://github.com/darkraise/flashpoint-web) by darkraise
+(GPL-3.0), with game streaming added under `streaming/`. Coding standards and
+architecture notes are in `CLAUDE.md`; feature and deployment docs are in
+`docs/`.
+
+## Upstream
+
+- **Upstream** is https://github.com/darkraise/flashpoint-web, branch `master`.
+  It's read-only from here: don't push to it or open pull requests there unless
+  asked.
+- **Most of upstream's code is still used:** the backend, the frontend's data
+  layer, the player, and the account and admin pages. Fixes to these should come
+  in from upstream rather than being rewritten here.
+- **To see what upstream has changed** since it was last merged, and how each
+  touched file stands here, run `scripts/upstream-status.sh`.
+- **To pull changes in**, follow `docs/08-development/upstream-sync.md`. In
+  short:
+  - merge upstream's `master` on a branch (don't cherry-pick);
+  - keep files removed here removed;
+  - keep the Dotcom Arcade branding;
+  - keep upstream's migration file names;
+  - have the pull request merged with a merge commit, never squashed, so the
+    next sync starts from the right place.
+- **Parts that are only ours:**
+  - `streaming/`;
+  - `scripts/`;
+  - `frontend/src/components/portal/`;
+  - `views/GamePage.tsx`;
+  - `lib/saves/flashSaves.ts`;
+  - the discovery backend;
+  - `.gitea/`;
+  - migrations upstream doesn't have.
+
+## CI
+
+Workflows are in `.gitea/workflows`. Jobs that run docker builds use the
+`dejima-privileged` runner label; anything else uses `dejima-confined`. The
+privileged job image has no Docker client, so install `docker-ce-cli` first.
+Never hardcode the forge's host name: read it at run time, or take it from the
+`REGISTRY_HOST` repository variable.

README.md

@@ -192,6 +192,11 @@ and development. Useful starting points:
 - [Ruffle](https://ruffle.rs/), for running Flash in modern browsers.
 - [Selkies](https://github.com/selkies-project/selkies), which streams the
   games.
+- [Texgriddy](https://github.com/Weestuarty/texgriddy-es-de) by Weestuarty, an
+  ES-DE theme that inspired the site's look. The look was recreated from scratch
+  in CSS; none of Texgriddy's files, images or fonts are used.
+- [Fredoka](https://fontsource.org/fonts/fredoka) (SIL Open Font License), the
+  display font.
 
 ## Legal
 

docs/08-development/README.md

@@ -15,6 +15,7 @@ the Dotcom Arcade application.
 5. **[Testing Guide](./testing-guide.md)** - Testing approach and best practices
 6. **[Debugging Guide](./debugging.md)** - Debugging tips and troubleshooting
 7. **[Common Pitfalls](./common-pitfalls.md)** - Known issues and solutions
+8. **[Upstream Sync](./upstream-sync.md)** - Merging changes from flashpoint-web
 
 ## Quick Start
 

docs/08-development/upstream-sync.md

@@ -0,0 +1,99 @@
+# Syncing with upstream flashpoint-web
+
+Dotcom Arcade is a heavily modified fork of
+[flashpoint-web](https://github.com/darkraise/flashpoint-web) by darkraise
+(branch `master`). Most of upstream's code is still used here: the backend, the
+frontend's data layer, the player, and the account and admin pages. Upstream's
+fixes and features to those parts should keep coming in, so the two projects
+don't drift apart.
+
+## What's whose
+
+Comparing this repository with the last upstream commit merged into it:
+
+- **Unchanged here:** most of the backend, the API client, hooks, stores and
+  types, the player, the account and admin views, and most docs. Upstream
+  changes to these merge as they are.
+- **Changed here:** files upstream also owns that this project has modified.
+  The main ones:
+  - `GameService` (mature filter, release ranges, shuffle, related games);
+  - `server.ts` and `config.ts` (stream proxy, update check);
+  - `App.tsx` (portal routes);
+  - `GamePlayer` (streaming, saves);
+  - the browse layout and filters;
+  - the rebrand across names and docs.
+
+  Upstream changes to these need merging by hand.
+- **Removed here:**
+  - the Flash and HTML5 sections;
+  - the GitHub stars endpoint;
+  - the old homepage pieces;
+  - the dev compose file;
+  - upstream's CI workflow and design notes.
+
+  These stay removed.
+- **Only here:**
+  - `streaming/`;
+  - `scripts/`;
+  - `frontend/src/components/portal/`;
+  - `views/GamePage.tsx`;
+  - `lib/saves/flashSaves.ts`;
+  - `.gitea/`;
+  - the discovery backend;
+  - migrations upstream doesn't have.
+
+  Upstream never touches these, but a change elsewhere can still need matching
+  work here.
+
+`scripts/upstream-status.sh` lists upstream's new commits and puts every file
+they touch in one of these groups, so this list doesn't need keeping up to date
+by hand.
+
+## When
+
+Run `scripts/upstream-status.sh` when upstream publishes a release, and at
+least monthly. Sync when it shows new commits.
+
+## How
+
+1. On a branch from `main`, merge upstream:
+
+   ```sh
+   scripts/upstream-status.sh
+   git checkout -b upstream-sync-$(date +%Y-%m-%d) origin/main
+   git merge refs/remotes/upstream/master
+   ```
+
+2. Resolve conflicts by group:
+   - **Changed here:** keep this project's behavior and take upstream's fix or
+     feature with it. Where they truly conflict, prefer this project's
+     behavior, and note what was left out in the pull request.
+   - **Removed here:** keep the file deleted
+     (`git rm <file>`). If upstream's change matters elsewhere, such as a bug
+     fix in shared logic, port that part to where the code lives now.
+   - **New upstream:** take it, unless it belongs to something removed here.
+     New UI usually lands in the old application layout. Decide whether it
+     belongs in the portal, and if so, follow up with a portal version.
+   - **Branding:** keep "Dotcom Arcade", our image and container names, and our
+     update check (`APP_UPDATE_URL`); don't take back upstream's.
+3. Migrations: keep upstream's migration files under their own names, even
+   when a number clashes with ours. The migrations table records full names,
+   so `009_app_tagline` and an upstream `009_something` both run, in name
+   order. Renaming an upstream migration would run it a second time on
+   databases that already applied it. Check that ours and theirs don't depend
+   on each other's order.
+4. Check the result:
+   - backend and frontend typecheck, tests and build;
+   - the session manager's tests if `streaming/` was affected;
+   - `docker compose up -d --build` with and without the `streaming` profile;
+   - the docs, for anything the merge made inaccurate.
+5. Open a pull request listing upstream's commits and how each conflict was
+   resolved. **Merge it with a merge commit, not squash or rebase.** The merge
+   records how far upstream has been synced, so the next run of
+   `scripts/upstream-status.sh` starts from the right place.
+
+## Sending changes upstream
+
+Fixes made here to code upstream still shares, such as backend bugs, may be
+worth offering upstream. Do that separately, from a branch based on upstream's
+`master`, and only when the project's maintainer decides to.

docs/10-features/12-discovery-portal.md

@@ -30,9 +30,15 @@ tokens are applied to `<body>` while the portal is mounted, which takes
 precedence over the app theme on `<html>` and also reaches dialogs and menus
 rendered into the body.
 
-One scheme ships, **Chilled** (slate background, sage boxes). Adding a scheme
+One scheme ships, **Slate** (slate background, sage boxes). Adding a scheme
 means adding an entry to `PORTAL_SCHEMES`; no component changes are needed.
 
+The portal's look is inspired by
+[Texgriddy](https://github.com/Weestuarty/texgriddy-es-de), an ES-DE theme by
+Weestuarty: the framed page, glossy tiles with a metallic ring, outlined info
+boxes, capsule buttons and the slate and sage palette. It was recreated from
+scratch in CSS; none of Texgriddy's files, images or fonts are used.
+
 Display text uses Fredoka (OFL, bundled via `@fontsource-variable/fredoka`);
 body text uses Verdana.
 

docs/README.md

@@ -125,6 +125,7 @@ for Flashpoint Ultimate.
 - [Testing Guide](08-development/testing-guide.md) - Testing approach
 - [Debugging](08-development/debugging.md) - Debugging tips
 - [Common Pitfalls](08-development/common-pitfalls.md) - Known issues
+- [Upstream Sync](08-development/upstream-sync.md) - Merging changes from flashpoint-web
 
 ### 09. Deployment
 

frontend/src/components/portal/PortalLayout.tsx

@@ -25,7 +25,7 @@ export function PortalLayout() {
   const searchRef = useRef<HTMLInputElement>(null);
   const [rolling, setRolling] = useState(false);
 
-  useEffect(() => applyPortalScheme('chilled'), []);
+  useEffect(() => applyPortalScheme('slate'), []);
   useMatureSync();
 
   const playRandom = async () => {

frontend/src/components/portal/schemes.ts

@@ -8,8 +8,8 @@ export interface PortalScheme {
 }
 
 export const PORTAL_SCHEMES = {
-  chilled: {
-    label: 'Chilled',
+  slate: {
+    label: 'Slate',
     tokens: {
       '--p-bg': '#4c6878',
       '--p-bg-mix': '#2f4553',

scripts/upstream-status.sh

@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+# Shows what upstream flashpoint-web has changed since it was last merged, and
+# how this project stands on each file it touches. Changes nothing but the
+# refs/remotes/upstream/master ref. See docs/08-development/upstream-sync.md.
+set -euo pipefail
+
+URL=${UPSTREAM_URL:-https://github.com/darkraise/flashpoint-web.git}
+BRANCH=master
+UPSTREAM=refs/remotes/upstream/$BRANCH
+
+cd "$(git rev-parse --show-toplevel)"
+git fetch -q "$URL" "+$BRANCH:$UPSTREAM"
+base=$(git merge-base HEAD "$UPSTREAM")
+count=$(git rev-list --count "$base..$UPSTREAM")
+
+echo "Last merged from upstream: $(git log -1 --format='%h %ad %s' --date=short "$base")"
+if ((count == 0)); then
+  echo "Up to date with upstream $BRANCH."
+  exit 0
+fi
+
+echo "$count new upstream commit(s):"
+git log --format='  %h %ad %s' --date=short "$base..$UPSTREAM"
+echo
+echo "Files they change, by where this project stands on each:"
+echo "  unchanged here: merges as is   changed here: merge both sides"
+echo "  removed here: keep it removed   new upstream: take it, unless it belongs to a removed area"
+echo
+git diff --name-only "$base" "$UPSTREAM" | while IFS= read -r file; do
+  if ! git cat-file -e "HEAD:$file" 2>/dev/null; then
+    if git cat-file -e "$base:$file" 2>/dev/null; then
+      echo "removed here    $file"
+    else
+      echo "new upstream    $file"
+    fi
+  elif git diff --quiet "$base" HEAD -- "$file"; then
+    echo "unchanged here  $file"
+  else
+    echo "changed here    $file"
+  fi
+done | sort

streaming/session-manager/src/docker.ts

@@ -46,8 +46,9 @@ export class DockerDriver implements Driver {
         `PUID=${uid}`,
         `PGID=${gid}`,
         `SUBFOLDER=${streamPath(id)}`,
-        `SELKIES_MANUAL_WIDTH=${size.width}`,
-        `SELKIES_MANUAL_HEIGHT=${size.height}`,
+        // The game's size, rounded up to even for H.264
+        `SELKIES_MANUAL_WIDTH=${size.width + (size.width % 2)}`,
+        `SELKIES_MANUAL_HEIGHT=${size.height + (size.height % 2)}`,
         'SELKIES_UI_SHOW_SIDEBAR=false',
         'HARDEN_DESKTOP=true',
         'RESTART_APP=true',

streaming/session-manager/src/games.ts

@@ -59,7 +59,7 @@ export class GameCatalog {
         return null;
       }
       const page = await this.readContent(gameId, row, row.cmd).catch(() => null);
-      return even((page && appletSize(page.toString('latin1'))) ?? DEFAULT_SIZE);
+      return ((page && appletSize(page.toString('latin1'))) ?? DEFAULT_SIZE);
     }
     const program = await this.launchProgram(row);
     if (!program) {
@@ -68,19 +68,19 @@ export class GameCatalog {
     const url = row.cmd.match(/https?:\/\/[^\s"']+/)?.[0];
     if (!url) {
       const exe = await readFile(program).catch(() => null);
-      return even((exe && projectorStageSize(exe)) ?? DEFAULT_SIZE);
+      return ((exe && projectorStageSize(exe)) ?? DEFAULT_SIZE);
     }
     const content = await this.readContent(gameId, row, url).catch(() => null);
     const extension = path.extname(new URL(url).pathname).toLowerCase();
     if (['.dcr', '.dir', '.dxr', '.cct', '.cst'].includes(extension)) {
-      return even((content && directorStageSize(content)) ?? DEFAULT_SIZE);
+      return ((content && directorStageSize(content)) ?? DEFAULT_SIZE);
     }
     if (extension === '.swf') {
-      // The exact stage: in full screen, games that set noScale keep their native size
-      return even((content && swfStageSize(content)) ?? DEFAULT_SIZE);
+      // The exact stage, so the movie fills the display without scaling
+      return ((content && swfStageSize(content)) ?? DEFAULT_SIZE);
     }
     const embed = content && embedSize(content.toString('latin1'));
-    return even(embed ? { width: embed.width + 2 * PAGE_MARGIN, height: embed.height + 2 * PAGE_MARGIN } : BROWSER_SIZE);
+    return (embed ? { width: embed.width + 2 * PAGE_MARGIN, height: embed.height + 2 * PAGE_MARGIN } : BROWSER_SIZE);
   }
 
   /**
@@ -151,11 +151,6 @@ async function resolveCaseInsensitive(root: string, relative: string): Promise<s
   return (await stat(current).catch(() => null))?.isFile() ? current : null;
 }
 
-/** H.264 needs even dimensions. */
-function even({ width, height }: Size): Size {
-  return { width: width + (width % 2), height: height + (height % 2) };
-}
-
 /** Reads the stage size from a SWF header: a rect of twips after the 8-byte header. */
 export function swfStageSize(swf: Buffer): Size | null {
   const signature = swf.toString('latin1', 0, 3);

streaming/session-manager/test/games.test.ts

@@ -177,11 +177,11 @@ test('GameCatalog streams allowed platforms whose launcher resolves, sized from
     legacy: { width: 300, height: 200 },
     missing: { width: 800, height: 600 },
     webstart: null,
-    spr: { width: 500, height: 426 },
-    oldcpu: { width: 500, height: 426 },
+    spr: { width: 500, height: 425 },
+    oldcpu: { width: 500, height: 425 },
     secure: { width: 640, height: 480 },
     navigator: { width: 528, height: 400 },
-    flash: { width: 500, height: 376 },
+    flash: { width: 500, height: 375 },
     unity: null,
     'no-program': null,
     exe: { width: 800, height: 600 },

streaming/streamer/Dockerfile

@@ -30,6 +30,7 @@ RUN apt-get update && \
     libxtst6 \
     libxxf86vm1 \
     sqlite3 \
+    x11-utils \
     xdotool && \
   rm -rf /var/lib/apt/lists/*
 

streaming/streamer/README.md

@@ -52,6 +52,11 @@ writable mirror of the install, since they write beside themselves; a
 Director projector's `Prefs`, where it saves, is linked into `/config/saves`, as
 is Flash Player's save directory, so the session manager syncs both to the
 player's browser. OldCPUSimulator hangs under Wine, so its target runs under
-`cpulimit` at the same share of the CPU. Flash projectors switch to full screen,
-which hides their menu bar and scales the movie to the display, and Flashpoint
-Navigator's toolbars are hidden.
+`cpulimit` at the same share of the CPU. A Flash projector's window is
+stretched to the display and moved up by its menu bar's height, which hides the
+menu bar. GTK draws the menu bar in the window itself and the movie in a child
+window below it, so the movie's offset, read with `xwininfo`, is the menu bar's
+height. The window is made override-redirect first, since the window manager
+keeps windows on screen, and is kept in place while it's open, since Flash
+Player moves it back once the movie has loaded. Its full-screen mode isn't used, since some games, such
+as Bloons, don't play right in it. Flashpoint Navigator's toolbars are hidden.

streaming/streamer/root/usr/local/bin/fp-launch

@@ -47,8 +47,42 @@ case ${app,,} in
     mkdir -p "$HOME/saves/Flash_Player" "$HOME/.macromedia"
     [[ -e $HOME/.macromedia/Flash_Player ]] || ln -s "$HOME/saves/Flash_Player" "$HOME/.macromedia/Flash_Player"
     if [[ ${app,,} == 'fpsoftware\flash\'* ]]; then
-      # Full screen hides the projector's menu bar and scales the movie to the display
-      (xdotool search --sync --onlyvisible --name 'Flash Player' windowactivate --sync key ctrl+f) &
+      # Hide the projector's menu bar by moving it above the display, and fill
+      # the display with the movie. Not full screen, which some games, such as
+      # Bloons, don't play right in. The window manager keeps windows on
+      # screen, so the window is taken out of its hands first.
+      (
+        set +eo pipefail
+        wid=$(xdotool search --sync --onlyvisible --name 'Flash Player' | head -1)
+        read -r width height < <(xdotool getdisplaygeometry)
+        # GTK draws the menu bar in the window itself, and the movie in the
+        # largest child window, below it: the movie's offset is the menu bar's height
+        menu=$(xwininfo -children -id "$wid" | awk '
+          match($0, /[0-9]+x[0-9]+\+-?[0-9]+\+-?[0-9]+/) {
+            split(substr($0, RSTART, RLENGTH), g, /[x+]/)
+            if (g[1] * g[2] > area) { area = g[1] * g[2]; top = g[4] }
+          }
+          END { print top + 0 }')
+        ((menu >= 0 && menu < height)) || menu=0
+        echo "Flash Player window $wid: menu bar ${menu}px, display ${width}x${height}"
+        xdotool windowunmap --sync "$wid"
+        xdotool set_window --overrideredirect 1 "$wid"
+        xdotool windowmap --sync "$wid"
+        place() { xdotool windowsize "$wid" "$width" $((height + menu)) windowmove "$wid" 0 $((-menu)); }
+        place
+        xdotool windowfocus "$wid"
+        # Flash Player moves its window back once the movie has loaded, so it's
+        # kept in place for as long as it's open
+        moves=0
+        while geometry=$(xdotool getwindowgeometry --shell "$wid" 2>/dev/null); do
+          eval "$geometry"
+          if ((X != 0 || Y != -menu || WIDTH != width || HEIGHT != height + menu)); then
+            ((++moves <= 3)) && echo "Flash Player window moved to $X,$Y (${WIDTH}x${HEIGHT}); putting it back"
+            place
+          fi
+          sleep 0.5
+        done
+      ) &
     fi
     cd "${argv[0]}"
     exec "${argv[@]:1}" ;;