diff --git a/.env.example b/.env.example index 0ce276a..f14fb1c 100644 --- a/.env.example +++ b/.env.example @@ -22,8 +22,6 @@ JWT_SECRET=replace-with-a-secure-random-string # These map host ports to container ports (container ports are hardcoded) WEB_PORT=80 API_PORT=3100 -PROXY_PORT=22500 -GAMEZIP_PORT=22501 # --------------------------------------------------------------------------- # Data Storage @@ -34,7 +32,7 @@ DATA_PATH=./data # --------------------------------------------------------------------------- # Logging # --------------------------------------------------------------------------- -# Host path for logs (shared by all services: backend.log, game-service.log, client-errors.log) +# Host path for logs (backend.log, client-errors.log) LOGS_PATH=./logs LOG_LEVEL=info diff --git a/.github/workflows/docker-build-push.yml b/.github/workflows/docker-build-push.yml index e665368..74b4826 100644 --- a/.github/workflows/docker-build-push.yml +++ b/.github/workflows/docker-build-push.yml @@ -23,9 +23,6 @@ jobs: - name: frontend context: . dockerfile: ./frontend/Dockerfile - - name: game-service - context: . - dockerfile: ./game-service/Dockerfile permissions: contents: read @@ -93,7 +90,6 @@ jobs: echo "### Images Built:" >> $GITHUB_STEP_SUMMARY echo "- \`flashpoint-backend\`" >> $GITHUB_STEP_SUMMARY echo "- \`flashpoint-frontend\`" >> $GITHUB_STEP_SUMMARY - echo "- \`flashpoint-game-service\`" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY echo "### Registry:" >> $GITHUB_STEP_SUMMARY echo "- Docker Hub: \`${{ secrets.DOCKERHUB_USERNAME }}/flashpoint-*\`" >> $GITHUB_STEP_SUMMARY @@ -102,5 +98,4 @@ jobs: echo "\`\`\`bash" >> $GITHUB_STEP_SUMMARY echo "docker pull ${{ secrets.DOCKERHUB_USERNAME }}/flashpoint-backend:latest" >> $GITHUB_STEP_SUMMARY echo "docker pull ${{ secrets.DOCKERHUB_USERNAME }}/flashpoint-frontend:latest" >> $GITHUB_STEP_SUMMARY - echo "docker pull ${{ secrets.DOCKERHUB_USERNAME }}/flashpoint-game-service:latest" >> $GITHUB_STEP_SUMMARY echo "\`\`\`" >> $GITHUB_STEP_SUMMARY diff --git a/.prettierignore b/.prettierignore index b65f958..3f63efb 100644 --- a/.prettierignore +++ b/.prettierignore @@ -62,7 +62,6 @@ Thumbs.db !src/**/*.d.ts !frontend/src/**/*.d.ts !backend/src/**/*.d.ts -!game-service/src/**/*.d.ts # Documentation (preserve original formatting) docs/ diff --git a/CLAUDE.md b/CLAUDE.md index 4f79bde..572db15 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,20 +8,14 @@ code in this repository. **IMPORTANT:** This project has comprehensive documentation in the `docs/` directory (100+ files). When handling requests: -1. **Use Documentation Instead of Context**: Always reference the relevant - documentation files instead of repeating information in responses. This - prevents filling up context windows with redundant information. - -2. **Point to Documentation**: When answering questions, provide the path to the - relevant documentation file (e.g., "See `docs/04-frontend/architecture.md` - for details on frontend architecture"). - -3. **Read Documentation When Needed**: Use the Read tool to check documentation - files when you need specific information about: +1. **Reference docs instead of repeating information** — prevents filling + context windows with redundant content. +2. **Point to docs** when answering questions (e.g., "See + `docs/04-frontend/architecture.md`"). +3. **Read docs when needed** using the Read tool: - Architecture: `docs/02-architecture/` - Backend: `docs/03-backend/` - Frontend: `docs/04-frontend/` - - Game Service: `docs/05-game-service/` - API Reference: `docs/06-api-reference/` - Design System: `docs/07-design-system/` - Development: `docs/08-development/` @@ -29,319 +23,151 @@ directory (100+ files). When handling requests: - Features: `docs/10-features/` - Diagrams: `docs/11-diagrams/` - Reference: `docs/12-reference/` - -4. **Update Documentation After Changes**: After completing any feature, - enhancement, bug fix, or significant code change, **ALWAYS ask the user**: - - > "I've completed the changes. Should I update the relevant documentation in - > the `docs/` folder to reflect these changes?" - - Then update the appropriate documentation files to keep them in sync with the - codebase. +4. **Update docs after changes** — always ask the user if relevant documentation + should be updated. ### Quick Documentation Reference -**For Common Questions, Point Users/Read From:** - -| Question | Documentation File | -| ------------------------------- | ----------------------------------------------------------------------------------- | -| How do I set up the project? | `docs/08-development/setup-guide.md` | -| What commands are available? | `docs/08-development/commands.md` | -| How does authentication work? | `docs/10-features/01-authentication-authorization.md` | -| What are all the API endpoints? | `docs/06-api-reference/README.md` | -| What's the database schema? | `docs/12-reference/database-schema-reference.md` | -| How do components work? | `docs/04-frontend/components/component-overview.md` | -| How do I deploy? | `docs/09-deployment/README.md` | -| What's the architecture? | `docs/02-architecture/system-architecture.md` | -| Design system/theming? | `docs/07-design-system/theme-system.md` | -| Common errors/pitfalls? | `docs/08-development/common-pitfalls.md` and "Common Pitfalls" section in CLAUDE.md | -| TypeScript types? | `docs/12-reference/type-definitions.md` | -| Environment variables? | `docs/09-deployment/environment-variables.md` | -| How to add new API endpoints? | "Frontend API Guidelines" section in CLAUDE.md | +| Question | Documentation File | +| ------------------------------- | ----------------------------------------------------- | +| How do I set up the project? | `docs/08-development/setup-guide.md` | +| What commands are available? | `docs/08-development/commands.md` | +| How does authentication work? | `docs/10-features/01-authentication-authorization.md` | +| What are all the API endpoints? | `docs/06-api-reference/README.md` | +| What's the database schema? | `docs/12-reference/database-schema-reference.md` | +| How do components work? | `docs/04-frontend/components/component-overview.md` | +| How do I deploy? | `docs/09-deployment/README.md` | +| What's the architecture? | `docs/02-architecture/system-architecture.md` | +| Design system/theming? | `docs/07-design-system/theme-system.md` | +| Common errors/pitfalls? | `docs/08-development/common-pitfalls.md` | +| TypeScript types? | `docs/12-reference/type-definitions.md` | +| Environment variables? | `docs/09-deployment/environment-variables.md` | +| Game content serving? | `docs/05-game-service/architecture.md` | +| Security measures? | `docs/12-reference/security-measures.md` | ## Project Overview Flashpoint Web is a self-hosted web application for browsing and playing games -from the Flashpoint Archive. The project is a monorepo containing three -independent services: +from the Flashpoint Archive. Monorepo with two services: -- **backend**: REST API server (Express/TypeScript, port 3100) -- **frontend**: React web UI (Vite/React/TypeScript, port 5173) -- **game-service**: Game content proxy and ZIP server (Express/TypeScript, ports - 22500/22501) +- **backend** (Express/TypeScript, port 3100): REST API + integrated game + content serving via `/game-proxy/*` and `/game-zip/*` routes +- **frontend** (Vite/React/TypeScript, port 5173): SPA with TanStack Query, + Zustand, Tailwind CSS, Ruffle emulator -The architecture separates concerns: the backend handles game metadata and user -management via SQLite databases, the game-service serves actual game files and -handles proxying, and the frontend provides the user interface. +Two SQLite databases: `flashpoint.sqlite` (read-only, game metadata) and +`user.db` (app data — users, roles, playlists, play sessions). See +`docs/02-architecture/system-architecture.md` for full architecture details. ## Development Commands -### Monorepo (Root Level) - ```bash -# Install all dependencies for all services -npm run install:all +# Root level +npm run install:all # Install all dependencies +npm run dev # Start all services (concurrent) +npm run dev:backend # Backend only +npm run dev:frontend # Frontend only +npm run build # Build all +npm run typecheck # Type check all +npm run typecheck && npm run build # Verify before committing -# Start all services in development mode (concurrent) -npm run dev +# Backend (cd backend) +npm run dev # Dev server with hot reload (tsx) +npm run build # Build to dist/ +npm test # Run tests (vitest) +npm run typecheck # Type check only -# Start individual services -npm run dev:backend -npm run dev:frontend -npm run dev:game-service +# Frontend (cd frontend) +npm run dev # Dev server with HMR (Vite) +npm run build # Build to dist/ +npm run typecheck # Type check only +npm run copy-ruffle # Copy Ruffle emulator files (auto on postinstall) -# Build all services -npm run build - -# TypeScript type checking across all services -npm run typecheck - -# Verify build (typecheck + build) - use this before committing -npm run typecheck && npm run build - -# Clean all build artifacts and dependencies -npm run clean +# Docker +docker-compose up -d # Production +docker-compose -f docker-compose.dev.yml up -d # Development ``` -### Backend +## Frontend API Guidelines -```bash -cd backend - -# Development server with hot reload (tsx) -npm run dev - -# Build TypeScript to dist/ -npm run build - -# Run production build -npm start - -# TypeScript type checking only -npm run typecheck - -# Lint code -npm run lint - -# Run tests (vitest) -npm test -``` - -### Frontend - -```bash -cd frontend - -# Development server with HMR (Vite) -npm run dev - -# Build for production (outputs to dist/) -npm run build - -# Preview production build locally -npm run preview - -# TypeScript type checking only -npm run typecheck - -# Lint code -npm run lint - -# Copy Ruffle emulator files to public/ (runs automatically on postinstall) -npm run copy-ruffle -``` - -### Game Service - -```bash -cd game-service - -# Development server with hot reload -npm run dev - -# Build TypeScript -npm run build - -# Run production build -npm start - -# TypeScript type checking -npm run typecheck -``` - -### Docker - -```bash -# Production: Uses pre-built images from registry (docker-compose.yml) -docker-compose up -d - -# Development: Build images locally (docker-compose.dev.yml) -docker-compose -f docker-compose.dev.yml build -docker-compose -f docker-compose.dev.yml up -d - -# Verify Docker build (builds all images locally) -JWT_SECRET=test FLASHPOINT_HOST_PATH=/tmp docker-compose -f docker-compose.dev.yml build - -# View logs -docker-compose logs -f -docker-compose logs -f backend - -# Stop all services -docker-compose down -``` - -Set `FLASHPOINT_HOST_PATH` environment variable to point to your Flashpoint -installation before running Docker commands. - -## Architecture - -### Backend Service (Port 3100) - -The backend is a REST API built with Express and TypeScript. It manages two -separate SQLite databases: - -1. **Flashpoint Database** (`flashpoint.sqlite`): Read-only access to the - official Flashpoint game metadata database. The DatabaseService watches this - file for changes and hot-reloads when the Flashpoint Launcher updates it. - -2. **User Database** (`user.db`): Application-specific database for user - accounts, roles, permissions, playlists, favorites, play tracking, and - authentication. - -**Key architectural patterns:** - -- Service layer pattern: Business logic in `src/services/`, routes in - `src/routes/` -- JWT-based authentication with role-based access control (RBAC) -- File watching: Automatically detects changes to flashpoint.sqlite from the - Flashpoint Launcher -- Proxy delegation: Backend proxies game file requests to game-service instead - of serving them directly - -**Important services:** - -- `DatabaseService`: Manages connection to flashpoint.sqlite with hot-reload - support -- `UserDatabaseService`: Manages user.db with schema migrations -- `GameService`: Game metadata queries from flashpoint.sqlite -- `AuthService`: User authentication and JWT token management -- `PlayTrackingService`: Tracks user play sessions with cleanup of abandoned - sessions - -### Frontend Service (Port 5173) - -React 18 single-page application using modern patterns: - -**Tech stack:** - -- Vite for build tooling and dev server -- React Router for routing -- TanStack Query (React Query) for server state management -- Zustand for client-side UI state -- Tailwind CSS for styling -- Ruffle emulator for Flash content - -**State management approach:** - -- Server state (games, playlists, user data): TanStack Query with caching -- UI state (sidebar visibility, view modes): Zustand stores -- URL state (filters, search, pagination): React Router search params -- Auth state: Zustand store (`store/auth.ts`) with localStorage persistence - -**Component organization:** - -- `components/`: Reusable components organized by domain (auth, library, player, - playlist, etc.) -- `views/`: Top-level page components -- `lib/api.ts`: Centralized API client with typed endpoints - -**Player implementation:** - -- Flash games: Ruffle WebAssembly emulator embedded in iframe -- HTML5 games: Served directly via game-service proxy -- Animations: Separate view with same player infrastructure - -**Frontend API Guidelines:** - -**CRITICAL:** Always use the authenticated axios instance for backend API calls. - -- **Centralized API client**: All backend API calls MUST go through - `frontend/src/lib/api.ts` -- **Never use raw `fetch()`** for backend endpoints - it bypasses authentication -- **Axios interceptors**: The `api` instance automatically adds - `Authorization: Bearer ` headers -- **Pattern to follow**: Create typed API methods in appropriate objects (e.g., - `gamesApi`, `authApi`, `githubApi`) - -**Example (CORRECT):** +**CRITICAL:** All backend API calls MUST go through `frontend/src/lib/api.ts`. +Never use raw `fetch()` — it bypasses authentication (JWT headers, token +refresh, maintenance mode bypass for admins). ```typescript -// In frontend/src/lib/api.ts -export const githubApi = { - getStarCount: async (): Promise<{ stars: number }> => { - const { data } = await api.get<{ - success: boolean; - data: { stars: number }; - }>('/github/stars'); +// CORRECT — in frontend/src/lib/api.ts +export const exampleApi = { + getData: async () => { + const { data } = await api.get<{ success: boolean; data: Result }>( + '/example' + ); return data.data; }, }; -// In component -import { githubApi } from '@/lib/api'; -const result = await githubApi.getStarCount(); // ✅ Authenticated +// WRONG — bypasses auth +const response = await fetch('/api/example'); // Never do this ``` -**Example (WRONG):** +**When adding new endpoints:** -```typescript -// In component - DO NOT DO THIS -const response = await fetch('/api/github/stars'); // ❌ No authentication! -``` +1. Create backend route in `backend/src/routes/` +2. Add typed API method to `frontend/src/lib/api.ts` +3. Use the API method in components +4. Test with maintenance mode enabled -**Why this matters:** +## Common Pitfalls -- Maintenance mode blocks unauthenticated requests with 503 -- Protected endpoints require JWT tokens -- Axios interceptors handle token refresh automatically -- Consistent error handling across all API calls +1. **Route ordering**: Static routes (`/random`) MUST come before parameterized + routes (`/:id`) in Express routers +2. **`||` vs `??`**: Use `??` for config defaults — `||` treats `""`, `0`, + `false` as falsy +3. **`Array.sort()` mutates**: Use `[...arr].sort()` when original must be + preserved +4. **`parseInt()` validation**: Always check `isNaN()` — NaN silently passes to + SQL +5. **Limit caps**: Cap `limit` query params with `Math.min()` to prevent data + dumps +6. **`asyncHandler`**: Required on ALL async Express handlers +7. **`res.headersSent`**: Check before error responses in SSE endpoints +8. **Raw `fetch()`**: Never use for backend calls — see Frontend API Guidelines +9. **flashpoint.sqlite is read-only**: Never write to it +10. **Stream cleanup**: Handle `res.on('close')` to destroy streams on client + disconnect +11. **Atomic swap for hot-reload**: Open new resource, swap reference, close old +12. **Security allowlists > denylists**: For CGI env vars, download domains, + etc. +13. **Path security**: Always validate resolved paths stay within base + directories using `path.resolve()` + startsWith check +14. **Ruffle files missing**: Run `npm run copy-ruffle` in frontend -**When adding new backend endpoints:** +See `docs/08-development/common-pitfalls.md` for the full list with details. -1. Create the backend route in `backend/src/routes/` -2. Add the typed API method to `frontend/src/lib/api.ts` -3. Use the API method in components (never raw `fetch()`) -4. Test with maintenance mode enabled to ensure authentication works +## Key Architecture Notes -### Game Service (Ports 22500, 22501) - -Independent Node.js service that replaces the original Go-based Flashpoint Game -Server. Runs two HTTP servers: - -1. **HTTP Proxy Server (22500)**: Serves legacy web content with fallback chain: - - Local htdocs (`D:/Flashpoint/Legacy/htdocs`) - - Game data directory - - ZIP archives (via zip-manager) - - External CDN fallback (infinity.flashpointarchive.org) - - Local file cache for downloaded content - -2. **GameZip Server (22501)**: Mounts and serves files from ZIP archives in - `Data/Games/` without extraction using node-stream-zip - -**Key features:** - -- MIME type detection for 199+ file types -- CORS headers for cross-domain game content -- Streaming for large files -- External CDN fallback with caching -- File path normalization and URL decoding +- **Game play flow**: Frontend calls `GET /api/games/:id/launch` → backend + mounts ZIP → if downloading, frontend polls every 2s → when ready, renders + player +- **Game content routes** (`/game-proxy/*`, `/game-zip/*`) are registered BEFORE + auth middleware for cross-origin access +- **Edition auto-detection**: Read from `{FLASHPOINT_PATH}/version.txt`, stored + in `config` object (not DB). Frontend gets it via `/api/settings/public` +- **JWT auth with RBAC**: Permissions cached 5min via `PermissionCache`. + Maintenance mode blocks non-admins (503) +- **Token storage**: Refresh tokens in HTTP-only cookies (`fp_refresh`, path + `/api/auth`, 30-day expiry, secure flag in prod, SameSite=lax). Access tokens + in memory only. Session recovery on page reload via `/api/auth/refresh` + (cookie sent automatically) +- **CSRF protection**: Axios client uses `withCredentials: true` to enable + cookie transmission with requests +- **Database hot-reload**: `DatabaseService` watches flashpoint.sqlite and + reloads on changes from Flashpoint Launcher +- **Resource limits**: Server has `keepAliveTimeout`, `headersTimeout`, + `maxConnections`; external requests use dedicated axios with `maxSockets: 10`; + concurrent downloads capped at 3 ## Environment Configuration -### Backend (.env) - -Required variables: - ```bash # Required FLASHPOINT_PATH=D:/Flashpoint @@ -349,454 +175,265 @@ JWT_SECRET=your-secure-random-string # Optional DOMAIN=http://localhost:5173 -GAME_SERVICE_HOST=localhost LOG_LEVEL=info ``` -> **Note:** All paths (database, images, logos, htdocs, games) are automatically -> derived from `FLASHPOINT_PATH`. You only need to set the base Flashpoint -> installation path. - -> **Edition auto-detection:** The Flashpoint edition (Infinity or Ultimate) is -> automatically detected from `{FLASHPOINT_PATH}/version.txt` on startup. No -> environment variable is needed. The detected edition and version string are -> stored in the backend `config` object (not in the database). The frontend -> receives edition/version via `/api/settings/public` (injected from `config`). - -See `backend/.env.example` for complete configuration options including rate -limiting, SQLite optimization, and OpenTelemetry settings. - -### Game Service (.env) - -```bash -# Required -FLASHPOINT_PATH=D:/Flashpoint - -# Optional -LOG_LEVEL=info -``` - -> **Note:** All paths (htdocs, games) are automatically derived from -> `FLASHPOINT_PATH`. - -### Frontend - -The frontend does not require environment variables for local development. API -calls are proxied through Vite to the backend. - -```bash -VITE_APP_VERSION=1.0.0 # Optional: displayed in settings -``` - -Vite proxy configuration in `vite.config.ts` handles `/api/*` routing to -backend. In production, Nginx proxies API requests to the backend container. - -## Database Schema - -### Flashpoint Database (flashpoint.sqlite) - -Read-only database from Flashpoint project. Key tables: - -- `game`: Main game metadata (id, title, platform, library, developer, - publisher, etc.) -- `platform`: Gaming platforms -- `tag`: Tags and categories -- `game_tags_tag`: Many-to-many relationship between games and tags -- `playlist`: User playlists -- `playlist_game`: Playlist entries - -**Important:** This database is managed by the Flashpoint Launcher. The backend -watches for file changes and automatically reloads connections when updates -occur. - -### User Database (user.db) - -Application-specific SQLite database. Schema managed via migrations in -`backend/src/migrations/`. Tables include: - -- `users`: User accounts with hashed passwords (bcrypt) -- `roles`: RBAC roles (admin, moderator, user, guest) -- `role_permissions`: Permissions assigned to roles -- `user_roles`: User role assignments -- `user_playlists`: User-created playlists -- `user_favorites`: Favorited games per user -- `play_sessions`: Game play tracking with duration, completion status -- `system_settings`: Global system-wide configuration (auth, app, metadata, - features, game, storage, rate limiting) - -Migrations run automatically on server startup via UserDatabaseService. - -## Authentication & Authorization - -The application uses JWT-based authentication with RBAC: - -**Auth flow:** - -1. User logs in via `/api/auth/login` with username/password -2. Backend validates credentials, returns JWT access token + refresh token -3. Frontend stores tokens in localStorage and Zustand auth store -4. All subsequent API requests include JWT in Authorization header -5. Backend middleware validates JWT and checks permissions - -**Permission system:** - -- Permissions are strings like "games.play", "playlists.create", "users.manage" -- Roles bundle permissions together -- ProtectedRoute component in frontend enforces permission requirements -- Backend RBAC middleware validates permissions on protected endpoints - -**Guest mode:** - -- Users can browse games without authentication -- Playing games requires authentication and "games.play" permission - -**Maintenance mode:** - -- Controlled via `app.maintenance_mode` setting in `system_settings` table -- When enabled, blocks all non-admin users from accessing endpoints - (returns 503) -- Admin users identified by `settings.update` permission bypass maintenance mode -- Minimal public paths always accessible: `/health`, `/api/auth/login`, - `/api/auth/refresh`, `/api/settings/public`, `/proxy/*` -- Implemented in `backend/src/middleware/maintenanceMode.ts` -- Depends on `softAuth` middleware to populate `req.user` before checking - permissions - -**Permission caching:** - -- User permissions cached for 5 minutes via `PermissionCache` (in-memory) -- Cache automatically invalidated when user roles or role permissions change -- Cache cleared on server restart -- Debug issues by checking backend logs for `[Maintenance]` entries - -## Testing - -Backend includes vitest configuration. Run tests with: - -```bash -cd backend -npm test -``` - -Frontend testing setup TBD. - -## Important Implementation Notes - -### Database Access Patterns - -- **Never** use write operations on flashpoint.sqlite - it's read-only -- User database writes use prepared statements with transactions for data - consistency -- DatabaseService includes file watcher to detect external changes from - Flashpoint Launcher -- Use BetterSqlite3 synchronous API (not async wrapper) for better performance - -### Game File Serving - -Game files are **not** served by the backend. The flow is: - -1. Frontend requests game launch data from backend -2. Backend returns launch configuration pointing to game-service URLs -3. Frontend creates player iframe pointing to game-service (port 22500) -4. Game-service handles all file serving, proxying, and ZIP mounting - -This separation prevents backend from blocking on large file transfers and -allows independent scaling. - -### Ruffle Integration - -Ruffle emulator files are copied from npm package to `frontend/public/ruffle/` -during `npm install` (postinstall hook). The player component loads Ruffle from -`/ruffle/ruffle.js` and creates instances for Flash content. - -**Scale modes supported:** exactfit, noborder, showall (default), noscale - -### Play Session Tracking - -Play sessions have three states: - -1. Active: User is playing -2. Completed: User finished normally -3. Abandoned: Session older than 24 hours without completion - -Background job runs every 6 hours to clean up abandoned sessions. - -### CORS Configuration - -- Backend CORS origin must match frontend URL -- Game-service enables CORS for all origins (cross-domain game content) -- Vite proxy handles development routing to avoid CORS issues - -## Common Pitfalls - -1. **Flashpoint path not found**: Verify `FLASHPOINT_PATH` in .env points to - valid Flashpoint installation -2. **Database locked errors**: Flashpoint Launcher may lock flashpoint.sqlite - - close it before migrations -3. **Port conflicts**: Ports 3100, 5173, 22500, 22501 must be available -4. **Game files not loading**: Ensure game-service is running before backend -5. **JWT secret in production**: Change default JWT_SECRET in production - environments -6. **Ruffle files missing**: Run `npm run copy-ruffle` in frontend if Ruffle - doesn't load -7. **503 errors in maintenance mode**: If admin users get 503 errors, check if - components are using raw `fetch()` instead of the authenticated `api` - instance from `lib/api.ts`. The axios interceptors automatically add - authentication headers. -8. **Unauthenticated API calls**: Never use raw `fetch()` for backend API - endpoints. Always add new endpoints to `frontend/src/lib/api.ts` and use the - typed API methods. See "Frontend API Guidelines" section above. - -## Recent Changes & Lessons Learned - -### 2026-01-28: Maintenance Mode Admin Bypass Fix - -**Issue:** Admin users were getting 503 errors when accessing -`/api/github/stars` while maintenance mode was enabled. - -**Root Cause:** The `GitHubButton` component was using raw `fetch()` instead of -the authenticated axios instance, so requests had no `Authorization` header. The -maintenance mode middleware blocked these unauthenticated requests. - -**Fix:** - -1. Added `githubApi.getStarCount()` method to `frontend/src/lib/api.ts` -2. Updated `GitHubButton.tsx` to use the authenticated API method -3. The axios instance automatically adds JWT token via request interceptor - -**Key Lesson:** **NEVER use raw `fetch()` for backend API calls.** Always use -the centralized `api` instance from `lib/api.ts` which handles authentication, -token refresh, and error handling automatically. - -**Verification Steps Taken:** - -1. Checked database - confirmed admin role has `settings.update` permission ✅ -2. Tested backend directly - confirmed maintenance mode allows admin access ✅ -3. Identified frontend was making unauthenticated requests ❌ -4. Fixed component to use authenticated axios instance ✅ - -**Prevention:** See "Frontend API Guidelines" section for the correct pattern. -When adding new API endpoints, always: - -1. Add typed method to appropriate API object in `lib/api.ts` -2. Import and use the API method in components -3. Test with maintenance mode enabled - ---- - -### 2026-01-29: Game Service Directory Traversal Protection - -**Issue:** Game service lacked comprehensive path validation, potentially -allowing directory traversal attacks through crafted URLs. - -**Implementation:** Created centralized security utilities in -`game-service/src/utils/pathSecurity.ts`: - -1. **Path Sanitization & Validation**: - - `sanitizeAndValidatePath()` - Ensures paths stay within allowed directories - - Platform-aware (Windows case-insensitive, Unix case-sensitive) - - Validates resolved paths against base directories - -2. **URL Path Sanitization**: - - `sanitizeUrlPath()` - Detects null bytes, URL-encoded traversal, backslash - attacks - - Runs before path resolution for early detection - -3. **Applied to both servers**: - - **Legacy Server**: URL sanitization + path validation before all file - access - - **GameZip Server**: URL sanitization in file requests - -**Test Coverage**: 17 comprehensive test cases (all passing) - -**Security Guarantees**: - -- ✅ Blocks `../../../etc/passwd` attempts -- ✅ Blocks absolute path escapes (`/etc/passwd`) -- ✅ Blocks URL-encoded traversal (`..%2F`, `..%5C`) -- ✅ Blocks null byte injection (`file.swf\0.txt`) -- ✅ Blocks backslash traversal (`..\..\..\`) -- ✅ All blocked attempts logged with full details - -**Key Lesson:** Always validate that resolved file paths stay within allowed -base directories. Use `path.resolve()` and check the resolved path starts with -the base directory. Never trust user input, even after URL decoding. - -**Documentation**: - -- Detailed guide: `docs/13-security/directory-traversal-protection.md` -- Updated: `docs/12-reference/security-measures.md` - -**Phase**: Phase 1 (Security) - Critical priority fix from comprehensive project -review - ---- - -### 2026-02-04: Domain Settings for Playlist Sharing & Dynamic CORS - -**Change:** Added admin-configurable domains for playlist sharing URLs and -dynamic CORS. Admins can add multiple domains, set a default, and choose which -domain to use when sharing playlists. Regular users automatically use the -default domain. Falls back to `window.location.origin` when no domains are -configured. - -**How it works:** - -- New `domains` table in user.db stores hostname, is_default, created_by -- `DomainService` provides CRUD operations with 60s in-memory cache -- Backend CORS is now dynamic: checks configured domains + env var fallback -- Default domain injected into `/api/settings/public` response (same pattern as - `homeRecentHours` and `flashpointEdition`) -- `shareUrl` removed from backend `ShareLinkData` — frontend constructs URLs - locally using `buildShareUrl()` from `useDomains` hook -- Admin users see a domain selector dropdown in the Share Playlist dialog -- Non-admin users use the default domain from `usePublicSettings()` - -**New files:** - -- `backend/src/migrations/002_domains.sql` — Domains table schema -- `backend/src/services/DomainService.ts` — Domain CRUD + CORS cache -- `backend/src/routes/domains.ts` — REST API for domains -- `frontend/src/lib/api/domains.ts` — Frontend API client -- `frontend/src/hooks/useDomains.ts` — React Query hooks + `buildShareUrl()` - -**Modified files:** - -- `backend/src/routes/index.ts` — Register domains router -- `backend/src/server.ts` — Dynamic CORS origin function -- `backend/src/services/UserPlaylistService.ts` — Removed `shareUrl` from - `ShareLinkData`, removed unused `config` import -- `backend/src/routes/system-settings.ts` — Inject `defaultDomain` into - `/public` response -- `frontend/src/types/settings.ts` — Added `Domain` interface, updated - `PublicSettings` -- `frontend/src/types/playlist.ts` — Removed `shareUrl` from `ShareLinkData` -- `frontend/src/lib/api/index.ts` — Exported `domainsApi` -- `frontend/src/components/settings/AppSettingsTab.tsx` — Domain Settings card -- `frontend/src/components/playlist/SharePlaylistDialog.tsx` — Domain selector + - local URL construction - -**Key Lesson:** Moving URL construction to the frontend (with `buildShareUrl()`) -eliminates the need for the backend to know the frontend's domain for share -links. The admin-configurable domains table plus the `usePublicSettings()` hook -provide a clean way for both admin and regular users to get the right domain -without extra API calls. - -**Documentation:** - -- API reference: `docs/06-api-reference/domains-api.md` -- Feature guide: `docs/10-features/09-system-settings.md` (Domain Settings - section) -- Database schema: `docs/12-reference/database-schema-reference.md` (domains - table) -- CORS decision: `docs/12-reference/cors-security-decision.md` (dynamic CORS - section) - ---- - -### 2026-02-03: Auto-detect Flashpoint Edition from version.txt - -**Change:** Replaced the `FLASHPOINT_EDITION` environment variable with -automatic detection from `{FLASHPOINT_PATH}/version.txt`. Edition and version -are held in the backend `config` object — **not** stored in the -`system_settings` database table. Backend services read `config` directly. The -frontend receives edition/version via `/api/settings/public` (injected from -`config`). - -**How it works:** - -- On startup, `backend/src/config.ts` reads `version.txt` and parses the edition - (Infinity/Ultimate) and full version string -- Values stored in `config.flashpointEdition` and - `config.flashpointVersionString` (not in the database) -- Backend services (GameService, MetadataSyncService, MetadataUpdateService) - import and read `config` directly -- The `/api/settings/public` endpoint injects edition/version from `config` into - the response (same pattern as `homeRecentHours`) -- Frontend uses the `usePublicSettings()` hook (already cached, no extra - request) -- Falls back to `'infinity'` if `version.txt` is missing or unparseable - -**Files changed:** - -- `backend/src/config.ts` - Added `parseVersionFile()`, replaced env var -- `backend/src/routes/system-settings.ts` - Injects edition/version into - `/public` response from config -- `backend/src/server.ts` - Removed DB upserts, kept log lines only -- `backend/src/migrations/001_complete_schema.sql` - Removed - `metadata.flashpoint_edition` and `metadata.flashpoint_version` seed rows -- `backend/src/services/GameService.ts` - Uses `config` directly, removed - `CachedSystemSettingsService` dependency -- `backend/src/services/MetadataUpdateService.ts` - Uses `config` directly, - removed `CachedSystemSettingsService` dependency -- `backend/src/services/MetadataSyncService.ts` - Uses `config` directly, - removed `CachedSystemSettingsService` dependency -- `frontend/src/components/settings/GeneralSettingsTab.tsx` - Uses - `usePublicSettings()` instead of `systemSettingsApi.getCategory("metadata")` -- `frontend/src/views/SettingsView.tsx` - Uses `usePublicSettings()` instead of - `systemSettingsApi.getCategory("metadata")` -- `frontend/src/types/settings.ts` - Added `metadata` to `PublicSettings`, - removed `flashpointEdition` from `MetadataSettings` -- `backend/.env.example` - Removed `FLASHPOINT_EDITION` -- `docker-compose.yml` - Removed `FLASHPOINT_EDITION` -- `docker-compose.dev.yml` - Removed `FLASHPOINT_EDITION` - -**Key Lesson:** Auto-detection from the Flashpoint installation itself is more -reliable than manual configuration. Keeping runtime-detected values in `config` -(not in the database) avoids unnecessary DB writes on every startup and -simplifies the service layer by removing `CachedSystemSettingsService` -dependencies from services that only need the edition. - ---- - -### 2026-01-28: Database Migration Consolidation - -**Change:** Consolidated 16 separate migration files into 2 files: - -- `001_initialize_schema.sql`: All table definitions -- `002_seed_default_data.sql`: All seed data (roles, permissions, settings) - -**Added:** Migration registry system in `bootstrap.sql` to track applied -migrations with checksums and execution times. - -**Benefits:** - -- Cleaner migration directory -- Industry-standard migration tracking (similar to Flyway, Knex) -- Backward compatibility with existing databases -- All archived migrations saved in `migrations/archived/` for reference - ---- +All paths derived from `FLASHPOINT_PATH`. See `backend/.env.example` for all +options and `docs/09-deployment/environment-variables.md` for details. ## Source Code References -- Flashpoint Launcher source code: D:\Repositories\Community\launcher +- Flashpoint Launcher source: D:\Repositories\Community\launcher - Flashpoint Game Server: D:\Repositories\Community\FlashpointGameServer - -## Flashpoint App Reference: - - Flashpoint Infinity Edition: "D:\Flashpoint Infinity 14.0.3" - Flashpoint Ultimate Edition: "E:\Flashpoint Ultimate" +## Recent Changes & Lessons Learned + +### 2026-02-07: Navigation Revamp - Breadcrumb Bar with Back Button + +**Change:** Redesigned the Breadcrumbs component as a navigation bar with an +integrated back button. Added breadcrumbs to GamePlayerView (previously had +none). Removed standalone "Back" text links from GameDetailView, GamePlayerView, +and SharedPlaylistView. + +**What changed:** + +- `Breadcrumbs.tsx`: Restyled as nav bar (`bg-muted/50 rounded-lg`) with + ArrowLeft back button, vertical divider, and pill-style breadcrumb items. New + `showBackButton` prop (default: true). Added `BreadcrumbContext` (moved from + GameCard.tsx) and `PlayerBreadcrumbContext` types. +- `GameDetailView.tsx`: Removed standalone back button. Play button and + auto-play now pass `PlayerBreadcrumbContext` via React Router navigation + state. +- `GamePlayerView.tsx`: Added breadcrumbs showing + `Home > [Context] > Game Title > Play`. Reads context from navigation state, + falls back to `Browse > Game > Play` for direct URL access. Hidden in + fullscreen. +- `SharedPlaylistView.tsx`: Removed standalone "Back to my playlists" link. +- 5 library components: Updated `BreadcrumbContext` import source from + `GameCard.tsx` to `Breadcrumbs.tsx`. + +**Files modified (9):** + +- `frontend/src/components/common/Breadcrumbs.tsx` +- `frontend/src/views/GameDetailView.tsx` +- `frontend/src/views/GamePlayerView.tsx` +- `frontend/src/views/SharedPlaylistView.tsx` +- `frontend/src/components/library/GameCard.tsx` +- `frontend/src/components/library/GameBrowseLayout.tsx` +- `frontend/src/components/library/GameGrid.tsx` +- `frontend/src/components/library/GameList.tsx` +- `frontend/src/components/library/GameListItem.tsx` + +**Key Lesson:** Breadcrumb context must flow through navigation state +(`location.state`) since React Router doesn't preserve component state across +page transitions. Use a typed context interface (`PlayerBreadcrumbContext`) and +always provide sensible fallbacks for direct URL access. + +### 2026-02-07: Low-Priority Code Quality Fixes (~48 issues) + +**Change:** Final round of code quality improvements from the comprehensive code +review. Focused on type safety, dead code removal, DRY violations, import +consistency, and theme token standardization. + +**Backend fixes (14 files):** + +- Replaced `any` types with proper types across 6 services (MetadataSyncService, + PlaylistService, UserPlaylistService, JobExecutionService, + CommunityPlaylistService, RuffleService) +- Removed `.js` extension from import, simplified redundant + `parseInt(String(...))` calls +- Added `.catch()` on fire-and-forget promises, `.unref()` on cleanup intervals +- Demoted noisy `logger.info` to `logger.debug`, removed duplicate default + export + +**Game service fixes (9 files):** + +- Added `headersSent` guard to `sendError()` in proxy-request-handler to prevent + write-after-headers errors +- Made `findActiveDownload` private and `PRIVATE_IP_RANGES` static readonly +- Removed unused imports (`AxiosError`), aliases (`realpath`), and convenience + re-exports +- Removed `export` from internal-only `CUSTOM_MIME_TYPES` constant + +**Frontend DRY + types fixes (12 files):** + +- Deleted duplicate `errorUtils.ts` - consolidated to `types/api-error.ts` +- Replaced duplicate `formatDuration`/`formatPlaytime` functions with shared + import from `cron-utils.ts` +- Fixed `any` types in `logger.ts` (`any[]` → `unknown[]`) and + `createQueryHook.ts` +- Removed dead `Window.opera` type declaration, redundant variable aliases, + duplicate conditional rendering +- Fixed `LazyBackgroundImage` callback refs to prevent effect dependency issues + +**Frontend imports + theming fixes (15 files):** + +- Standardized 10 files from relative `../` imports to `@/` alias imports +- Replaced hardcoded Tailwind colors with theme tokens across 4 components + (e.g., `text-gray-400` → `text-muted-foreground`, `border-primary-500` → + `border-primary`) +- Fixed index-based list keys → content-based keys in GameListItem + +**Key Lessons:** + +- Duplicate utility functions (`errorUtils.ts` vs `api-error.ts`) cause + confusion - consolidate early +- Callback props in `useEffect` dependencies cause re-render loops - use + `useRef` pattern +- Fire-and-forget promises need `.catch()` to prevent unhandled rejections +- `setInterval` without `.unref()` prevents Node.js graceful shutdown +- Internal-only constants should not be exported - reduces public API surface +- Theme tokens (`text-muted-foreground`) adapt to dark/light mode; hardcoded + colors don't + +### 2026-02-07: ~78 Medium-Priority Code Quality Fixes + +**Change:** Applied ~78 medium-priority fixes across backend routes, backend +services, game service, and frontend from comprehensive code review. + +**Backend Routes (27 fixes):** + +- `asyncHandler` added to: `game-zip.ts`, `github.ts`, `game-proxy.ts`, + `authenticate`/`optionalAuth` middleware +- Zod validation: `community-playlists.ts` (downloadUrl), `shared-playlists.ts` + (newTitle), `cache.ts` (cacheType), `activities.ts` (timeRange) +- `parseInt` NaN fallback: `favorites.ts`, `jobs.ts` offset; `config.ts` 6 env + vars +- Bug fix: `cache.ts` overlapping `if` → `switch` (was clearing wrong caches) +- `.catch()` on fire-and-forget cache write in `proxy.ts` +- `.unref()` on 3 cleanup intervals in `server.ts` + +**Backend Services (12 fixes):** + +- Removed `any` types: `shared-playlists.ts`, `activities.ts`, + `CommunityPlaylistService.ts` +- Consolidated `ALLOWED_DOWNLOAD_DOMAINS` as single exported constant +- Fixed timeout leak in `UpdateService.ts` +- Simplified duplicate validation in `DomainService.ts` +- Removed redundant `ensureLogDir` in `errors.ts` + +**Game Service (19 fixes):** + +- `.catch()` on `downloadAndMountInBackground`; `.unref()` on cleanup interval +- Shared `MAX_BUFFERED_FILE_SIZE` constant replaces 50MB magic number in 3 files +- Hostname port stripping: `host.split(':')[0]` +- JSON parse vs ENOENT differentiation in config loading +- 5 defensive checks in `GameDataDownloader.ts` +- Removed unused `hasActiveDownloads()`; added `LegacyServer.clearCache()` +- Fixed hostname variation skip for already-prefixed hostnames +- Per-request logging downgraded info→debug + +**Frontend (20 fixes):** + +- `id!` → `id ?? ''` in `PlaylistDetailView.tsx`; NaN guard in + `UserPlaylistDetailView.tsx` +- `|| 24` → `?? 24` in `HomeView.tsx` +- `LazyImage.tsx`: cancellation flag + ref-stored callbacks +- `NetworkStatusIndicator.tsx`: timeout moved to useRef +- Tag keys index→`tag.trim()` in 2 views +- `console.error`/`console.debug` → `logger` in 3 files +- Removed `window.opera` dead code; added `GameCard.displayName` +- Removed `isInView` from `useLazyLoad` deps + +**Key Lessons:** + +- Use `switch` for mutually exclusive branching, not overlapping `if` blocks +- Add `.catch()` to all fire-and-forget promises +- Call `.unref()` on non-critical intervals to allow graceful shutdown +- Strip port from `req.headers.host` before hostname validation +- Store setTimeout/setInterval IDs in `useRef`, not `let` variables +- Return cleanup functions from `useEffect` for `new Image()` preloads +- Export shared constants to eliminate magic number duplication + +### 2026-02-07: 22 High-Priority Security & Data Integrity Fixes + +**Change:** Applied 22 high-priority fixes across backend, game service, and +frontend from targeted code review. + +**Backend Auth & Data Integrity (7 fixes):** + +- `middleware/auth.ts`: Guest users (id=0) excluded from shared access bypass +- `AuthService.ts`: Registration uniqueness checks moved inside transaction + (TOCTOU fix) +- `UserService.ts`: `createUser()` wrapped in transaction (TOCTOU fix) +- `SystemSettingsService.ts`: `updateCategory()` wrapped in transaction +- `user-playlists.ts`: Zod schema limits aligned (500→100) +- `UserPlaylistService.ts`: `cloneSharedPlaylist()` wrapped in transaction +- `downloads.ts`: SSE write-after-close prevention with `closed` flag + +**DownloadManager (1 fix):** + +- `DownloadManager.ts`: AbortSignal listener cleanup in finally blocks (memory + leak fix) + +**Game Service (8 fixes):** + +- `zip-manager.ts`: ZIP handle closed on mount failure + `shuttingDown` flag for + graceful shutdown +- `proxy-request-handler.ts`: `collectRequestBody` double-settlement prevented + with `settled` flag +- `legacy-server.ts`: Only cache ENOENT (not validation errors) + URL + constructor for path safety +- `GameDataDownloader.ts`: DNS rebinding SSRF prevention via `dns.lookup()` + pre-check +- `cgi-executor.ts`: Symlink bypass prevention via `fs.realpath()` before path + validation +- `gamezipserver.ts`: Downloaded file path re-validated after download + +**Frontend (6 fixes):** + +- `store/auth.ts`: Guest permissions restricted to read-only (removed + `games.play`) +- `store/theme.ts`: Server data validated before applying (mode + primaryColor) +- `AuthContext.tsx`: Maintenance mode check uses authenticated `apiClient`; + permissions refreshed from server after token refresh +- `useSharedPlaylistAccess.ts`: `encodeURIComponent(shareToken)` for URL safety +- `usePlayTracking.ts`: `isMountedRef` guard prevents StrictMode double session + start + +**Key Lessons:** + +- Wrap all uniqueness-check-then-insert patterns in `db.transaction()` to + prevent TOCTOU races +- Track `closed` state for SSE connections and check before every `res.write()` +- Close ZIP handles in catch blocks when mount operations fail after open +- Use `shuttingDown` flag to prevent new resource acquisition during shutdown +- Only cache definitive ENOENT errors, never transient failures (negative cache + poisoning) +- Use `URL` constructor for safe URL resolution, not string concatenation +- Pre-check DNS resolution to block private addresses (DNS rebinding SSRF) +- Resolve symlinks with `fs.realpath()` before path security validation +- Validate server data (theme settings) before applying to prevent crashes +- Use `isMountedRef` + timeout to survive React StrictMode unmount/remount + cycles + +**Files Modified (19 files):** + +- Backend: `auth.ts` (middleware), `AuthService.ts`, `UserService.ts`, + `SystemSettingsService.ts`, `user-playlists.ts`, `UserPlaylistService.ts`, + `downloads.ts`, `DownloadManager.ts` +- Game service: `zip-manager.ts`, `proxy-request-handler.ts`, + `legacy-server.ts`, `GameDataDownloader.ts`, `cgi-executor.ts`, + `gamezipserver.ts` +- Frontend: `store/auth.ts`, `store/theme.ts`, `AuthContext.tsx`, + `useSharedPlaylistAccess.ts`, `usePlayTracking.ts` + ## Documentation Maintenance **After ANY code change, always:** 1. **Run Prettier** on all new and modified files: - `npx prettier --write ...` When using sub-agents (Task tool), - instruct each agent to run Prettier on every file it creates or modifies - before finishing. + `npx prettier --write ...` — also instruct sub-agents to do + this. 2. Verify no build errors: `npm run typecheck` and `npm run build` -3. Check if documentation needs updates -4. Ask the user if relevant documentation should be updated -5. Update the following docs as needed: - - Architecture changes → `docs/02-architecture/` - - New/modified API endpoints → `docs/06-api-reference/` - - Database schema changes → `docs/12-reference/database-schema-reference.md` - - New features → `docs/10-features/` - - Component changes → `docs/04-frontend/components/` - - Configuration changes → `docs/03-backend/configuration.md` or +3. Ask the user if relevant documentation should be updated +4. Update docs as needed: + - Architecture → `docs/02-architecture/` + - API endpoints → `docs/06-api-reference/` + - DB schema → `docs/12-reference/database-schema-reference.md` + - Features → `docs/10-features/` + - Components → `docs/04-frontend/components/` + - Config → `docs/03-backend/configuration.md` or `docs/09-deployment/environment-variables.md` - -**Documentation is a first-class concern** - keeping it current ensures the -project remains maintainable. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 256d80f..af262b5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -70,10 +70,8 @@ npm run install:all # Copy environment files cp backend/.env.example backend/.env -cp frontend/.env.example frontend/.env -cp game-service/.env.example game-service/.env -# Edit .env files with your Flashpoint paths +# Edit .env file with your Flashpoint path # See docs/08-development/setup-guide.md for details # Start development servers @@ -89,7 +87,6 @@ npm run typecheck # Run tests cd backend && npm test cd ../frontend && npm test -cd ../game-service && npm test ``` For detailed setup instructions, see @@ -397,9 +394,8 @@ Update documentation when you: docs/ ├── 01-overview/ # Project introduction ├── 02-architecture/ # System design -├── 03-backend/ # Backend service docs +├── 03-backend/ # Backend service docs (including game content serving) ├── 04-frontend/ # Frontend service docs -├── 05-game-service/ # Game service docs ├── 06-api-reference/ # API documentation ├── 07-design-system/ # UI/UX guidelines ├── 08-development/ # Development guides @@ -419,11 +415,12 @@ docs/ ``` flashpoint-web/ -├── backend/ # Express REST API +├── backend/ # Express REST API with game content serving │ ├── src/ │ │ ├── routes/ # API route handlers │ │ ├── services/ # Business logic │ │ ├── middleware/# Express middleware +│ │ ├── game/ # Game content serving (proxy, ZIP mounting) │ │ └── utils/ # Utility functions │ └── package.json │ @@ -436,12 +433,6 @@ flashpoint-web/ │ │ └── store/ # State management │ └── package.json │ -├── game-service/ # Game file server -│ ├── src/ -│ │ ├── servers/ # HTTP servers -│ │ └── utils/ # Utilities -│ └── package.json -│ ├── docs/ # Documentation └── package.json # Root package ``` @@ -472,15 +463,14 @@ Contributors are recognized in: ## Quick Reference -| Task | Command | -| -------------------- | ------------------------- | +| Task | Command | +| ------------------- | ------------------------- | | Install dependencies | `npm run install:all` | -| Start dev servers | `npm run dev` | -| Type check | `npm run typecheck` | -| Format code | `npm run format` | -| Run backend tests | `cd backend && npm test` | -| Run frontend tests | `cd frontend && npm test` | -| Build all | `npm run build` | +| Start dev servers | `npm run dev` | +| Type check | `npm run typecheck` | +| Format code | `npm run format` | +| Run backend tests | `cd backend && npm test` | +| Build all | `npm run build` | --- diff --git a/DOCUMENTATION_UPDATES.md b/DOCUMENTATION_UPDATES.md new file mode 100644 index 0000000..c55ceaf --- /dev/null +++ b/DOCUMENTATION_UPDATES.md @@ -0,0 +1,120 @@ +# Documentation Updates - Second Pass Code Review + +Date: 2026-02-06 + +Updated the following game-service documentation files to reflect medium and low +severity fixes from the second pass code review: + +## Files Updated + +### 1. docs/05-game-service/gamezip-server.md + +Added section "Download Reliability Improvements" documenting: + +- `findActiveDownload()` now correlates downloads with hostname from request path + instead of returning first arbitrary download (ensures correct download status + on concurrent requests) +- Stale download cleanup extracted to shared `cleanupStaleDownloads()` method + (DRY principle, shared between functions) +- `new URL()` calls wrapped in try-catch for proper 400 error handling on + malformed URLs (prevents 500 errors) + +### 2. docs/05-game-service/zip-manager.md + +Added section "Performance Improvement - findFile() Refactoring" documenting: + +- `findFile()` method optimized for single-pass lookup +- Entries read directly without redundant delegation through `getFile()` +- Process: index check → entry existence check → size check → read +- Eliminates intermediate function calls and extra lookups +- Result: Faster file discovery with fewer function invocations + +### 3. docs/05-game-service/legacy-server.md + +Added section "Dynamic Configuration Loading" documenting: + +- Settings now fetched fresh via `ConfigManager.getSettings()` at the start of + each `serveLegacy()` call instead of being captured once at construction +- Ensures configuration changes (Brotli, external URLs, override paths) take + effect immediately without restart +- Benefit: Admin users can update proxy settings via UI without downtime + +### 4. docs/05-game-service/proxy-server.md + +Updated "CORS Handling" section: + +- CORS headers now use shared `setCorsHeaders` utility from `utils/cors.ts` +- Utility includes all required methods: GET, POST, DELETE, OPTIONS +- Eliminates duplication across both proxy servers +- Consistent CORS configuration across the module + +### 5. docs/05-game-service/architecture.md + +Added major section "Code Quality & Security Improvements (Second Pass Review)" +documenting all medium and low severity fixes: + +**HTML Injection & Logging:** +- `htmlInjector.ts` now uses project logger instead of `console.log` +- Dead code branch removed from polyfill injection logic + +**PreferencesService Initialization:** +- Logs warning on re-initialization with different path +- Prevents silent configuration changes affecting game downloads + +**Schema Validation:** +- Hostname schema minimum length corrected to 2 (matches regex requirement) + +**Regex Pattern Fixes:** +- `pathSecurity.ts` regex patterns no longer use `/g` flag with `test()` +- Prevents stateful matching bugs on consecutive calls + +**Code Cleanup:** +- Unused exports removed from: + - `mimeTypes.ts` + - `validation/schemas.ts` + - `utils/cors.ts` + - `utils/pathSecurity.ts` +- Reduces module surface area and maintenance burden + +**Download Validation:** +- `GameDataDownloader.getFilename()` validates timestamps +- Rejects NaN timestamps to prevent corrupted filenames + +**Resource Leak Prevention:** +- `GameDataDownloader` destroys response stream on size limit +- Prevents resource leaks from oversized downloads + +**DRY Principle:** +- `GameDataService` uses `GameDataDownloader.getFilename()` instead of + duplicating logic +- Centralizes filename generation + +### 6. docs/05-game-service/configuration.md + +Updated "CGI Configuration (via proxySettings.json)" section with information +about `PreferencesService`: + +- Documents singleton pattern for reading `preferences.json` +- Explains re-initialization warning logging +- Helps identify configuration issues during development +- Examples of warning messages logged + +## Impact + +These documentation updates ensure that: + +1. **Code clarity**: All improvements are documented for future maintainers +2. **Knowledge preservation**: Medium and low severity fixes are recorded +3. **Developer onboarding**: New team members understand recent improvements +4. **Architecture understanding**: Readers see complete picture of design +5. **Best practices**: Documents explain why each improvement matters + +## Markdown Formatting + +All updated files follow consistent markdown formatting with: + +- Clear section headers +- Code examples where relevant +- Bullet points for key improvements +- Context about why changes were made +- Benefits and implications explained diff --git a/README.md b/README.md index cfa4208..13096cc 100644 --- a/README.md +++ b/README.md @@ -61,13 +61,15 @@ support, play tracking, and an intuitive interface. ## Architecture -Flashpoint Web consists of three independent services: +Flashpoint Web consists of two independent services: -| Service | Port | Description | -| ---------------- | ----------- | ----------------------------------------------------- | -| **Backend** | 3100 | REST API for game metadata, users, and authentication | -| **Frontend** | 5173 | React web application | -| **Game Service** | 22500/22501 | File serving proxy and ZIP archive server | +| Service | Port | Description | +| ------------ | ---- | --------------------------------------------------------------- | +| **Backend** | 3100 | REST API for metadata, users, auth, and game content serving | +| **Frontend** | 5173 | React web application | + +Game content is served directly by the backend via `/game-proxy/*` and +`/game-zip/*` routes, enabling efficient file serving and ZIP archive mounting. ## Quick Start @@ -107,25 +109,9 @@ DOMAIN=http://localhost:5173 LOG_LEVEL=info ``` -> **Note:** All database and asset paths are automatically derived from -> `FLASHPOINT_PATH`. - -**2. Game Service** (`game-service/.env`) - -```bash -cp game-service/.env.example game-service/.env -``` - -```env -# Required -FLASHPOINT_PATH=D:/Flashpoint - -# Optional -LOG_LEVEL=info -``` - -> **Note:** Frontend requires no `.env` file for local development - API calls -> are proxied through Vite. +> **Note:** All database, asset, and game content paths are automatically +> derived from `FLASHPOINT_PATH`. Frontend requires no `.env` file for local +> development - API calls are proxied through Vite. ### Run Development Servers @@ -134,9 +120,8 @@ LOG_LEVEL=info npm run dev # Or start individually -npm run dev:backend # http://localhost:3100 -npm run dev:frontend # http://localhost:5173 -npm run dev:game-service # http://localhost:22500 +npm run dev:backend # http://localhost:3100 +npm run dev:frontend # http://localhost:5173 ``` Open http://localhost:5173 in your browser. @@ -164,9 +149,13 @@ docker compose -f docker-compose.dev.yml up -d --build docker compose logs -f ``` -**Default ports:** | Service | URL | |---------|-----| | Frontend | -http://localhost (port 80) | | Backend API | http://localhost:3100 | | Game -Proxy | http://localhost:22500 | | GameZip Server | http://localhost:22501 | +**Default ports:** + +| Service | URL | +| --- | --- | +| Frontend | http://localhost (port 80) | +| Backend API | http://localhost:3100 | +| Game Content Routes | http://localhost:3100/game-proxy/* and /game-zip/* | See [Docker Deployment Guide](docs/09-deployment/docker-deployment.md) for advanced configuration. @@ -266,21 +255,6 @@ Comprehensive documentation is available in the [`docs/`](docs/) directory: - Shadcn UI + Radix - Ruffle Flash Emulator - - - -### Game Service - -

- Node.js - Express -

- -- Express 4.18 -- node-stream-zip 1.15 -- Winston logging -- CDN fallback support - @@ -314,8 +288,9 @@ npm run copy-ruffle **Games not loading:** -- Ensure game-service is running (ports 22500/22501) -- Check game-service logs: `npm run dev:game-service` +- Ensure backend is running on port 3100 +- Check backend logs: `npm run dev:backend` +- Verify game content routes are accessible at `/game-proxy/*` and `/game-zip/*` See [Common Pitfalls](docs/08-development/common-pitfalls.md) for more solutions. diff --git a/backend/.env.example b/backend/.env.example index adb41b1..285da40 100644 --- a/backend/.env.example +++ b/backend/.env.example @@ -10,9 +10,6 @@ NODE_ENV=development LOG_LEVEL=info DOMAIN=http://localhost:5173 -# Backend -> game-service connection (local development) -GAME_SERVICE_HOST=localhost - # Rate limiting RATE_LIMIT_WINDOW_MS=60000 RATE_LIMIT_MAX_REQUESTS=100 diff --git a/backend/README.md b/backend/README.md index fb02554..26f77ef 100644 --- a/backend/README.md +++ b/backend/README.md @@ -6,7 +6,7 @@ Node.js/Express API server for the Flashpoint Archive web application. - RESTful API for game metadata - Direct SQLite database access (read-only) -- Proxy to Flashpoint Game Server +- Integrated game content serving (proxy + ZIP) - Image and logo serving - Playlist management - Platform and tag endpoints @@ -45,6 +45,7 @@ See `../../api-specification.md` for complete API documentation. src/ ├── routes/ # Express route handlers ├── services/ # Business logic layer +├── game/ # Integrated game service (proxy, ZIP, CGI) ├── middleware/ # Express middleware ├── utils/ # Helper functions ├── config.ts # Configuration management @@ -83,7 +84,6 @@ Optional: - `PORT` - Server port (default: 3100) - `NODE_ENV` - Environment (development/production) -- `GAME_SERVER_URL` - Game Server URL (default: http://localhost:22500) - `DOMAIN` - Frontend origin (default: http://localhost:5173) See `.env.example` for all options. diff --git a/backend/package.json b/backend/package.json index f1cf386..c4210bb 100644 --- a/backend/package.json +++ b/backend/package.json @@ -37,6 +37,7 @@ "better-sqlite3": "^12.6.0", "cheerio": "^1.1.2", "compression": "^1.7.4", + "cookie-parser": "^1.4.7", "cors": "^2.8.5", "cron-parser": "^5.5.0", "dotenv": "^16.4.5", @@ -53,10 +54,11 @@ }, "devDependencies": { "@types/adm-zip": "^0.5.7", - "@types/cheerio": "^0.22.35", "@types/bcrypt": "^6.0.0", "@types/better-sqlite3": "^7.6.13", + "@types/cheerio": "^0.22.35", "@types/compression": "^1.7.5", + "@types/cookie-parser": "^1.4.10", "@types/cors": "^2.8.17", "@types/express": "^4.17.21", "@types/jsonwebtoken": "^9.0.10", diff --git a/backend/scripts/kill-port.js b/backend/scripts/kill-port.js deleted file mode 100644 index c4112d8..0000000 --- a/backend/scripts/kill-port.js +++ /dev/null @@ -1,67 +0,0 @@ -#!/usr/bin/env node - -/** - * Helper script to kill processes using specific ports - * Usage: node scripts/kill-port.js [port] - * Example: node scripts/kill-port.js 22500 - */ - -const { execSync } = require('child_process'); - -const port = process.argv[2] || '22500'; - -console.log(`🔍 Checking for processes using port ${port}...`); - -try { - // Find process using the port (Windows) - const result = execSync(`netstat -ano | findstr :${port}`, { encoding: 'utf-8' }); - - if (!result.trim()) { - console.log(`✓ Port ${port} is free!`); - process.exit(0); - } - - console.log(result); - - // Extract PIDs - const lines = result.trim().split('\n'); - const pids = new Set(); - - for (const line of lines) { - const parts = line.trim().split(/\s+/); - const pid = parts[parts.length - 1]; - if (pid && !isNaN(pid)) { - pids.add(pid); - } - } - - if (pids.size === 0) { - console.log(`✓ Port ${port} is free!`); - process.exit(0); - } - - console.log(`\n⚠️ Found ${pids.size} process(es) using port ${port}`); - console.log(`PIDs: ${Array.from(pids).join(', ')}`); - - // Kill each process - for (const pid of pids) { - try { - console.log(`\n🔨 Killing process ${pid}...`); - // Use /F and /PID (single slash) for Windows - execSync(`taskkill /F /PID ${pid}`, { encoding: 'utf-8' }); - console.log(`✓ Process ${pid} terminated`); - } catch (error) { - console.error(`✗ Failed to kill process ${pid}:`, error.message); - } - } - - console.log(`\n✅ Port ${port} should now be free!`); -} catch (error) { - if (error.status === 1 && error.stderr === '') { - // findstr returns exit code 1 when nothing is found - console.log(`✓ Port ${port} is free!`); - } else { - console.error('❌ Error:', error.message); - process.exit(1); - } -} diff --git a/backend/src/config.ts b/backend/src/config.ts index dde5bcf..090961b 100644 --- a/backend/src/config.ts +++ b/backend/src/config.ts @@ -27,6 +27,7 @@ function getJwtSecret(): string { // In development, generate a random secret and warn const devSecret = `INSECURE-DEV-ONLY-${crypto.randomBytes(32).toString('hex')}`; + // NOTE: console.warn used intentionally - logger not yet initialized at config load time console.warn('\n⚠️ WARNING: Using auto-generated JWT secret for development.'); console.warn(' In production, set JWT_SECRET environment variable.\n'); @@ -95,27 +96,12 @@ export const config = { flashpointPlaylistsPath: `${flashpointPath}/Data/Playlists`, flashpointGamesPath: `${flashpointPath}/Data/Games`, - // Game Service URLs (external service, not built-in) - // Internal URLs: Used for backend-to-game-service communication within Docker - // Ports hardcoded (22500/22501) - only host is configurable (localhost for dev, game-service for docker) - gameServerUrl: `http://${process.env.GAME_SERVICE_HOST || 'localhost'}:22500`, - gameServiceGameZipUrl: `http://${process.env.GAME_SERVICE_HOST || 'localhost'}:22501`, - gameServerHttpPort: 22501, - - // External Game Service URL: Used for URLs returned to the browser - // In Docker: Use relative paths through nginx proxy (/game-proxy) - // In local dev: Use direct localhost URLs (http://localhost:22500) - // Can be overridden with GAME_SERVICE_EXTERNAL_URL env var - gameServiceExternalUrl: - process.env.GAME_SERVICE_EXTERNAL_URL || - (process.env.NODE_ENV === 'production' ? '/game-proxy' : `http://localhost:22500`), - // Frontend domain (used for CORS and share URL generation) domain: process.env.DOMAIN || 'http://localhost:5173', // Rate limiting - rateLimitWindowMs: parseInt(process.env.RATE_LIMIT_WINDOW_MS || '60000', 10), - rateLimitMaxRequests: parseInt(process.env.RATE_LIMIT_MAX_REQUESTS || '100', 10), + rateLimitWindowMs: parseInt(process.env.RATE_LIMIT_WINDOW_MS || '', 10) || 60000, + rateLimitMaxRequests: parseInt(process.env.RATE_LIMIT_MAX_REQUESTS || '', 10) || 100, // Logging // In Docker (production): defaults to /app/logs/backend.log @@ -135,10 +121,10 @@ export const config = { jwtExpiresIn: process.env.JWT_EXPIRES_IN || '1h', // Security - bcryptSaltRounds: parseInt(process.env.BCRYPT_SALT_ROUNDS || '10', 10), + bcryptSaltRounds: parseInt(process.env.BCRYPT_SALT_ROUNDS || '', 10) || 10, // Home Page Configuration - homeRecentHours: parseInt(process.env.HOME_RECENT_HOURS || '24', 10), + homeRecentHours: parseInt(process.env.HOME_RECENT_HOURS || '', 10) || 24, // Local Database Copy (for network storage optimization) // When enabled, copies flashpoint.sqlite to local storage for faster access @@ -147,8 +133,8 @@ export const config = { localDbPath: '/app/data/flashpoint.sqlite', // Only used when enableLocalDbCopy is true // SQLite Performance Tuning - sqliteCacheSize: parseInt(process.env.SQLITE_CACHE_SIZE || '-64000', 10), // Negative = KB, -64000 = 64MB - sqliteMmapSize: parseInt(process.env.SQLITE_MMAP_SIZE || '268435456', 10), // 256MB default + sqliteCacheSize: parseInt(process.env.SQLITE_CACHE_SIZE || '', 10) || -64000, // Negative = KB, -64000 = 64MB + sqliteMmapSize: parseInt(process.env.SQLITE_MMAP_SIZE || '', 10) || 268435456, // 256MB default // Cache Pre-warming enableCachePrewarm: process.env.ENABLE_CACHE_PREWARM !== 'false', // Enabled by default diff --git a/game-service/src/cgi/cgi-executor.ts b/backend/src/game/cgi/cgi-executor.ts similarity index 77% rename from game-service/src/cgi/cgi-executor.ts rename to backend/src/game/cgi/cgi-executor.ts index e9a1a71..792e886 100644 --- a/game-service/src/cgi/cgi-executor.ts +++ b/backend/src/game/cgi/cgi-executor.ts @@ -1,7 +1,7 @@ import { spawn } from 'child_process'; import path from 'path'; import fs from 'fs/promises'; -import { logger } from '../utils/logger'; +import { logger } from '../../utils/logger'; import { sanitizeAndValidatePath } from '../utils/pathSecurity'; /** @@ -42,24 +42,39 @@ export interface CgiExecutorConfig { } /** - * Sensitive environment variables that should NOT be passed to CGI scripts + * Allowed environment variables to pass to CGI scripts */ -const SENSITIVE_ENV_VARS = [ - 'JWT_SECRET', - 'DATABASE_URL', - 'DB_PASSWORD', - 'API_KEY', - 'SECRET_KEY', - 'PRIVATE_KEY', - 'AWS_SECRET', - 'AZURE_SECRET', - 'GCP_SECRET', - 'ENCRYPTION_KEY', - 'PASSWORD', - 'TOKEN', - 'CREDENTIAL', +const ALLOWED_ENV_VARS = [ + 'PATH', + 'TEMP', + 'TMP', + 'SYSTEMROOT', + 'COMSPEC', + 'WINDIR', + 'HOME', + 'LANG', + 'LC_ALL', ]; +/** + * Allowed HTTP headers to pass to CGI scripts + */ +const ALLOWED_CGI_HEADERS = new Set([ + 'host', + 'user-agent', + 'accept', + 'accept-language', + 'accept-encoding', + 'accept-charset', + 'cookie', + 'referer', + 'origin', + 'if-modified-since', + 'if-none-match', + 'cache-control', + 'pragma', +]); + /** * CGI Executor - Executes PHP scripts via php-cgi * @@ -96,8 +111,8 @@ export class CgiExecutor { * @throws Error if execution fails or times out */ async execute(scriptPath: string, request: CgiRequest): Promise { - // Validate script path is within allowed directories - this.validateScriptPath(scriptPath); + // Validate script path is within allowed directories (resolves symlinks) + await this.validateScriptPath(scriptPath); // Build CGI environment variables const env = this.buildCgiEnvironment(scriptPath, request); @@ -106,8 +121,8 @@ export class CgiExecutor { logger.debug(`[CGI] Method: ${request.method}, Query: ${request.url.search}`); return new Promise((resolve, reject) => { - // Spawn php-cgi process - const proc = spawn(this.config.phpCgiPath, [], { + // Spawn php-cgi process with script path as explicit argument (CVE-2012-1823 mitigation) + const proc = spawn(this.config.phpCgiPath, [scriptPath], { cwd: path.dirname(scriptPath), env: { ...this.sanitizeProcessEnv(), ...env }, stdio: ['pipe', 'pipe', 'pipe'], @@ -221,18 +236,26 @@ export class CgiExecutor { /** * Validate that the script path is within allowed directories + * Resolves symlinks to prevent bypass attacks */ - private validateScriptPath(scriptPath: string): void { + private async validateScriptPath(scriptPath: string): Promise { + let realPath: string; + try { + realPath = await fs.realpath(scriptPath); + } catch { + throw new Error('CGI script not found'); + } + // Script must be in either htdocs or cgi-bin try { - sanitizeAndValidatePath(this.config.documentRoot, scriptPath); + sanitizeAndValidatePath(this.config.documentRoot, realPath); return; } catch { // Not in htdocs, try cgi-bin } try { - sanitizeAndValidatePath(this.config.cgiBinPath, scriptPath); + sanitizeAndValidatePath(this.config.cgiBinPath, realPath); return; } catch { // Not in cgi-bin either @@ -242,6 +265,30 @@ export class CgiExecutor { throw new Error('CGI script path is not within allowed directories'); } + /** + * Sanitize query string to prevent CVE-2012-1823 argument injection. + * Rejects query strings that contain segments starting with '-' without '=' + * which php-cgi may interpret as command-line arguments. + */ + private sanitizeQueryString(queryString: string): string { + if (!queryString) return ''; + // Split on & and check each parameter + const params = queryString.split('&'); + const safeParams = params.filter((param) => { + // If a parameter contains '=', it's a key=value pair (safe) + if (param.includes('=')) return true; + // If it starts with '-', it could be interpreted as a CLI flag (reject) + if (param.startsWith('-') || param.startsWith('%2d') || param.startsWith('%2D')) { + logger.warn( + `[CGI] Rejected potential argument injection in query string: ${param.substring(0, 20)}` + ); + return false; + } + return true; + }); + return safeParams.join('&'); + } + /** * Build CGI environment variables according to RFC 3875 */ @@ -249,6 +296,10 @@ export class CgiExecutor { const url = request.url; const headers = request.headers; + // Sanitize query string to prevent argument injection (CVE-2012-1823) + const rawQueryString = url.search.startsWith('?') ? url.search.substring(1) : url.search; + const sanitizedQueryString = this.sanitizeQueryString(rawQueryString); + // Build base CGI environment const env: Record = { // Required by PHP-CGI security (prevents direct execution attacks) @@ -268,7 +319,7 @@ export class CgiExecutor { SCRIPT_FILENAME: scriptPath, PATH_INFO: '', PATH_TRANSLATED: scriptPath, - QUERY_STRING: url.search.startsWith('?') ? url.search.substring(1) : url.search, + QUERY_STRING: sanitizedQueryString, // Document root DOCUMENT_ROOT: this.config.documentRoot, @@ -286,8 +337,15 @@ export class CgiExecutor { // Convert HTTP headers to CGI HTTP_* variables for (const [key, value] of Object.entries(headers)) { + const lowerKey = key.toLowerCase(); + // Skip content headers (handled separately) - if (key.toLowerCase() === 'content-type' || key.toLowerCase() === 'content-length') { + if (lowerKey === 'content-type' || lowerKey === 'content-length') { + continue; + } + + // Only pass allowed headers + if (!ALLOWED_CGI_HEADERS.has(lowerKey)) { continue; } @@ -300,25 +358,18 @@ export class CgiExecutor { } /** - * Remove sensitive environment variables from process.env + * Pass only allowed environment variables to CGI scripts */ private sanitizeProcessEnv(): Record { - const sanitized: Record = {}; + const env: Record = {}; - for (const [key, value] of Object.entries(process.env)) { - if (value === undefined) continue; - - // Check if key matches any sensitive pattern - const isSensitive = SENSITIVE_ENV_VARS.some( - (pattern) => key.toUpperCase().includes(pattern) || key.toUpperCase().startsWith(pattern) - ); - - if (!isSensitive) { - sanitized[key] = value; + for (const key of ALLOWED_ENV_VARS) { + if (process.env[key]) { + env[key] = process.env[key]!; } } - return sanitized; + return env; } /** diff --git a/game-service/src/cgi/index.ts b/backend/src/game/cgi/index.ts similarity index 100% rename from game-service/src/cgi/index.ts rename to backend/src/game/cgi/index.ts diff --git a/game-service/src/config.ts b/backend/src/game/config.ts similarity index 62% rename from game-service/src/config.ts rename to backend/src/game/config.ts index 585daab..ffd31cf 100644 --- a/game-service/src/config.ts +++ b/backend/src/game/config.ts @@ -1,6 +1,13 @@ import path from 'path'; import fs from 'fs/promises'; -import { logger } from './utils/logger'; +import { logger } from '../utils/logger'; +import { z } from 'zod'; + +/** + * Maximum file size for buffering in memory (50MB) + * Used consistently across gamezipserver, zip-manager, and legacy-server + */ +export const MAX_BUFFERED_FILE_SIZE = 50 * 1024 * 1024; export interface ServerSettings { // Proxy server port (default: 22500) @@ -24,10 +31,6 @@ export interface ServerSettings { mad4fpEnabled: boolean; mad4fpPaths: string[]; - // Performance settings - chunkSize: number; - maxConcurrentDownloads: number; - // Feature flags allowCrossDomain: boolean; enableBrotli: boolean; @@ -40,6 +43,23 @@ export interface ServerSettings { cgiMaxResponseSize: number; } +// Zod schema for proxySettings.json validation +const proxySettingsSchema = z + .object({ + legacyOverridePaths: z.array(z.string()).default([]), + externalFilePaths: z.array(z.string()).default([]), + legacyHTDOCSPath: z.string().optional(), + infinityServerURL: z.string().optional(), + mad4fpEnabled: z.boolean().optional(), + mad4fpPaths: z.array(z.string()).optional(), + enableCGI: z.boolean().default(false), + phpCgiPath: z.string().optional(), + cgiTimeout: z.number().default(30000), + cgiMaxBodySize: z.number().optional(), + cgiMaxResponseSize: z.number().optional(), + }) + .passthrough(); + export class ConfigManager { private static settings: ServerSettings | null = null; @@ -47,10 +67,27 @@ export class ConfigManager { * Load configuration from proxySettings.json and environment */ static async loadConfig(flashpointPath: string): Promise { + if (ConfigManager.settings) { + logger.warn('[ConfigManager] Re-initializing configuration - this may indicate a bug'); + } + try { const proxySettingsPath = path.join(flashpointPath, 'Server', 'proxySettings.json'); const data = await fs.readFile(proxySettingsPath, 'utf-8'); - const proxySettings = JSON.parse(data); + const parsed = JSON.parse(data); + + // Validate with Zod schema + let proxySettings; + try { + proxySettings = proxySettingsSchema.parse(parsed); + logger.info('[ConfigManager] Configuration validated successfully'); + } catch (error) { + if (error instanceof z.ZodError) { + logger.error('[ConfigManager] Schema validation failed:', error.errors); + throw new Error('Invalid proxySettings.json structure'); + } + throw error; + } // Build settings from proxySettings.json and defaults // Ports hardcoded - use docker-compose port mapping to expose on different ports @@ -75,9 +112,6 @@ export class ConfigManager { mad4fpEnabled: proxySettings.mad4fpEnabled || false, mad4fpPaths: proxySettings.mad4fpPaths || [], - chunkSize: 64 * 1024, // 64KB chunks for streaming - maxConcurrentDownloads: 10, - allowCrossDomain: true, enableBrotli: true, enableCGI: process.env.ENABLE_CGI === 'true', @@ -95,16 +129,22 @@ export class ConfigManager { cgiMaxResponseSize: proxySettings.cgiMaxResponseSize || 50 * 1024 * 1024, // 50MB }; - logger.info('[ProxyConfig] Configuration loaded successfully'); - logger.info(`[ProxyConfig] Proxy port: ${this.settings.proxyPort}`); - logger.info(`[ProxyConfig] GameZip port: ${this.settings.gameZipPort}`); - logger.info(`[ProxyConfig] HTDOCS path: ${this.settings.legacyHTDOCSPath}`); - logger.info(`[ProxyConfig] Override paths: ${this.settings.legacyOverridePaths.length}`); - logger.info(`[ProxyConfig] External sources: ${this.settings.externalFilePaths.length}`); + logger.info('[ConfigManager] Configuration loaded successfully'); + logger.info(`[ConfigManager] Proxy port: ${this.settings.proxyPort}`); + logger.info(`[ConfigManager] GameZip port: ${this.settings.gameZipPort}`); + logger.info(`[ConfigManager] HTDOCS path: ${this.settings.legacyHTDOCSPath}`); + logger.info(`[ConfigManager] Override paths: ${this.settings.legacyOverridePaths.length}`); + logger.info(`[ConfigManager] External sources: ${this.settings.externalFilePaths.length}`); return this.settings; - } catch (error) { - logger.error('[ProxyConfig] Failed to load proxySettings.json, using defaults', error); + } catch (error: unknown) { + // Differentiate between file not found and JSON parse errors + if (error && typeof error === 'object' && 'code' in error && error.code === 'ENOENT') { + logger.warn('[ConfigManager] proxySettings.json not found, using defaults'); + } else { + const errorMsg = error instanceof Error ? error.message : 'Unknown error'; + logger.error('[ConfigManager] Failed to parse proxySettings.json:', errorMsg); + } // Fallback to defaults this.settings = { @@ -120,8 +160,6 @@ export class ConfigManager { ], mad4fpEnabled: false, mad4fpPaths: [], - chunkSize: 64 * 1024, - maxConcurrentDownloads: 10, allowCrossDomain: true, enableBrotli: true, enableCGI: false, diff --git a/backend/src/game/gamezipserver.ts b/backend/src/game/gamezipserver.ts new file mode 100644 index 0000000..49eae22 --- /dev/null +++ b/backend/src/game/gamezipserver.ts @@ -0,0 +1,614 @@ +import http from 'http'; +import path from 'path'; +import fs from 'fs/promises'; +import { logger } from '../utils/logger'; +import { zipManager } from './zip-manager'; +import { ConfigManager, MAX_BUFFERED_FILE_SIZE } from './config'; +import { getMimeType } from './mimeTypes'; +import { injectPolyfills } from './utils/htmlInjector'; +import { setCorsHeaders } from './utils/cors'; +import { sanitizeUrlPath, sanitizeErrorMessage } from './utils/pathSecurity'; +import { validateGameId, validateHostname } from './validation/schemas'; +import { gameDataDownloader } from './services'; + +/** Escape HTML special characters to prevent XSS in loading page */ +function escapeHtml(str: string): string { + return str + .replace(/&/g, '&') + .replace(//g, '>') + .replace(/"/g, '"') + .replace(/'/g, ''') + .replace(/`/g, '`'); +} + +/** + * Download progress tracking for in-progress downloads + * Used to show loading page while game is being downloaded + */ +interface DownloadProgress { + startTime: number; + progress: number; + total: number | null; + source: string; + zipPath: string; +} + +/** Map of gameId -> download progress for in-progress downloads */ +const downloadsInProgress = new Map(); + +/** Maximum concurrent downloads */ +const MAX_CONCURRENT_DOWNLOADS = 3; + +/** Cleanup stale downloads older than 10 minutes */ +const DOWNLOAD_STALE_MS = 10 * 60 * 1000; + +/** + * GameZip Server - Direct-call interface for mounting/serving files from ZIP archives + * Refactored from game-service to expose methods instead of running an HTTP server + */ +export class GameZipServer { + private cleanupInterval: NodeJS.Timeout | null = null; + + constructor() { + // Start periodic cleanup of stale downloads (every 60 seconds) + this.cleanupInterval = setInterval(() => { + this.cleanupStaleDownloads(); + }, 60000); + this.cleanupInterval.unref(); // Don't prevent graceful shutdown + } + + /** + * Dispose of the server and stop cleanup intervals + */ + dispose(): void { + if (this.cleanupInterval) { + clearInterval(this.cleanupInterval); + this.cleanupInterval = null; + } + } + + /** + * Mount a ZIP file directly (no HTTP) + * Returns { success, downloading?, statusCode } + */ + async mountZip(params: { + id: string; + zipPath: string; + gameId?: string; + dateAdded?: string; + sha256?: string; + }): Promise<{ success: boolean; downloading?: boolean; statusCode: number }> { + const { id, zipPath, gameId, dateAdded, sha256 } = params; + + // Validate mount ID using Zod schema + try { + validateGameId(id); + } catch (error) { + logger.warn(`[Security] Invalid mount ID detected: ${id}`); + return { success: false, statusCode: 400 }; + } + + if (!zipPath) { + return { success: false, statusCode: 400 }; + } + + // Validate ZIP path to prevent directory traversal + const flashpointPath = + process.env.FLASHPOINT_PATH || + (process.env.NODE_ENV === 'production' ? '/data/flashpoint' : 'D:/Flashpoint'); + const allowedGamesPath = + process.env.FLASHPOINT_GAMES_PATH || path.join(flashpointPath, 'Data', 'Games'); + + try { + const normalizedZipPath = path.normalize(zipPath); + const resolvedZipPath = path.resolve(normalizedZipPath); + const resolvedGamesPath = path.resolve(allowedGamesPath); + + if ( + !resolvedZipPath.startsWith(resolvedGamesPath + path.sep) && + resolvedZipPath !== resolvedGamesPath + ) { + logger.warn(`[Security] ZIP path outside allowed directory: ${zipPath}`); + return { success: false, statusCode: 403 }; + } + } catch (error) { + logger.error('[Security] ZIP path validation error:', error); + return { success: false, statusCode: 400 }; + } + + // Check if ZIP file exists locally + let zipExists = false; + try { + await fs.access(zipPath); + zipExists = true; + } catch { + zipExists = false; + } + + // If ZIP doesn't exist and we have download parameters, start background download + if (!zipExists && gameId && dateAdded) { + if (downloadsInProgress.has(gameId)) { + logger.info(`[GameZipServer] Download already in progress for game ${gameId}`); + return { success: true, downloading: true, statusCode: 202 }; + } + + // Check if we've hit the concurrent download limit + if (downloadsInProgress.size >= MAX_CONCURRENT_DOWNLOADS) { + logger.warn( + `[GameZipServer] Too many concurrent downloads (${downloadsInProgress.size}), rejecting ${gameId}` + ); + return { success: false, statusCode: 503 }; + } + + logger.info( + `[GameZipServer] ZIP not found locally, starting background download for game ${gameId}` + ); + + downloadsInProgress.set(gameId, { + startTime: Date.now(), + progress: 0, + total: null, + source: 'Starting...', + zipPath, + }); + + this.downloadAndMountInBackground( + id, + gameId, + dateAdded, + sha256, + zipPath, + allowedGamesPath + ).catch((err) => logger.error('[GameZipServer] Background download failed:', err)); + + return { success: true, downloading: true, statusCode: 202 }; + } else if (!zipExists) { + return { success: false, statusCode: 404 }; + } + + // Mount the ZIP (file exists locally) + try { + await zipManager.mount(id, zipPath); + + // After the mount succeeds, verify the real path hasn't changed (TOCTOU protection) + try { + const realPath = await fs.realpath(zipPath); + const resolvedReal = path.resolve(realPath); + const resolvedGamesPath = path.resolve(allowedGamesPath); + if ( + !resolvedReal.startsWith(resolvedGamesPath + path.sep) && + resolvedReal !== resolvedGamesPath + ) { + // Symlink pointed outside allowed directory - unmount immediately + logger.warn(`[GameZipServer] Symlink detected: ${zipPath} -> ${realPath}. Unmounting.`); + await zipManager.unmount(id); + return { success: false, statusCode: 403 }; + } + } catch (realPathError) { + logger.warn(`[GameZipServer] Failed to resolve real path for ${zipPath}:`, realPathError); + // Continue - the file was already successfully opened by zipManager + } + + return { success: true, statusCode: 200 }; + } catch (error) { + const errorMessage = error instanceof Error ? error.message : 'Mount failed'; + const safeMessage = sanitizeErrorMessage(errorMessage); + logger.error(`[GameZipServer] Mount failed: ${safeMessage}`); + return { success: false, statusCode: 500 }; + } + } + + /** + * Unmount a ZIP file directly (no HTTP) + */ + async unmountZip(id: string): Promise { + try { + validateGameId(id); + } catch (error) { + logger.warn(`[Security] Invalid mount ID detected: ${id}`); + return false; + } + return await zipManager.unmount(id); + } + + /** + * List all mounted ZIPs + */ + listMounts(): Array<{ id: string; zipPath: string; mountTime: Date; fileCount: number }> { + return zipManager.getMountedZips(); + } + + /** + * Handle file request from mounted ZIPs (compatible with both raw HTTP and Express) + * Supports proxy-style requests: GET http://domain.com/path HTTP/1.1 + */ + async handleFileRequest(req: http.IncomingMessage, res: http.ServerResponse): Promise { + if (!req.url) { + this.sendError(res, 400, 'Bad Request: No URL'); + return; + } + + let hostname: string; + let urlPath: string; + + // Normalize malformed URLs: /http:/domain → /http://domain + // This is a common pattern in legacy Flash games + let normalizedUrl = req.url; + if (normalizedUrl.match(/^\/https?:\/[^\/]/)) { + // Single slash after protocol - insert the missing slash + normalizedUrl = normalizedUrl.replace(/^(\/https?:)\//, '$1//'); + logger.debug(`[GameZipServer] Normalized malformed URL: ${req.url} → ${normalizedUrl}`); + } + + // Parse URL (same logic as main proxy server) + try { + if (normalizedUrl.startsWith('http://') || normalizedUrl.startsWith('https://')) { + // Proxy-style: GET http://domain.com/path HTTP/1.1 + const targetUrl = new URL(normalizedUrl); + hostname = targetUrl.hostname; + urlPath = targetUrl.pathname + targetUrl.search; + } else if (normalizedUrl.startsWith('/http://') || normalizedUrl.startsWith('/https://')) { + // Path-based: GET /http://domain.com/path HTTP/1.1 + const urlWithoutSlash = normalizedUrl.substring(1); + const targetUrl = new URL(urlWithoutSlash); + hostname = targetUrl.hostname; + urlPath = targetUrl.pathname + targetUrl.search; + } else { + // Regular: GET /path HTTP/1.1 + // Strip port from hostname (e.g., "localhost:3100" -> "localhost") + hostname = (req.headers.host || 'localhost').split(':')[0]; + urlPath = normalizedUrl; + } + } catch (error) { + logger.error(`[GameZipServer] Invalid URL: ${normalizedUrl}`); + this.sendError(res, 400, 'Invalid URL format'); + return; + } + + // Validate hostname using Zod schema + try { + hostname = validateHostname(hostname); + } catch (error) { + logger.error(`[GameZipServer] Invalid hostname: ${hostname}`, error); + this.sendError(res, 400, 'Invalid hostname'); + return; + } + + // Sanitize URL path to prevent null bytes and dangerous patterns + try { + urlPath = sanitizeUrlPath(urlPath); + } catch (error) { + logger.error(`[GameZipServer] Invalid URL path: ${urlPath}`); + this.sendError(res, 400, 'Invalid URL path'); + return; + } + + // Strip query string from path for ZIP file lookup + // Files in ZIPs don't have query strings (e.g., file.swf?token=xyz → file.swf) + const pathWithoutQuery = urlPath.split('?')[0]; + + // Build relative path + const relPath = path.posix.join(hostname, pathWithoutQuery); + + logger.debug(`[GameZipServer] Looking for: ${relPath}`); + + // Search mounted ZIPs + const result = await zipManager.findFile(relPath); + + if (!result) { + // Check if a download is in progress for any game + // The mount ID is typically the gameId + const downloadStatus = this.findActiveDownload(relPath); + + if (downloadStatus) { + logger.info( + `[GameZipServer] File not ready, download in progress for ${downloadStatus.gameId}` + ); + this.sendLoadingPage(res, downloadStatus.gameId, downloadStatus.progress); + return; + } + + logger.debug(`[GameZipServer] File not found in any mounted ZIP: ${relPath}`); + this.sendError(res, 404, 'File not found in mounted ZIPs'); + return; + } + + // Determine content type (use path without query string) + const ext = path.extname(pathWithoutQuery).substring(1).toLowerCase(); + const contentType = getMimeType(ext); + + // Process HTML files to inject polyfills for better game compatibility + let fileData = result.data; + if (ext === 'html' || ext === 'htm') { + fileData = injectPolyfills(result.data); + logger.info(`[GameZipServer] Injected polyfills into HTML file: ${relPath}`); + } + + // Send file + logger.info( + `[GameZipServer] ✓ Serving from ZIP ${result.mountId}: ${relPath} (${fileData.length} bytes)` + ); + + // Get settings for CORS configuration + const settings = ConfigManager.getSettings(); + + // CORS headers + setCorsHeaders(res, settings); + + res.setHeader('Content-Type', contentType); + res.setHeader('Content-Length', fileData.length); + res.setHeader('Cache-Control', 'public, max-age=86400'); // Cache for 24 hours + if (process.env.NODE_ENV !== 'production') { + res.setHeader('X-Source', `gamezipserver:${result.mountId}`); + } + res.setHeader('Connection', 'keep-alive'); + + res.writeHead(200); + res.end(fileData); + } + + /** + * Download ZIP and mount it in background + * This runs asynchronously - errors are logged but don't affect the HTTP response + */ + private async downloadAndMountInBackground( + mountId: string, + gameId: string, + dateAdded: string, + sha256: string | undefined, + zipPath: string, + targetPath: string + ): Promise { + try { + const result = await gameDataDownloader.download( + { + gameId, + dateAdded, + sha256, + targetPath, + }, + (downloaded, total, source) => { + try { + // Update progress tracking + const existing = downloadsInProgress.get(gameId); + if (existing) { + downloadsInProgress.set(gameId, { + ...existing, + progress: downloaded, + total, + source, + }); + } + const progress = total ? Math.round((downloaded / total) * 100) : 0; + logger.debug( + `[GameZipServer] Download progress for ${gameId}: ${progress}% from ${source}` + ); + } catch (error) { + logger.error(`[GameZipServer] Progress callback error for ${gameId}:`, error); + } + } + ); + + if (result.success && result.filePath) { + logger.info(`[GameZipServer] Successfully downloaded ZIP from "${result.sourceUsed}"`); + + // Validate downloaded file path before mounting + const resolvedFilePath = path.resolve(result.filePath); + const resolvedAllowedPath = path.resolve(targetPath); + if ( + !resolvedFilePath.startsWith(resolvedAllowedPath + path.sep) && + resolvedFilePath !== resolvedAllowedPath + ) { + logger.error( + `[GameZipServer] Downloaded file path outside allowed directory: ${result.filePath}` + ); + return; + } + + // Mount the downloaded ZIP + try { + await zipManager.mount(mountId, result.filePath); + logger.info(`[GameZipServer] ✓ ZIP downloaded and mounted for game ${gameId}`); + } catch (mountError) { + const errorMessage = mountError instanceof Error ? mountError.message : 'Mount failed'; + logger.error(`[GameZipServer] Failed to mount downloaded ZIP: ${errorMessage}`); + } + } else { + logger.error(`[GameZipServer] Download failed for ${gameId}: ${result.error}`); + } + } catch (error) { + const errorMsg = error instanceof Error ? error.message : 'Unknown error'; + logger.error(`[GameZipServer] Background download error for ${gameId}: ${errorMsg}`); + } finally { + // Remove from in-progress tracking + downloadsInProgress.delete(gameId); + } + } + + /** + * Clean up stale downloads older than DOWNLOAD_STALE_MS + */ + private cleanupStaleDownloads(): void { + const now = Date.now(); + for (const [gameId, progress] of downloadsInProgress.entries()) { + if (now - progress.startTime > DOWNLOAD_STALE_MS) { + logger.warn(`[GameZipServer] Removing stale download for ${gameId}`); + downloadsInProgress.delete(gameId); + } + } + } + + /** + * Find an active download that might be related to the requested file path + * Returns the download status if found, null otherwise + */ + private findActiveDownload( + relPath: string + ): { gameId: string; progress: DownloadProgress } | null { + // Clean up stale downloads first + this.cleanupStaleDownloads(); + + // Extract hostname from relPath (format: hostname/path/to/file) + const hostname = relPath.split('/')[0]?.toLowerCase(); + + // Try to find a download matching this hostname + if (hostname) { + for (const [gameId, download] of downloadsInProgress.entries()) { + // Check if mount ID or ZIP path contains the hostname + if (gameId.toLowerCase().includes(hostname)) { + return { gameId, progress: download }; + } + // Also check the ZIP path for hostname match + const zipFilename = download.zipPath.toLowerCase(); + if (zipFilename.includes(hostname)) { + return { gameId, progress: download }; + } + } + } + + // No matching download found - return null for a proper 404 + return null; + } + + /** + * Send a loading page while game is being downloaded + * Uses auto-refresh to poll until game is ready + */ + private sendLoadingPage( + res: http.ServerResponse, + gameId: string, + progress: DownloadProgress + ): void { + if (res.headersSent) { + res.end(); + return; + } + + const progressPercent = progress.total + ? Math.round((progress.progress / progress.total) * 100) + : 0; + + const progressMB = (progress.progress / (1024 * 1024)).toFixed(1); + const totalMB = progress.total ? (progress.total / (1024 * 1024)).toFixed(1) : '?'; + + const elapsedSec = Math.round((Date.now() - progress.startTime) / 1000); + + const html = ` + + + + + + Downloading Game... + + + +
+ +

Downloading Game Data...

+
+
+
+

${escapeHtml(progressMB)} MB / ${escapeHtml(totalMB)} MB (${escapeHtml(String(progressPercent))}%)

+

Downloading from configured source...

+

Elapsed: ${escapeHtml(String(elapsedSec))}s · Page refreshes automatically

+
+ +`; + + const settings = ConfigManager.getSettings(); + setCorsHeaders(res, settings); + res.setHeader('Content-Type', 'text/html; charset=utf-8'); + res.setHeader('Cache-Control', 'no-cache, no-store, must-revalidate'); + res.setHeader( + 'Content-Security-Policy', + "default-src 'none'; style-src 'unsafe-inline'; img-src 'none'; script-src 'none'" + ); + res.writeHead(202); // 202 Accepted - still processing + res.end(html); + } + + /** + * Send error response + */ + private sendError(res: http.ServerResponse, statusCode: number, message: string): void { + if (res.headersSent) { + res.end(); + return; + } + + logger.warn(`[GameZipServer] Sending error ${statusCode}: ${message}`); + + // Set CORS headers even for error responses + const settings = ConfigManager.getSettings(); + setCorsHeaders(res, settings); + + res.setHeader('Content-Type', 'text/plain'); + res.writeHead(statusCode); + res.end(message); + } +} + +export const gameZipServer = new GameZipServer(); diff --git a/game-service/src/legacy-server.ts b/backend/src/game/legacy-server.ts similarity index 69% rename from game-service/src/legacy-server.ts rename to backend/src/game/legacy-server.ts index 260a59c..d803c3e 100644 --- a/game-service/src/legacy-server.ts +++ b/backend/src/game/legacy-server.ts @@ -2,8 +2,8 @@ import path from 'path'; import fs from 'fs/promises'; import { createReadStream } from 'fs'; import { Readable } from 'stream'; -import { logger } from './utils/logger'; -import { ConfigManager } from './config'; +import { logger } from '../utils/logger'; +import { ConfigManager, MAX_BUFFERED_FILE_SIZE } from './config'; import { getMimeType } from './mimeTypes'; import { sanitizeAndValidatePath, @@ -11,16 +11,38 @@ import { sanitizeErrorMessage, } from './utils/pathSecurity'; import { CgiExecutor, CgiRequest } from './cgi'; -import axios, { AxiosError } from 'axios'; +import axios from 'axios'; import zlib from 'zlib'; import { promisify } from 'util'; +import http from 'http'; +import https from 'https'; + +// Shared HTTP agents with connection pool limits to prevent exhaustion +const httpAgent = new http.Agent({ + keepAlive: true, + maxSockets: 10, // Max 10 concurrent connections per host + maxFreeSockets: 2, + timeout: 30000, +}); + +const httpsAgent = new https.Agent({ + keepAlive: true, + maxSockets: 10, + maxFreeSockets: 2, + timeout: 30000, +}); + +const axiosInstance = axios.create({ + httpAgent, + httpsAgent, +}); // HTTP request timeout constants (G-H2) -const EXTERNAL_REQUEST_TIMEOUT_MS = 45000; // 45 seconds for external sources +const EXTERNAL_REQUEST_TIMEOUT_MS = 15000; // 15 seconds for external sources const EXTERNAL_REQUEST_MAX_REDIRECTS = 5; // Retry configuration for external requests -const MAX_RETRIES = 3; +const MAX_RETRIES = 2; const RETRY_BASE_DELAY_MS = 1000; // 1 second base delay const brotliDecompress = promisify(zlib.brotliDecompress); @@ -28,7 +50,9 @@ const brotliDecompress = promisify(zlib.brotliDecompress); // Script file extensions const SCRIPT_EXTENSIONS = ['php', 'php5', 'phtml', 'pl']; -// Index file extensions (in priority order) +// Index files that legacy Flashpoint games commonly use as entry points. +// These are checked when a directory is requested without a specific file. +// Note: 'swf' is included because many Flash game directories have index.swf as the entry point. const INDEX_EXTENSIONS = ['html', 'htm', 'php', 'swf']; // Common subdomain prefixes used in Flashpoint content @@ -69,15 +93,54 @@ export interface HttpRequestContext { } export class LegacyServer { - private settings = ConfigManager.getSettings(); private cgiExecutor: CgiExecutor | null = null; private cgiValidated = false; + // Negative lookup cache to avoid repeated fs.stat() calls for missing files + private static readonly negativeLookupCache = new Map(); + private static readonly NEGATIVE_CACHE_TTL_MS = 30000; // 30 seconds + private static readonly MAX_NEGATIVE_CACHE_SIZE = 5000; + + /** + * Clear the negative lookup cache (called during shutdown) + */ + static clearCache(): void { + LegacyServer.negativeLookupCache.clear(); + logger.info('[LegacyServer] Negative lookup cache cleared'); + } + + /** + * Check if a path is in the negative lookup cache + */ + private isNegativelyCached(fullPath: string): boolean { + const cachedAt = LegacyServer.negativeLookupCache.get(fullPath); + if (cachedAt === undefined) return false; + if (Date.now() - cachedAt > LegacyServer.NEGATIVE_CACHE_TTL_MS) { + LegacyServer.negativeLookupCache.delete(fullPath); + return false; + } + return true; + } + + /** + * Cache a negative lookup (file not found) with TTL and size limit + */ + private cacheNegativeLookup(fullPath: string): void { + // Evict oldest entries if cache is full + if (LegacyServer.negativeLookupCache.size >= LegacyServer.MAX_NEGATIVE_CACHE_SIZE) { + const firstKey = LegacyServer.negativeLookupCache.keys().next().value; + if (firstKey) LegacyServer.negativeLookupCache.delete(firstKey); + } + LegacyServer.negativeLookupCache.set(fullPath, Date.now()); + } + /** * Initialize CGI executor if enabled */ private async initCgiExecutor(): Promise { - if (!this.settings.enableCGI) { + const settings = ConfigManager.getSettings(); + + if (!settings.enableCGI) { return null; } @@ -86,12 +149,12 @@ export class LegacyServer { } this.cgiExecutor = new CgiExecutor({ - phpCgiPath: this.settings.phpCgiPath, - documentRoot: this.settings.legacyHTDOCSPath, - cgiBinPath: this.settings.legacyCGIBINPath, - timeout: this.settings.cgiTimeout, - maxBodySize: this.settings.cgiMaxBodySize, - maxResponseSize: this.settings.cgiMaxResponseSize, + phpCgiPath: settings.phpCgiPath, + documentRoot: settings.legacyHTDOCSPath, + cgiBinPath: settings.legacyCGIBINPath, + timeout: settings.cgiTimeout, + maxBodySize: settings.cgiMaxBodySize, + maxResponseSize: settings.cgiMaxResponseSize, }); // Validate binary exists (only once) @@ -118,6 +181,9 @@ export class LegacyServer { urlPath: string, requestContext?: HttpRequestContext ): Promise { + // Get fresh settings at the start of the request + const settings = ConfigManager.getSettings(); + // Sanitize URL path to prevent null bytes and dangerous patterns try { urlPath = sanitizeUrlPath(urlPath); @@ -132,36 +198,49 @@ export class LegacyServer { // Build relative path: {host}/{path} const relPath = path.posix.join(host, urlPath); - logger.info(`[LegacyServer] Serving: ${host}${urlPath}`); - logger.info(`[LegacyServer] Relative path: ${relPath}`); + logger.debug(`[LegacyServer] Serving: ${host}${urlPath}`); + logger.debug(`[LegacyServer] Relative path: ${relPath}`); // Step 1: Build all path candidates - const pathCandidates = this.buildPathCandidates(relPath, urlPath); + const pathCandidates = this.buildPathCandidates(relPath, urlPath, settings); // Step 2: Try each candidate in order for (const candidate of pathCandidates) { logger.debug(`[LegacyServer] Trying: ${candidate.path} (${candidate.type})`); + // Check negative cache to skip known-missing files + if (this.isNegativelyCached(candidate.path)) { + logger.debug(`[LegacyServer] Skipping cached negative lookup: ${candidate.path}`); + continue; + } + try { // Validate path is within allowed base directory before accessing - this.validateCandidatePath(candidate.path, candidate.type); + this.validateCandidatePath(candidate.path, candidate.type, settings); const stats = await fs.stat(candidate.path); if (stats.isFile()) { logger.info(`[LegacyServer] ✓ Found file: ${candidate.path}`); const isCgiBin = candidate.type === 'cgi-bin' || candidate.type === 'cgi-bin-no-query'; - return await this.serveFile(candidate.path, isCgiBin, requestContext); + return await this.serveFile(candidate.path, isCgiBin, requestContext, settings); } } catch (error) { - // File doesn't exist or path validation failed, continue to next candidate + // Only cache file-not-found errors, not security validation failures + if ( + error instanceof Error && + 'code' in error && + (error as NodeJS.ErrnoException).code === 'ENOENT' + ) { + this.cacheNegativeLookup(candidate.path); + } continue; } } // Step 3: No local file found, try external sources logger.info(`[LegacyServer] File not found locally, trying external sources...`); - return await this.tryExternalSources(relPath); + return await this.tryExternalSources(relPath, settings); } /** @@ -170,7 +249,8 @@ export class LegacyServer { */ private buildPathCandidates( relPath: string, - urlPath: string + urlPath: string, + settings: ReturnType ): Array<{ path: string; type: string }> { const candidates: Array<{ path: string; type: string }> = []; const seenPaths = new Set(); @@ -199,7 +279,7 @@ export class LegacyServer { // 1. Exact file paths (with and without query string) addCandidate( - path.join(this.settings.legacyHTDOCSPath, variantRelPath), + path.join(settings.legacyHTDOCSPath, variantRelPath), hostVariation === hostname ? 'exact' : `subdomain:${hostVariation}` ); @@ -207,7 +287,7 @@ export class LegacyServer { const pathWithoutQuery = variantRelPath.split('?')[0]; if (pathWithoutQuery !== variantRelPath) { addCandidate( - path.join(this.settings.legacyHTDOCSPath, pathWithoutQuery), + path.join(settings.legacyHTDOCSPath, pathWithoutQuery), hostVariation === hostname ? 'exact-no-query' : `subdomain:${hostVariation}-no-query` ); } @@ -217,15 +297,12 @@ export class LegacyServer { const pathWithoutQuery = relPath.split('?')[0]; // 2. Override paths (exact files in override directories) - for (const override of this.settings.legacyOverridePaths) { - addCandidate( - path.join(this.settings.legacyHTDOCSPath, override, relPath), - `override:${override}` - ); + for (const override of settings.legacyOverridePaths) { + addCandidate(path.join(settings.legacyHTDOCSPath, override, relPath), `override:${override}`); if (pathWithoutQuery !== relPath) { addCandidate( - path.join(this.settings.legacyHTDOCSPath, override, pathWithoutQuery), + path.join(settings.legacyHTDOCSPath, override, pathWithoutQuery), `override:${override}-no-query` ); } @@ -233,27 +310,21 @@ export class LegacyServer { // 3. CGI-BIN paths (for script files) if (this.isScriptUrl(urlPath)) { - addCandidate(path.join(this.settings.legacyCGIBINPath, relPath), 'cgi-bin'); + addCandidate(path.join(settings.legacyCGIBINPath, relPath), 'cgi-bin'); if (pathWithoutQuery !== relPath) { - addCandidate( - path.join(this.settings.legacyCGIBINPath, pathWithoutQuery), - 'cgi-bin-no-query' - ); + addCandidate(path.join(settings.legacyCGIBINPath, pathWithoutQuery), 'cgi-bin-no-query'); } } // 4. Index files (for directory requests) for (const ext of INDEX_EXTENSIONS) { - addCandidate( - path.join(this.settings.legacyHTDOCSPath, relPath, `index.${ext}`), - `index:${ext}` - ); + addCandidate(path.join(settings.legacyHTDOCSPath, relPath, `index.${ext}`), `index:${ext}`); // Index files in override paths - for (const override of this.settings.legacyOverridePaths) { + for (const override of settings.legacyOverridePaths) { addCandidate( - path.join(this.settings.legacyHTDOCSPath, override, relPath, `index.${ext}`), + path.join(settings.legacyHTDOCSPath, override, relPath, `index.${ext}`), `index-override:${override}:${ext}` ); } @@ -266,17 +337,22 @@ export class LegacyServer { * Validate that a candidate path is within its allowed base directory * @param candidatePath The full path to validate * @param candidateType The type of candidate (e.g., 'exact', 'cgi-bin', 'override:*') + * @param settings Current configuration settings * @throws Error if the path escapes its allowed base directory */ - private validateCandidatePath(candidatePath: string, candidateType: string): void { + private validateCandidatePath( + candidatePath: string, + candidateType: string, + settings: ReturnType + ): void { // Determine the allowed base directory based on candidate type let allowedBase: string; if (candidateType === 'cgi-bin' || candidateType === 'cgi-bin-no-query') { - allowedBase = this.settings.legacyCGIBINPath; + allowedBase = settings.legacyCGIBINPath; } else { // All other types use htdocs path - allowedBase = this.settings.legacyHTDOCSPath; + allowedBase = settings.legacyHTDOCSPath; } // Validate that the candidate path is within the allowed base @@ -302,37 +378,54 @@ export class LegacyServer { * @param filePath Absolute path to file * @param isCGI Whether to execute as CGI script * @param requestContext HTTP request context for CGI execution + * @param settings Current configuration settings */ private async serveFile( filePath: string, isCGI: boolean = false, - requestContext?: HttpRequestContext + requestContext?: HttpRequestContext, + settings?: ReturnType ): Promise { + // Get settings if not provided + if (!settings) { + settings = ConfigManager.getSettings(); + } // Determine content type first const ext = path.extname(filePath).substring(1).toLowerCase(); const contentType = getMimeType(ext); // HTML files need polyfill injection, so buffer them // Brotli-compressed files also need buffering for decompression - const needsBuffering = + let needsBuffering = contentType.includes('text/html') || - (this.settings.enableBrotli && filePath.endsWith('.br')) || + (settings.enableBrotli && filePath.endsWith('.br')) || isCGI; + // Get file stats for size check + const stat = await fs.stat(filePath); + + // If file is too large for buffering, skip polyfill injection and stream instead + if (needsBuffering && stat.size > MAX_BUFFERED_FILE_SIZE) { + logger.warn( + `[LegacyServer] File too large for buffering (${stat.size} bytes), streaming without polyfill injection: ${filePath}` + ); + needsBuffering = false; + } + if (needsBuffering) { // Read entire file into memory let data = await fs.readFile(filePath); // Handle Brotli compression - if (this.settings.enableBrotli && filePath.endsWith('.br')) { + if (settings.enableBrotli && filePath.endsWith('.br')) { logger.debug(`[LegacyServer] Decompressing Brotli file: ${filePath}`); data = await brotliDecompress(data); filePath = filePath.substring(0, filePath.length - 3); // Remove .br extension } // Execute CGI scripts if enabled - if (isCGI && this.settings.enableCGI) { - return await this.executeCgiScript(filePath, requestContext); + if (isCGI && settings.enableCGI) { + return await this.executeCgiScript(filePath, requestContext, settings); } return { @@ -343,7 +436,6 @@ export class LegacyServer { } // Non-HTML, non-compressed files: stream for memory efficiency - const stat = await fs.stat(filePath); const stream = createReadStream(filePath); logger.debug( @@ -362,30 +454,36 @@ export class LegacyServer { * Execute a CGI script and return the response * @param scriptPath Absolute path to the script file * @param requestContext HTTP request context (method, headers, body) + * @param settings Current configuration settings */ private async executeCgiScript( scriptPath: string, - requestContext?: HttpRequestContext + requestContext?: HttpRequestContext, + settings?: ReturnType ): Promise { + // Get settings if not provided + if (!settings) { + settings = ConfigManager.getSettings(); + } const executor = await this.initCgiExecutor(); if (!executor) { - logger.warn('[LegacyServer] CGI executor not available, serving script as static file'); - const data = await fs.readFile(scriptPath); + logger.warn('[LegacyServer] CGI executor not available for script'); return { - data, + data: Buffer.from('CGI service unavailable'), contentType: 'text/plain', - source: 'local-htdocs', + statusCode: 503, + source: 'cgi-error', }; } // Build URL from the script path for CGI environment // Extract the relative path from the script path let relativePath = scriptPath; - if (scriptPath.startsWith(this.settings.legacyHTDOCSPath)) { - relativePath = scriptPath.substring(this.settings.legacyHTDOCSPath.length); - } else if (scriptPath.startsWith(this.settings.legacyCGIBINPath)) { - relativePath = scriptPath.substring(this.settings.legacyCGIBINPath.length); + if (scriptPath.startsWith(settings.legacyHTDOCSPath)) { + relativePath = scriptPath.substring(settings.legacyHTDOCSPath.length); + } else if (scriptPath.startsWith(settings.legacyCGIBINPath)) { + relativePath = scriptPath.substring(settings.legacyCGIBINPath.length); } // Normalize path separators for URL relativePath = relativePath.replace(/\\/g, '/'); @@ -431,18 +529,21 @@ export class LegacyServer { /** * Try external sources (Infinity Server, MAD4FP, etc.) */ - private async tryExternalSources(relPath: string): Promise { + private async tryExternalSources( + relPath: string, + settings: ReturnType + ): Promise { // Try Infinity Server first - if (this.settings.infinityServerURL) { + if (settings.infinityServerURL) { logger.info(`[LegacyServer] Trying Infinity Server...`); - const result = await this.tryExternalSource(this.settings.infinityServerURL, relPath); + const result = await this.tryExternalSource(settings.infinityServerURL, relPath); if (result) { return { ...result, source: 'infinity-server' }; } } // Try other external paths - for (const externalPath of this.settings.externalFilePaths) { + for (const externalPath of settings.externalFilePaths) { logger.info(`[LegacyServer] Trying: ${externalPath}`); const result = await this.tryExternalSource(externalPath, relPath); if (result) { @@ -451,8 +552,8 @@ export class LegacyServer { } // Try MAD4FP if enabled - if (this.settings.mad4fpEnabled) { - for (const mad4fpPath of this.settings.mad4fpPaths) { + if (settings.mad4fpEnabled) { + for (const mad4fpPath of settings.mad4fpPaths) { logger.info(`[LegacyServer] Trying MAD4FP: ${mad4fpPath}`); const result = await this.tryExternalSource(mad4fpPath, relPath); if (result) { @@ -506,7 +607,18 @@ export class LegacyServer { relPath: string ): Promise<{ data: Buffer; contentType: string } | null> { const normalizedBase = baseUrl.endsWith('/') ? baseUrl : `${baseUrl}/`; - const fullUrl = `${normalizedBase}${relPath}`; + let fullUrl: string; + try { + const resolved = new URL(relPath, normalizedBase); + // Verify resolved URL still starts with base + if (!resolved.href.startsWith(normalizedBase)) { + logger.warn(`[LegacyServer] Path traversal blocked in external URL: ${relPath}`); + return null; + } + fullUrl = resolved.href; + } catch { + return null; + } logger.info(`[LegacyServer] Downloading: ${fullUrl}`); @@ -519,7 +631,7 @@ export class LegacyServer { }, EXTERNAL_REQUEST_TIMEOUT_MS); try { - const response = await axios.get(fullUrl, { + const response = await axiosInstance.get(fullUrl, { responseType: 'arraybuffer', timeout: EXTERNAL_REQUEST_TIMEOUT_MS, maxRedirects: EXTERNAL_REQUEST_MAX_REDIRECTS, @@ -533,6 +645,7 @@ export class LegacyServer { maxBodyLength: 100 * 1024 * 1024, }); + clearTimeout(timeoutId); const data = Buffer.from(response.data); // Determine content type from file extension, not from response header (G-H5) @@ -560,15 +673,14 @@ export class LegacyServer { // Log the error appropriately if (axios.isAxiosError(error)) { - const axiosError = error as AxiosError; - if (axiosError.response?.status === 404) { + if (error.response?.status === 404) { logger.debug(`[LegacyServer] ✗ Not found (404)`); - } else if (axiosError.response?.status === 403) { + } else if (error.response?.status === 403) { logger.warn(`[LegacyServer] ✗ Forbidden (403)`); } else if ( - axiosError.code === 'ECONNABORTED' || - axiosError.code === 'ETIMEDOUT' || - axiosError.code === 'ERR_CANCELED' + error.code === 'ECONNABORTED' || + error.code === 'ETIMEDOUT' || + error.code === 'ERR_CANCELED' ) { if (shouldRetry) { logger.warn( @@ -581,7 +693,7 @@ export class LegacyServer { } } else { // Sanitize error message to prevent internal details leakage - const safeMessage = sanitizeErrorMessage(axiosError.message || 'Unknown error'); + const safeMessage = sanitizeErrorMessage(error.message || 'Unknown error'); if (shouldRetry) { logger.warn( `[LegacyServer] ✗ Error: ${safeMessage} (attempt ${attempt}/${MAX_RETRIES})` @@ -631,9 +743,12 @@ export class LegacyServer { private getHostnameVariations(hostname: string): string[] { const variations: string[] = [hostname]; - // Add subdomain variations + // Add subdomain variations (skip if hostname already has that prefix) for (const subdomain of COMMON_SUBDOMAINS) { - variations.push(`${subdomain}.${hostname}`); + // Avoid creating nonsensical prefixes like "www.www.example.com" + if (!hostname.startsWith(`${subdomain}.`)) { + variations.push(`${subdomain}.${hostname}`); + } } return variations; diff --git a/game-service/src/mimeTypes.ts b/backend/src/game/mimeTypes.ts similarity index 90% rename from game-service/src/mimeTypes.ts rename to backend/src/game/mimeTypes.ts index 2193c87..ad73815 100644 --- a/game-service/src/mimeTypes.ts +++ b/backend/src/game/mimeTypes.ts @@ -3,7 +3,7 @@ * Includes 199 custom MIME types for legacy web content */ -export const CUSTOM_MIME_TYPES: { [key: string]: string } = { +const CUSTOM_MIME_TYPES: { [key: string]: string } = { // Legacy multimedia formats aab: 'application/x-authorware-bin', aam: 'application/x-authorware-map', @@ -182,7 +182,6 @@ export const CUSTOM_MIME_TYPES: { [key: string]: string } = { const STANDARD_MIME_TYPES: { [key: string]: string } = { // Text txt: 'text/plain', - css: 'text/css', csv: 'text/csv', // Images @@ -253,26 +252,3 @@ export function getMimeType(extension: string): string { // Default to octet-stream for unknown types return 'application/octet-stream'; } - -/** - * Check if a file extension is a gzipped type - */ -export function isGzippedType(extension: string): boolean { - const gzippedTypes = ['svgz']; - return gzippedTypes.includes(extension.toLowerCase()); -} - -/** - * Check if a file extension is a script type - */ -export function isScriptType(extension: string): boolean { - const scriptTypes = ['php', 'php5', 'phtml', 'pl']; - return scriptTypes.includes(extension.toLowerCase()); -} - -/** - * Get list of index file types - */ -export function getIndexFileTypes(): string[] { - return ['html', 'htm', 'php', 'php5', 'phtml', 'asp', 'aspx', 'jsp', 'jspx']; -} diff --git a/game-service/src/proxy-request-handler.ts b/backend/src/game/proxy-request-handler.ts similarity index 67% rename from game-service/src/proxy-request-handler.ts rename to backend/src/game/proxy-request-handler.ts index 73b0ae7..5c3ef3d 100644 --- a/game-service/src/proxy-request-handler.ts +++ b/backend/src/game/proxy-request-handler.ts @@ -1,26 +1,39 @@ import { IncomingMessage, ServerResponse } from 'http'; -import { URL } from 'url'; -import { Readable } from 'stream'; -import { logger } from './utils/logger'; +import path from 'path'; +import { logger } from '../utils/logger'; import { ConfigManager } from './config'; import { LegacyServer, HttpRequestContext, LegacyFileResponse } from './legacy-server'; import { injectPolyfills } from './utils/htmlInjector'; import { sanitizeErrorMessage, sanitizeUrlPath } from './utils/pathSecurity'; -import axios from 'axios'; +import { validateHostname } from './validation/schemas'; +import { zipManager } from './zip-manager'; +import { getMimeType } from './mimeTypes'; +import { setCorsHeaders } from './utils/cors'; // Maximum request body size for CGI (10MB) const MAX_REQUEST_BODY_SIZE = 10 * 1024 * 1024; // Internal request timeout constants (G-H2) -const GAMEZIP_REQUEST_TIMEOUT_MS = 5000; // 5 seconds for local GameZip server const STREAM_TIMEOUT_MS = 120000; // 2 minutes for streaming responses // Cache control constants const CACHE_MAX_AGE_SECONDS = 86400; // 24 hours (in seconds) +// Allowlist of safe CGI response headers to forward +// Prevents header injection attacks (e.g., Set-Cookie, Location, CORS overrides) +const ALLOWED_CGI_HEADERS = new Set([ + 'cache-control', + 'expires', + 'last-modified', + 'etag', + 'pragma', + 'content-disposition', + 'content-language', + 'x-powered-by', +]); + export class ProxyRequestHandler { private legacyServer: LegacyServer; - private settings = ConfigManager.getSettings(); constructor() { this.legacyServer = new LegacyServer(); @@ -42,18 +55,6 @@ export class ProxyRequestHandler { return; } - // Security check: Sanitize URL path BEFORE any processing (G-H1) - // This catches double-encoded attacks and dangerous patterns early - try { - sanitizeUrlPath(req.url); - } catch (error) { - logger.warn( - `[Security] Invalid URL rejected: ${error instanceof Error ? error.message : 'Unknown error'}` - ); - this.sendError(res, 400, 'Bad Request: Invalid URL'); - return; - } - let targetUrl: URL; let hostname: string; let urlPath: string; @@ -87,10 +88,21 @@ export class ProxyRequestHandler { targetUrl = new URL(`http://${hostname}${urlPath}`); } + // Strip port from hostname (Host header may include port, e.g., "example.com:8080") + hostname = hostname.split(':')[0]; + + // Validate hostname to prevent SSRF and other host-based attacks + try { + hostname = validateHostname(hostname); + } catch (error) { + this.sendError(res, 400, 'Bad Request: Invalid hostname'); + return; + } + logger.info(`[ProxyHandler] Parsed - Host: ${hostname}, Path: ${urlPath}`); - // Step 1: Try GameZip server first (if enabled) - if (this.settings.gameZipPort) { + // Step 1: Try GameZip server first (in-process) + { logger.debug(`[ProxyHandler] Trying GameZip server...`); const gameZipResult = await this.tryGameZipServer(hostname, urlPath); if (gameZipResult) { @@ -100,10 +112,9 @@ export class ProxyRequestHandler { } } - // Step 2: Build request context for CGI support + // Step 2: Try legacy file serving (local htdocs + external CDN) + // This handles games whose content is in htdocs or available from external sources. const requestContext = await this.buildRequestContext(req); - - // Step 3: Try legacy file serving (local htdocs + external fallbacks) logger.debug(`[ProxyHandler] Trying legacy server...`); const legacyResult = await this.legacyServer.serveLegacy(hostname, urlPath, requestContext); @@ -115,58 +126,45 @@ export class ProxyRequestHandler { const safeMessage = sanitizeErrorMessage(errorMessage); logger.error(`[ProxyHandler] Error handling request: ${safeMessage}`); - if (error instanceof Error && error.message.includes('not found')) { - this.sendError(res, 404, 'File not found'); - } else { - // Never expose internal error details to client (G-H4) - this.sendError(res, 500, 'Internal server error'); - } + // Never expose internal error details to client (G-H4) + this.sendError(res, 500, 'Internal server error'); } } /** - * Try to fetch from GameZip server - * Note: GameZip server responses are kept buffered since they come from local ZIP archives - * and are typically already compressed/optimized - * - * Implements timeout enforcement (G-H2) + * Try to find file from GameZip server (direct in-process call) + * Returns the file data if found in a mounted ZIP, or null to fall through + * to the legacy server fallback chain. */ private async tryGameZipServer( hostname: string, urlPath: string - ): Promise<{ data: Buffer; contentType: string } | null> { - // Create abort controller for request timeout (G-H2) - const abortController = new AbortController(); - const timeoutId = setTimeout(() => { - abortController.abort(); - }, GAMEZIP_REQUEST_TIMEOUT_MS); - + ): Promise<{ data: Buffer; contentType: string; statusCode: number } | null> { try { - // Make proxy-style request to GameZip server - // The GameZip server also accepts proxy-style requests: GET http://hostname/path HTTP/1.1 - const gameZipServerUrl = `http://localhost:${this.settings.gameZipPort}`; - const targetUrl = `http://${hostname}${urlPath}`; - const requestUrl = `${gameZipServerUrl}/${targetUrl}`; + const pathWithoutQuery = urlPath.split('?')[0]; + const relPath = path.posix.join(hostname, pathWithoutQuery); - const response = await axios.get(requestUrl, { - responseType: 'arraybuffer', - timeout: GAMEZIP_REQUEST_TIMEOUT_MS, - signal: abortController.signal, - validateStatus: (status) => status === 200, - proxy: false, // Don't use system proxy - }); + logger.debug(`[ProxyHandler] Searching mounted ZIPs for: ${relPath}`); - const data = Buffer.from(response.data); - const contentType = response.headers['content-type'] || 'application/octet-stream'; + const result = await zipManager.findFile(relPath); - return { data, contentType }; + if (result) { + const ext = path.extname(pathWithoutQuery).substring(1).toLowerCase(); + const contentType = getMimeType(ext); + + // Don't inject polyfills here - sendResponse() handles it for all HTML responses + return { data: result.data, contentType, statusCode: 200 }; + } + + // File not found in any mounted ZIP. + // Return null so the request falls through to the legacy server, + // which can serve the file from local htdocs or external CDN sources. + // If a ZIP download is in progress, the frontend already shows a + // download UI and won't try to load game content until it completes. + return null; } catch (error) { - // GameZip server not available or file not found - // Don't log detailed errors to avoid path leakage (G-H4) - logger.debug(`[ProxyHandler] GameZip unavailable or file not found`); + logger.debug(`[ProxyHandler] GameZip lookup failed`); return null; - } finally { - clearTimeout(timeoutId); } } @@ -174,27 +172,31 @@ export class ProxyRequestHandler { * Send successful response (supports both buffered and streamed responses) */ private sendResponse(res: ServerResponse, result: LegacyFileResponse, source: string): void { + const settings = ConfigManager.getSettings(); const contentType = result.contentType; const statusCode = result.statusCode || 200; // CORS headers - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); - } + setCorsHeaders(res, settings); // Common headers res.setHeader('Cache-Control', `public, max-age=${CACHE_MAX_AGE_SECONDS}`); - res.setHeader('X-Source', source); // Debug header + if (process.env.NODE_ENV !== 'production') { + res.setHeader('X-Source', source); // Debug header + } res.setHeader('Connection', 'keep-alive'); - // Add CGI response headers if present + // Add CGI response headers if present (allowlist approach to prevent header injection) if (result.headers) { for (const [key, value] of Object.entries(result.headers)) { - // Skip headers that we handle separately const lowerKey = key.toLowerCase(); - if (lowerKey === 'content-type' || lowerKey === 'content-length' || lowerKey === 'status') { + // Only forward safe, known headers from CGI responses + if (!ALLOWED_CGI_HEADERS.has(lowerKey)) { + continue; + } + // Reject header values containing CR or LF to prevent header injection + if (/[\r\n]/.test(value)) { + logger.warn(`[Security] CGI header value contains CRLF, skipping: ${key}`); continue; } res.setHeader(key, value); @@ -219,6 +221,14 @@ export class ProxyRequestHandler { } }, STREAM_TIMEOUT_MS); + // Clean up stream if client disconnects + res.on('close', () => { + clearTimeout(streamTimeout); + if (result.stream && !result.stream.destroyed) { + result.stream.destroy(); + } + }); + result.stream.pipe(res); result.stream.on('error', (err) => { @@ -229,7 +239,9 @@ export class ProxyRequestHandler { if (!res.headersSent) { res.writeHead(500); } - res.end(); + if (!res.writableEnded) { + res.end(); + } }); result.stream.on('end', () => { @@ -262,15 +274,19 @@ export class ProxyRequestHandler { * Send error response */ private sendError(res: ServerResponse, statusCode: number, message: string): void { + if (res.headersSent) { + if (!res.writableEnded) { + res.end(); + } + return; + } + logger.warn(`[ProxyHandler] Sending error ${statusCode}: ${message}`); + const settings = ConfigManager.getSettings(); // IMPORTANT: Set CORS headers even for error responses // Otherwise browsers will show CORS error instead of the actual error - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); - } + setCorsHeaders(res, settings); res.setHeader('Content-Type', 'text/plain'); res.writeHead(statusCode); @@ -281,10 +297,10 @@ export class ProxyRequestHandler { * Handle OPTIONS request for CORS preflight */ handleOptionsRequest(req: IncomingMessage, res: ServerResponse): void { - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); + const settings = ConfigManager.getSettings(); + setCorsHeaders(res, settings); + + if (settings.allowCrossDomain) { res.setHeader('Access-Control-Max-Age', CACHE_MAX_AGE_SECONDS.toString()); } @@ -327,47 +343,38 @@ export class ProxyRequestHandler { return new Promise((resolve, reject) => { const chunks: Buffer[] = []; let totalSize = 0; - let completed = false; + let settled = false; - // Timeout for body collection (30 seconds) - const timeoutHandle = setTimeout(() => { - if (!completed) { - completed = true; - req.destroy(new Error('Request body collection timed out')); - reject(new Error('Request body collection timed out')); + const settle = (fn: () => T): T | undefined => { + if (!settled) { + settled = true; + clearTimeout(timeoutHandle); + return fn(); } + return undefined; + }; + + const timeoutHandle = setTimeout(() => { + settle(() => { + req.removeAllListeners('data'); + reject(new Error('Request body collection timed out')); + }); }, 30000); req.on('data', (chunk: Buffer) => { - if (completed) return; + if (settled) return; totalSize += chunk.length; if (totalSize > MAX_REQUEST_BODY_SIZE) { - completed = true; - clearTimeout(timeoutHandle); - req.destroy(new Error('Request body too large')); - reject(new Error('Request body too large')); + settle(() => reject(new Error('Request body too large'))); return; } chunks.push(chunk); }); - req.on('end', () => { - if (completed) return; - completed = true; - clearTimeout(timeoutHandle); - if (chunks.length === 0) { - resolve(undefined); - } else { - resolve(Buffer.concat(chunks)); - } - }); - - req.on('error', (error) => { - if (completed) return; - completed = true; - clearTimeout(timeoutHandle); - reject(error); - }); + req.on('end', () => + settle(() => resolve(chunks.length === 0 ? undefined : Buffer.concat(chunks))) + ); + req.on('error', (error) => settle(() => reject(error))); }); } } diff --git a/backend/src/game/services/GameDataDownloader.ts b/backend/src/game/services/GameDataDownloader.ts new file mode 100644 index 0000000..d0408bc --- /dev/null +++ b/backend/src/game/services/GameDataDownloader.ts @@ -0,0 +1,642 @@ +import fs from 'fs/promises'; +import { createWriteStream, createReadStream } from 'fs'; +import path from 'path'; +import crypto from 'crypto'; +import https from 'https'; +import http from 'http'; +import net from 'net'; +import dns from 'dns'; +import { promisify } from 'util'; +import { logger } from '../../utils/logger'; +import { PreferencesService, GameDataSource } from './PreferencesService'; + +const dnsLookup = promisify(dns.lookup); + +/** + * Parameters for downloading a game data ZIP + */ +export interface GameDataDownloadParams { + gameId: string; + dateAdded: string; // ISO date string from game_data table + sha256?: string; // Expected SHA256 hash for verification + targetPath?: string; // Override target path (default: Data/Games/) +} + +/** + * Result of a download attempt + */ +export interface DownloadResult { + success: boolean; + filePath?: string; + error?: string; + sourceUsed?: string; +} + +/** + * Progress callback for download tracking + */ +export type DownloadProgressCallback = ( + downloadedBytes: number, + totalBytes: number | null, + sourceName: string +) => void; + +/** + * Service for downloading game data ZIPs from configured sources + * Implements the same logic as Flashpoint Launcher's download.ts + */ +export class GameDataDownloader { + private static instance: GameDataDownloader; + private readonly MAX_RETRIES = 3; + private readonly RETRY_DELAY_MS = 1000; + private readonly DOWNLOAD_TIMEOUT_MS = 5 * 60 * 1000; // 5 minutes + private readonly MAX_FILE_SIZE = 500 * 1024 * 1024; // 500MB max + + /** + * Private/reserved IPv4 and IPv6 CIDR ranges that must be blocked + * to prevent SSRF attacks targeting internal services. + */ + private static readonly PRIVATE_IP_RANGES: Array<{ + subnet: string; + prefix: number; + family: 4 | 6; + }> = [ + // IPv4 private/reserved ranges + { subnet: '127.0.0.0', prefix: 8, family: 4 }, // Loopback + { subnet: '10.0.0.0', prefix: 8, family: 4 }, // Private Class A + { subnet: '172.16.0.0', prefix: 12, family: 4 }, // Private Class B + { subnet: '192.168.0.0', prefix: 16, family: 4 }, // Private Class C + { subnet: '169.254.0.0', prefix: 16, family: 4 }, // Link-local / cloud metadata + { subnet: '0.0.0.0', prefix: 8, family: 4 }, // Current network + // IPv6 private/reserved ranges + { subnet: '::1', prefix: 128, family: 6 }, // Loopback + { subnet: 'fc00::', prefix: 7, family: 6 }, // Unique local (fc00::/7) + { subnet: 'fe80::', prefix: 10, family: 6 }, // Link-local + ]; + + private constructor() {} + + static getInstance(): GameDataDownloader { + if (!GameDataDownloader.instance) { + GameDataDownloader.instance = new GameDataDownloader(); + } + return GameDataDownloader.instance; + } + + /** + * Generate the filename for a game data ZIP + * Matches Flashpoint Launcher's getGameDataFilename() logic + */ + static getFilename(gameId: string, dateAdded: string): string { + // Handle different date formats (with or without 'T') + let cleanDate: string; + if (dateAdded.includes('T')) { + cleanDate = dateAdded; + } else { + // Insert 'T' between date and time at position 10 + cleanDate = dateAdded.slice(0, 10) + 'T' + dateAdded.slice(11); + } + + const timestamp = new Date(cleanDate).getTime(); + + // Validate timestamp with fallback + if (isNaN(timestamp)) { + // Try fallback: append UTC suffix + const fallbackDate = `${dateAdded} +0000 UTC`; + const fallbackTimestamp = new Date(fallbackDate).getTime(); + if (!isNaN(fallbackTimestamp)) { + return `${gameId}-${fallbackTimestamp}.zip`; + } + throw new Error(`Invalid dateAdded value: ${dateAdded}`); + } + + return `${gameId}-${timestamp}.zip`; + } + + /** + * Download a game data ZIP from configured sources + * Tries each source sequentially until one succeeds + */ + async download( + params: GameDataDownloadParams, + onProgress?: DownloadProgressCallback + ): Promise { + const { gameId, dateAdded, sha256 } = params; + const filename = GameDataDownloader.getFilename(gameId, dateAdded); + + logger.info(`[GameDataDownloader] Starting download for ${filename}`); + + // Get configured sources + const prefsService = PreferencesService.getInstance(); + const sources = await prefsService.getGameDataSources(); + + if (sources.length === 0) { + logger.error('[GameDataDownloader] No game data sources configured in preferences.json'); + return { + success: false, + error: 'No game data sources configured. Add gameDataSources to preferences.json.', + }; + } + + // Determine target path + const targetDir = params.targetPath || (await prefsService.getDataPacksFolderPath()); + const targetPath = path.join(targetDir, filename); + const tempPath = `${targetPath}.temp`; + + // Ensure target directory exists + try { + await fs.mkdir(targetDir, { recursive: true }); + } catch (err) { + const errorMsg = err instanceof Error ? err.message : 'Unknown error'; + return { success: false, error: `Failed to create directory: ${errorMsg}` }; + } + + // Try each source sequentially + const errors: string[] = []; + + for (const source of sources) { + // Check if source has arguments and at least one URL + if (!source.arguments?.length || !source.arguments[0]) { + logger.warn(`[GameDataDownloader] Source "${source.name}" has no arguments, skipping`); + continue; + } + + let baseUrl = source.arguments[0]; + + // Enforce HTTPS for download sources + if (new URL(baseUrl).protocol === 'http:') { + const secureUrl = baseUrl.replace(/^http:/, 'https:'); + logger.warn(`[GameDataDownloader] Upgrading insecure source URL to HTTPS: ${baseUrl}`); + baseUrl = secureUrl; + } + + // Normalize baseUrl trailing slash before URL construction + const normalizedBase = baseUrl.endsWith('/') ? baseUrl : baseUrl + '/'; + const fullUrl = new URL(filename, normalizedBase).href; + logger.info(`[GameDataDownloader] Trying source "${source.name}": ${fullUrl}`); + + try { + // Download to temp file + await this.downloadFile(fullUrl, tempPath, source.name, onProgress); + + // Verify SHA256 if provided + if (sha256) { + const fileHash = await this.calculateSha256(tempPath); + if (fileHash.toLowerCase() !== sha256.toLowerCase()) { + logger.error( + `[GameDataDownloader] SHA256 mismatch! Expected: ${sha256}, Got: ${fileHash}` + ); + await this.safeUnlink(tempPath); + errors.push( + `Source "${source.name}": SHA256 mismatch (expected ${sha256.substring(0, 8)}..., got ${fileHash.substring(0, 8)}...)` + ); + continue; + } + logger.info(`[GameDataDownloader] SHA256 verified: ${fileHash.substring(0, 16)}...`); + } + + // Move temp file to final location + await fs.rename(tempPath, targetPath); + + logger.info( + `[GameDataDownloader] Successfully downloaded ${filename} from "${source.name}"` + ); + + return { + success: true, + filePath: targetPath, + sourceUsed: source.name, + }; + } catch (error) { + const errorMsg = error instanceof Error ? error.message : String(error); + logger.error(`[GameDataDownloader] Source "${source.name}" failed: ${errorMsg}`); + errors.push(`Source "${source.name}": ${errorMsg}`); + + // Clean up temp file if exists + await this.safeUnlink(tempPath); + } + } + + // All sources failed + const errorSummary = errors.join('\n'); + logger.error(`[GameDataDownloader] All sources failed for ${filename}:\n${errorSummary}`); + + return { + success: false, + error: `Failed to download from all ${sources.length} sources:\n${errorSummary}`, + }; + } + + /** + * Download a file from URL to local path with retry logic + */ + private async downloadFile( + url: string, + destPath: string, + sourceName: string, + onProgress?: DownloadProgressCallback + ): Promise { + let lastError: Error | null = null; + + for (let attempt = 1; attempt <= this.MAX_RETRIES; attempt++) { + try { + await this.downloadFileOnce(url, destPath, sourceName, onProgress); + return; + } catch (error) { + lastError = error instanceof Error ? error : new Error(String(error)); + logger.warn( + `[GameDataDownloader] Attempt ${attempt}/${this.MAX_RETRIES} failed: ${lastError.message}` + ); + + if (attempt < this.MAX_RETRIES) { + const delay = this.RETRY_DELAY_MS * attempt; + logger.info(`[GameDataDownloader] Retrying in ${delay}ms...`); + await this.sleep(delay); + } + } + } + + throw lastError || new Error('Download failed after all retries'); + } + + /** + * Single download attempt with SSRF protection and double-settlement guard + */ + private async downloadFileOnce( + url: string, + destPath: string, + sourceName: string, + onProgress?: DownloadProgressCallback, + maxRedirects: number = 5 + ): Promise { + return new Promise(async (resolve, reject) => { + // Guard against double-settlement: the 'data' handler can reject on size + // limit while the piped writeStream 'finish' event can still fire afterward. + let settled = false; + const safeResolve = () => { + if (!settled) { + settled = true; + resolve(); + } + }; + const safeReject = (error: Error) => { + if (!settled) { + settled = true; + reject(error); + } + }; + + if (maxRedirects <= 0) { + safeReject(new Error('Too many redirects')); + return; + } + + // Validate initial URL protocol - only HTTP(S) allowed + const parsedUrl = new URL(url); + if (parsedUrl.protocol !== 'https:' && parsedUrl.protocol !== 'http:') { + safeReject( + new Error( + `Unsupported protocol: ${parsedUrl.protocol} - only http: and https: are allowed` + ) + ); + return; + } + + // Validate initial URL is not targeting a private/reserved IP + if (this.isPrivateHost(parsedUrl.hostname)) { + logger.warn( + `[GameDataDownloader] Blocked download to private/reserved address: ${parsedUrl.hostname}` + ); + safeReject( + new Error( + `Download blocked: ${parsedUrl.hostname} resolves to a private/reserved address` + ) + ); + return; + } + + // DNS rebinding protection: resolve hostname and check for private IPs + const hostname = parsedUrl.hostname; + if (!net.isIPv4(hostname) && !net.isIPv6(hostname)) { + // It's a hostname, not an IP - perform DNS lookup + try { + const { address } = await dnsLookup(hostname); + if (this.isPrivateIPv4(address) || this.isPrivateIPv6(address)) { + logger.warn( + `[GameDataDownloader] Blocked: ${hostname} resolves to private address ${address}` + ); + safeReject(new Error(`Blocked: ${hostname} resolves to private address ${address}`)); + return; + } + } catch (error) { + if (error instanceof Error && error.message.startsWith('Blocked:')) { + safeReject(error); + return; + } + // DNS lookup failed - allow the request to proceed (it will fail on connect anyway) + } + } + + const protocol = parsedUrl.protocol === 'https:' ? https : http; + let writeStream: ReturnType | null = null; + + const request = protocol.get(url, { timeout: this.DOWNLOAD_TIMEOUT_MS }, (response) => { + // Handle redirects + if (response.statusCode && response.statusCode >= 300 && response.statusCode < 400) { + const redirectUrl = response.headers.location; + if (redirectUrl) { + response.resume(); + + // Validate redirect URL before following + try { + this.validateRedirectUrl(redirectUrl); + } catch (err) { + safeReject(err instanceof Error ? err : new Error(String(err))); + return; + } + + logger.info(`[GameDataDownloader] Following redirect to: ${redirectUrl}`); + this.downloadFileOnce(redirectUrl, destPath, sourceName, onProgress, maxRedirects - 1) + .then(safeResolve) + .catch(safeReject); + return; + } + } + + // Check for success status + if (response.statusCode !== 200) { + response.resume(); + safeReject(new Error(`HTTP ${response.statusCode}: ${response.statusMessage}`)); + return; + } + + // Check content length + const contentLength = response.headers['content-length']; + const totalBytes = contentLength ? parseInt(contentLength, 10) : null; + + if (totalBytes && totalBytes > this.MAX_FILE_SIZE) { + response.resume(); // Drain the response + safeReject(new Error(`File too large: ${totalBytes} bytes (max: ${this.MAX_FILE_SIZE})`)); + return; + } + + // Create write stream + writeStream = createWriteStream(destPath); + let downloadedBytes = 0; + + response.on('data', (chunk: Buffer) => { + downloadedBytes += chunk.length; + + // Check size limit during download + if (downloadedBytes > this.MAX_FILE_SIZE) { + response.destroy(); + request.destroy(); + if (writeStream) { + writeStream.destroy(); + } + safeReject(new Error(`Download exceeded size limit: ${this.MAX_FILE_SIZE} bytes`)); + return; + } + + if (onProgress) { + onProgress(downloadedBytes, totalBytes, sourceName); + } + }); + + response.pipe(writeStream); + + writeStream.on('finish', () => { + logger.info( + `[GameDataDownloader] Downloaded ${downloadedBytes} bytes from "${sourceName}"` + ); + safeResolve(); + }); + + writeStream.on('error', (error) => { + safeReject(error); + }); + + response.on('error', (error) => { + if (writeStream) { + writeStream.destroy(); + } + request.destroy(); + safeReject(error); + }); + }); + + request.on('error', (error) => { + if (writeStream) { + writeStream.destroy(); + } + safeReject(error); + }); + + request.on('timeout', () => { + if (writeStream) { + writeStream.destroy(); // Also close the file handle + } + request.destroy(); + safeReject(new Error(`Download timed out after ${this.DOWNLOAD_TIMEOUT_MS}ms`)); + }); + }); + } + + /** + * Validate a redirect URL to prevent SSRF attacks. + * Rejects non-HTTP(S) protocols and private/reserved IP addresses. + * Throws an Error if the URL is not safe to follow. + */ + private validateRedirectUrl(redirectUrl: string): void { + let parsed: URL; + try { + parsed = new URL(redirectUrl); + } catch { + throw new Error(`Invalid redirect URL: ${redirectUrl}`); + } + + // Only allow http: and https: protocols + if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') { + logger.warn( + `[GameDataDownloader] Blocked redirect to disallowed protocol: ${parsed.protocol} (${redirectUrl})` + ); + throw new Error( + `Redirect blocked: unsupported protocol ${parsed.protocol} - only http: and https: are allowed` + ); + } + + // Block private/reserved IP addresses + if (this.isPrivateHost(parsed.hostname)) { + logger.warn( + `[GameDataDownloader] Blocked redirect to private/reserved address: ${parsed.hostname} (${redirectUrl})` + ); + throw new Error( + `Redirect blocked: ${parsed.hostname} resolves to a private/reserved address` + ); + } + } + + /** + * Check if a hostname is a private or reserved IP address. + * Handles both IPv4 and IPv6, including bracketed IPv6 notation. + */ + private isPrivateHost(hostname: string): boolean { + // Strip brackets from IPv6 addresses (e.g., [::1] -> ::1) + const cleanHostname = hostname.replace(/^\[|\]$/g, ''); + + // Check for 'localhost' hostname + if (cleanHostname.toLowerCase() === 'localhost') { + return true; + } + + // Try to parse as IPv4 + if (net.isIPv4(cleanHostname)) { + return this.isPrivateIPv4(cleanHostname); + } + + // Try to parse as IPv6 + if (net.isIPv6(cleanHostname)) { + return this.isPrivateIPv6(cleanHostname); + } + + // Not an IP literal - it is a hostname. + // DNS resolution could map to a private IP, but performing async DNS + // lookup here is impractical in a synchronous validation path. + // The redirect-hop limit (5) plus the protocol check provides defense-in-depth. + return false; + } + + /** + * Check if an IPv4 address falls within a private/reserved CIDR range + */ + private isPrivateIPv4(ip: string): boolean { + const ipNum = this.ipv4ToNumber(ip); + if (ipNum === null) return false; + + for (const range of GameDataDownloader.PRIVATE_IP_RANGES) { + if (range.family !== 4) continue; + const subnetNum = this.ipv4ToNumber(range.subnet); + if (subnetNum === null) continue; + + const mask = range.prefix === 0 ? 0 : (~0 << (32 - range.prefix)) >>> 0; + if ((ipNum & mask) === (subnetNum & mask)) { + return true; + } + } + return false; + } + + /** + * Check if an IPv6 address falls within a private/reserved range + */ + private isPrivateIPv6(ip: string): boolean { + const normalized = ip.toLowerCase(); + + // Exact loopback check + if (normalized === '::1') return true; + + // Unique local addresses: fc00::/7 (fc00:: - fdff::) + if (normalized.startsWith('fc') || normalized.startsWith('fd')) return true; + + // Link-local: fe80::/10 + if (normalized.startsWith('fe80')) return true; + + // IPv4-mapped IPv6 (::ffff:x.x.x.x) - check the embedded IPv4 + const v4mappedMatch = normalized.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/); + if (v4mappedMatch) { + return this.isPrivateIPv4(v4mappedMatch[1]); + } + + return false; + } + + /** + * Convert an IPv4 address string to a 32-bit unsigned integer + */ + private ipv4ToNumber(ip: string): number | null { + const parts = ip.split('.'); + if (parts.length !== 4) return null; + + let num = 0; + for (const part of parts) { + const octet = parseInt(part, 10); + if (isNaN(octet) || octet < 0 || octet > 255) return null; + num = (num * 256 + octet) >>> 0; + } + return num; + } + + /** + * Calculate SHA256 hash of a file + */ + private calculateSha256(filePath: string): Promise { + return new Promise((resolve, reject) => { + const hash = crypto.createHash('sha256'); + const stream = createReadStream(filePath); + + stream.on('data', (data) => hash.update(data)); + stream.on('end', () => resolve(hash.digest('hex').toUpperCase())); + stream.on('error', reject); + }); + } + + /** + * Safely delete a file (ignore errors if file doesn't exist) + */ + private async safeUnlink(filePath: string): Promise { + try { + await fs.unlink(filePath); + } catch { + // Ignore errors (file may not exist) + } + } + + /** + * Sleep for specified milliseconds + */ + private sleep(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); + } + + /** + * Check if a game data ZIP exists locally + */ + async exists(gameId: string, dateAdded: string, targetDir?: string): Promise { + try { + const filename = GameDataDownloader.getFilename(gameId, dateAdded); + const prefsService = PreferencesService.getInstance(); + const dir = targetDir || (await prefsService.getDataPacksFolderPath()); + const filePath = path.join(dir, filename); + + try { + await fs.access(filePath); + return true; + } catch { + return false; + } + } catch { + // getFilename() can throw on invalid dateAdded + return false; + } + } + + /** + * Get the full path to a game data ZIP + */ + async getFilePath(gameId: string, dateAdded: string, targetDir?: string): Promise { + try { + const filename = GameDataDownloader.getFilename(gameId, dateAdded); + const prefsService = PreferencesService.getInstance(); + const dir = targetDir || (await prefsService.getDataPacksFolderPath()); + return path.join(dir, filename); + } catch (error) { + // getFilename() can throw on invalid dateAdded + throw new Error( + `Invalid dateAdded value: ${dateAdded} - ${error instanceof Error ? error.message : 'Unknown error'}` + ); + } + } +} + +// Export singleton instance +export const gameDataDownloader = GameDataDownloader.getInstance(); diff --git a/backend/src/game/services/PreferencesService.ts b/backend/src/game/services/PreferencesService.ts new file mode 100644 index 0000000..71d9e78 --- /dev/null +++ b/backend/src/game/services/PreferencesService.ts @@ -0,0 +1,140 @@ +import fs from 'fs/promises'; +import path from 'path'; +import { logger } from '../../utils/logger'; + +/** + * Game data source configuration from preferences.json + * Used for downloading game ZIPs when not present locally + */ +export interface GameDataSource { + type: string; // Currently only "raw" is used + name: string; // Display name for the source + arguments: string[]; // Array of arguments, first is the base URL +} + +/** + * Relevant fields from Flashpoint preferences.json + */ +export interface FlashpointPreferences { + gameDataSources: GameDataSource[]; + dataPacksFolderPath: string; + onDemandBaseUrl?: string; +} + +/** + * Service for reading Flashpoint preferences.json + * Provides access to gameDataSources for downloading game ZIPs + */ +export class PreferencesService { + private static instance: PreferencesService; + private preferences: FlashpointPreferences | null = null; + private flashpointPath: string; + private preferencesPath: string; + private lastLoadTime: number = 0; + private lastLoadFailed: boolean = false; + private readonly CACHE_TTL = 60 * 1000; // 1 minute cache + private readonly ERROR_CACHE_TTL = 5000; // 5 seconds cache for errors + + private constructor(flashpointPath: string) { + this.flashpointPath = flashpointPath; + this.preferencesPath = path.join(flashpointPath, 'preferences.json'); + } + + /** + * Initialize the PreferencesService singleton + */ + static initialize(flashpointPath: string): PreferencesService { + if (PreferencesService.instance) { + if (flashpointPath !== PreferencesService.instance.flashpointPath) { + logger.warn( + '[PreferencesService] Already initialized with different path, ignoring re-init' + ); + } + return PreferencesService.instance; + } + PreferencesService.instance = new PreferencesService(flashpointPath); + return PreferencesService.instance; + } + + /** + * Get the singleton instance + */ + static getInstance(): PreferencesService { + if (!PreferencesService.instance) { + throw new Error('PreferencesService not initialized. Call initialize() first.'); + } + return PreferencesService.instance; + } + + /** + * Load preferences from file with caching + */ + async loadPreferences(): Promise { + const now = Date.now(); + + // Use shorter TTL if last load failed + const cacheTTL = this.lastLoadFailed ? this.ERROR_CACHE_TTL : this.CACHE_TTL; + + // Return cached preferences if still valid + if (this.preferences && now - this.lastLoadTime < cacheTTL) { + return this.preferences; + } + + try { + const data = await fs.readFile(this.preferencesPath, 'utf-8'); + const parsed = JSON.parse(data); + + this.preferences = { + gameDataSources: parsed.gameDataSources || [], + dataPacksFolderPath: parsed.dataPacksFolderPath || 'Data/Games', + onDemandBaseUrl: parsed.onDemandBaseUrl, + }; + + this.lastLoadTime = now; + this.lastLoadFailed = false; + + logger.info( + `[PreferencesService] Loaded preferences: ${this.preferences.gameDataSources.length} game data sources` + ); + + return this.preferences; + } catch (error) { + logger.error('[PreferencesService] Failed to load preferences.json:', error); + + // Return defaults if file doesn't exist or is invalid + this.preferences = { + gameDataSources: [], + dataPacksFolderPath: 'Data/Games', + }; + + this.lastLoadTime = now; + this.lastLoadFailed = true; + + return this.preferences; + } + } + + /** + * Get game data sources for downloading ZIPs + */ + async getGameDataSources(): Promise { + const prefs = await this.loadPreferences(); + return prefs.gameDataSources; + } + + /** + * Get the data packs folder path (relative to FLASHPOINT_PATH) + */ + async getDataPacksFolderPath(): Promise { + const prefs = await this.loadPreferences(); + return path.join(this.flashpointPath, prefs.dataPacksFolderPath); + } + + /** + * Invalidate the cache to force reload on next access + */ + invalidateCache(): void { + this.lastLoadTime = 0; + logger.info('[PreferencesService] Cache invalidated'); + } +} diff --git a/backend/src/game/services/index.ts b/backend/src/game/services/index.ts new file mode 100644 index 0000000..8a13acd --- /dev/null +++ b/backend/src/game/services/index.ts @@ -0,0 +1,9 @@ +export { PreferencesService } from './PreferencesService'; +export type { GameDataSource, FlashpointPreferences } from './PreferencesService'; + +export { GameDataDownloader, gameDataDownloader } from './GameDataDownloader'; +export type { + GameDataDownloadParams, + DownloadResult, + DownloadProgressCallback, +} from './GameDataDownloader'; diff --git a/game-service/src/utils/cors.ts b/backend/src/game/utils/cors.ts similarity index 55% rename from game-service/src/utils/cors.ts rename to backend/src/game/utils/cors.ts index c5c459d..767c3b4 100644 --- a/game-service/src/utils/cors.ts +++ b/backend/src/game/utils/cors.ts @@ -28,27 +28,3 @@ export function setCorsHeaders(res: ServerResponse, settings: CorsSettings): voi res.setHeader('Access-Control-Allow-Headers', '*'); } } - -/** - * Set CORS headers with max age for preflight caching - * - * @param res - HTTP ServerResponse object - * @param settings - CORS configuration settings - * @param maxAge - Max age in seconds for preflight cache (default: 86400 = 24 hours) - * - * @example - * ```typescript - * setCorsHeadersWithMaxAge(res, { allowCrossDomain: true }, 86400); - * ``` - */ -export function setCorsHeadersWithMaxAge( - res: ServerResponse, - settings: CorsSettings, - maxAge = 86400 -): void { - setCorsHeaders(res, settings); - - if (settings.allowCrossDomain) { - res.setHeader('Access-Control-Max-Age', maxAge.toString()); - } -} diff --git a/game-service/src/utils/htmlInjector.ts b/backend/src/game/utils/htmlInjector.ts similarity index 95% rename from game-service/src/utils/htmlInjector.ts rename to backend/src/game/utils/htmlInjector.ts index 175e0c9..8c63c4b 100644 --- a/game-service/src/utils/htmlInjector.ts +++ b/backend/src/game/utils/htmlInjector.ts @@ -3,6 +3,8 @@ * for better compatibility with Unity WebGL and other game engines */ +import { logger } from '../../utils/logger'; + /** * Unity WebGL polyfills * These are minimal stubs to prevent errors when Unity games load @@ -106,17 +108,12 @@ export function injectPolyfills(html: Buffer | string): Buffer { // Add Unity polyfills if needed if (needsUnityPolyfills(htmlString)) { polyfillsToInject += UNITY_POLYFILLS; - console.log('[HTMLInjector] Injecting Unity WebGL polyfills'); + logger.debug('[HTMLInjector] Injecting Unity WebGL polyfills'); } // Always add general polyfills for compatibility polyfillsToInject += GENERAL_POLYFILLS; - // If no polyfills needed, return original - if (!polyfillsToInject) { - return Buffer.from(htmlString); - } - // Try to inject into tag const headMatch = htmlString.match(/]*>/i); if (headMatch) { diff --git a/game-service/src/utils/pathSecurity.ts b/backend/src/game/utils/pathSecurity.ts similarity index 81% rename from game-service/src/utils/pathSecurity.ts rename to backend/src/game/utils/pathSecurity.ts index 08c347d..4b057a3 100644 --- a/game-service/src/utils/pathSecurity.ts +++ b/backend/src/game/utils/pathSecurity.ts @@ -1,5 +1,5 @@ import path from 'path'; -import { logger } from './logger'; +import { logger } from '../../utils/logger'; /** * Path Security Utilities @@ -18,8 +18,8 @@ export function hasDoubleEncoding(urlPath: string): boolean { // %25 is the encoding for '%', so %252e = %2e, %252f = %2f, etc. // Double-encoding is a technique to bypass security filters that only decode once. const doubleEncodedPatterns = [ - /%25(?:2[eEfF]|5[cC])/gi, // %252e (%2e = .), %252f (%2f = /), %255c (%5c = \) - /%25(?:00)/gi, // %2500 = double-encoded null byte + /%25(?:2[eEfF]|5[cC])/i, // %252e (%2e = .), %252f (%2f = /), %255c (%5c = \) + /%25(?:00)/i, // %2500 = double-encoded null byte ]; for (const pattern of doubleEncodedPatterns) { @@ -88,6 +88,18 @@ export function sanitizeAndValidatePath(basePath: string, requestPath: string): logger.warn(`[Security] Base directory: ${resolvedBase}`); throw new Error('Invalid path: Directory traversal detected'); } + + // Verify path boundary to prevent sibling directory access + // e.g., "htdocs-backup" should not pass when base is "htdocs" + if ( + normalizedResolvedPath !== normalizedResolvedBase && + !normalizedResolvedPath.startsWith(normalizedResolvedBase + path.sep) + ) { + logger.warn(`[Security] Sibling directory access blocked: ${requestPath}`); + logger.warn(`[Security] Resolved to: ${resolvedPath}`); + logger.warn(`[Security] Base directory: ${resolvedBase}`); + throw new Error('Invalid path: Path escapes allowed directory'); + } } else { // Case-sensitive comparison for Unix-like systems if (!resolvedPath.startsWith(resolvedBase)) { @@ -96,38 +108,17 @@ export function sanitizeAndValidatePath(basePath: string, requestPath: string): logger.warn(`[Security] Base directory: ${resolvedBase}`); throw new Error('Invalid path: Directory traversal detected'); } - } - return resolvedPath; -} - -/** - * Validate that a path is within one of multiple allowed base directories - * - * @param allowedBases - Array of allowed base directories - * @param requestPath - The requested path - * @returns The validated absolute path - * @throws Error if the path is not within any allowed base directory - */ -export function validatePathInAllowedDirectories( - allowedBases: string[], - requestPath: string -): string { - // Try each base directory - for (const basePath of allowedBases) { - try { - const validatedPath = sanitizeAndValidatePath(basePath, requestPath); - return validatedPath; - } catch (error) { - // Path is not in this base, try next one - continue; + // Verify path boundary to prevent sibling directory access + if (resolvedPath !== resolvedBase && !resolvedPath.startsWith(resolvedBase + path.sep)) { + logger.warn(`[Security] Sibling directory access blocked: ${requestPath}`); + logger.warn(`[Security] Resolved to: ${resolvedPath}`); + logger.warn(`[Security] Base directory: ${resolvedBase}`); + throw new Error('Invalid path: Path escapes allowed directory'); } } - // Path is not in any allowed base directory - logger.warn(`[Security] Path not in any allowed directory: ${requestPath}`); - logger.warn(`[Security] Allowed bases: ${allowedBases.join(', ')}`); - throw new Error('Invalid path: Not in any allowed directory'); + return resolvedPath; } /** @@ -159,10 +150,10 @@ export function sanitizeUrlPath(urlPath: string): string { // Note: We allow '..' because it will be handled by sanitizeAndValidatePath // But we check for obviously malicious patterns const dangerousPatterns = [ - /\.\.\\/g, // ..\ (Windows path traversal) - /\.\.%2[fF]/g, // URL encoded ../ - /\.\.%5[cC]/g, // URL encoded ..\ - /%00/g, // URL encoded null byte + /\.\.\\/, // ..\ (Windows path traversal) + /\.\.%2[fF]/, // URL encoded ../ + /\.\.%5[cC]/, // URL encoded ..\ + /%00/, // URL encoded null byte ]; for (const pattern of dangerousPatterns) { diff --git a/backend/src/game/validation/schemas.ts b/backend/src/game/validation/schemas.ts new file mode 100644 index 0000000..a1f5bc7 --- /dev/null +++ b/backend/src/game/validation/schemas.ts @@ -0,0 +1,61 @@ +import { z } from 'zod'; + +/** + * Validation schemas for game-service endpoints + * Provides input validation to prevent security issues and malformed requests + */ + +/** + * Schema for mount endpoint + * POST /mount/:gameId + */ +export const mountGameSchema = z.object({ + gameId: z + .string() + .min(1, 'Game ID is required') + .max(255, 'Game ID is too long') + .regex(/^[a-zA-Z0-9\-_]+$/, 'Game ID contains invalid characters'), +}); + +/** + * Schema for unmount endpoint (reuses mountGameSchema) + * DELETE /mount/:gameId + */ +export const unmountGameSchema = mountGameSchema; + +/** + * Schema for hostname validation + */ +export const hostnameSchema = z + .string() + .min(2, 'Hostname is required') // Intentionally requires 2+ chars; single-char hostnames not supported for legacy content + .max(253, 'Hostname is too long') + .regex(/^[a-zA-Z0-9][a-zA-Z0-9\-_.]*[a-zA-Z0-9]$/, { + message: 'Invalid hostname format', + }); + +/** + * Validate and sanitize a game ID + */ +export function validateGameId(gameId: unknown): string { + const result = mountGameSchema.shape.gameId.safeParse(gameId); + + if (!result.success) { + throw new Error(`Invalid game ID: ${result.error.issues[0].message}`); + } + + return result.data; +} + +/** + * Validate and sanitize a hostname + */ +export function validateHostname(hostname: unknown): string { + const result = hostnameSchema.safeParse(hostname); + + if (!result.success) { + throw new Error(`Invalid hostname: ${result.error.issues[0].message}`); + } + + return result.data; +} diff --git a/game-service/src/zip-manager.ts b/backend/src/game/zip-manager.ts similarity index 54% rename from game-service/src/zip-manager.ts rename to backend/src/game/zip-manager.ts index 91e5bcb..dd70321 100644 --- a/game-service/src/zip-manager.ts +++ b/backend/src/game/zip-manager.ts @@ -2,20 +2,25 @@ import StreamZip from 'node-stream-zip'; import path from 'path'; import fs from 'fs/promises'; import { LRUCache } from 'lru-cache'; -import { logger } from './utils/logger'; +import { logger } from '../utils/logger'; import { sanitizeErrorMessage } from './utils/pathSecurity'; +import { MAX_BUFFERED_FILE_SIZE } from './config'; interface MountedZip { id: string; zipPath: string; zip: StreamZip.StreamZipAsync; mountTime: Date; + entryIndex: Set; // Set of all entry names for O(1) lookup } export class ZipManager { // Use LRU cache to prevent memory leaks from unlimited ZIP mounts // Max 100 concurrent mounts, 30-minute TTL, auto-close on eviction private mountedZips: LRUCache; + private mountingInProgress = new Set(); + private pendingCloses: Set> = new Set(); + private shuttingDown = false; constructor() { this.mountedZips = new LRUCache({ @@ -23,12 +28,20 @@ export class ZipManager { ttl: 30 * 60 * 1000, // 30-minute TTL dispose: async (value, key) => { // Automatically close ZIP when evicted from cache - try { - logger.info(`[ZipManager] Auto-closing evicted ZIP: ${key}`); - await value.zip.close(); - } catch (error) { - logger.error(`[ZipManager] Error closing evicted ZIP ${key}:`, error); - } + // Track the close operation to ensure cleanup completes + const closePromise = (async () => { + try { + logger.info(`[ZipManager] Auto-closing evicted ZIP: ${key}`); + await value.zip.close(); + } catch (error) { + logger.error(`[ZipManager] Error closing evicted ZIP ${key}:`, error); + } + })(); + + this.pendingCloses.add(closePromise); + closePromise.finally(() => { + this.pendingCloses.delete(closePromise); + }); }, // Update access time on get updateAgeOnGet: true, @@ -43,35 +56,63 @@ export class ZipManager { * @param zipPath Absolute path to the ZIP file */ async mount(id: string, zipPath: string): Promise { + // Check if shutting down + if (this.shuttingDown) { + throw new Error('ZipManager is shutting down'); + } + // Check if already mounted if (this.mountedZips.has(id)) { logger.debug(`[ZipManager] ZIP already mounted: ${id}`); return; } - // Verify ZIP file exists - try { - await fs.access(zipPath); - } catch (error) { - // Don't expose full path in error message (G-H4) - logger.error(`[ZipManager] ZIP file not found: ${zipPath}`); - throw new Error('ZIP file not found'); + // Check if currently mounting + if (this.mountingInProgress.has(id)) { + logger.debug(`[ZipManager] ZIP mount already in progress: ${id}`); + return; } - logger.info(`[ZipManager] Mounting ZIP: ${id} -> ${zipPath}`); + this.mountingInProgress.add(id); - // Create ZIP reader - const zip = new StreamZip.async({ file: zipPath }); + try { + // Verify ZIP file exists + try { + await fs.access(zipPath); + } catch (error) { + // Don't expose full path in error message (G-H4) + logger.error(`[ZipManager] ZIP file not found: ${zipPath}`); + throw new Error('ZIP file not found'); + } - // Store mounted ZIP - this.mountedZips.set(id, { - id, - zipPath, - zip, - mountTime: new Date(), - }); + logger.info(`[ZipManager] Mounting ZIP: ${id} -> ${zipPath}`); - logger.info(`[ZipManager] ✓ Mounted ZIP: ${id} (${await this.getFileCount(id)} files)`); + // Create ZIP reader + const zip = new StreamZip.async({ file: zipPath }); + + try { + // Build entry index for O(1) lookup + const entries = await zip.entries(); + const entryIndex = new Set(Object.keys(entries)); + + // Store mounted ZIP + this.mountedZips.set(id, { + id, + zipPath, + zip, + mountTime: new Date(), + entryIndex, + }); + + logger.info(`[ZipManager] ✓ Mounted ZIP: ${id} (${entryIndex.size} files)`); + } catch (innerError) { + // Close the ZIP handle if mounting fails + await zip.close().catch(() => {}); + throw innerError; + } + } finally { + this.mountingInProgress.delete(id); + } } /** @@ -79,7 +120,7 @@ export class ZipManager { * @param id Unique identifier of the mounted ZIP */ async unmount(id: string): Promise { - const mounted = this.mountedZips.get(id); + const mounted = this.mountedZips.peek(id); if (!mounted) { logger.warn(`[ZipManager] ZIP not mounted: ${id}`); return false; @@ -114,6 +155,22 @@ export class ZipManager { // Normalize path (remove leading slash) const normalizedPath = filePath.startsWith('/') ? filePath.substring(1) : filePath; + // Check index before reading file (O(1) vs reading entire file) + if (!mounted.entryIndex.has(normalizedPath)) { + logger.debug(`[ZipManager] File not in index ${id}: ${normalizedPath}`); + return null; + } + + // Check file size before buffering + const entries = await mounted.zip.entries(); + const entry = entries[normalizedPath]; + if (entry && entry.size > MAX_BUFFERED_FILE_SIZE) { + logger.warn( + `[ZipManager] File too large to buffer: ${normalizedPath} (${entry.size} bytes)` + ); + return null; + } + logger.debug(`[ZipManager] Reading from ZIP ${id}: ${normalizedPath}`); // Get file from ZIP @@ -146,10 +203,38 @@ export class ZipManager { // Search all mounted ZIPs for (const [id, mounted] of this.mountedZips) { for (const pathVariant of pathsToTry) { - const data = await this.getFile(id, pathVariant); - if (data) { + // Normalize path (remove leading slash) + const normalizedPath = pathVariant.startsWith('/') ? pathVariant.substring(1) : pathVariant; + + // Check index first (O(1)) + if (!mounted.entryIndex.has(normalizedPath)) { + continue; + } + + // Get entries to check if it's a directory and check size + const entries = await mounted.zip.entries(); + const entry = entries[normalizedPath]; + if (!entry || entry.isDirectory) { + continue; + } + + // Check size before reading + if (entry.size > MAX_BUFFERED_FILE_SIZE) { + logger.warn( + `[ZipManager] File too large to buffer: ${normalizedPath} (${entry.size} bytes)` + ); + continue; + } + + // Read entry data + try { + const data = await mounted.zip.entryData(normalizedPath); logger.info(`[ZipManager] ✓ Found in ${id}: ${pathVariant}`); return { data, mountId: id }; + } catch (error) { + // Entry read failed, continue to next variant + logger.debug(`[ZipManager] Failed to read entry ${normalizedPath}: ${error}`); + continue; } } } @@ -169,28 +254,13 @@ export class ZipManager { id, zipPath: mounted.zipPath, mountTime: mounted.mountTime, - fileCount: 0, // Will be populated async + fileCount: mounted.entryIndex.size, // O(1) from index }); } return result; } - /** - * Get number of files in a mounted ZIP - */ - private async getFileCount(id: string): Promise { - const mounted = this.mountedZips.get(id); - if (!mounted) return 0; - - try { - const entries = await mounted.zip.entries(); - return Object.keys(entries).length; - } catch (error) { - return 0; - } - } - /** * List files in a mounted ZIP (for debugging) */ @@ -203,8 +273,9 @@ export class ZipManager { let files = Object.keys(entries); if (pattern) { - const regex = new RegExp(pattern, 'i'); - files = files.filter((f) => regex.test(f)); + // Use simple substring matching instead of regex to prevent ReDoS + const lowerPattern = pattern.toLowerCase(); + files = files.filter((f) => f.toLowerCase().includes(lowerPattern)); } return files; @@ -228,11 +299,28 @@ export class ZipManager { * Unmount all ZIPs (cleanup on shutdown) */ async unmountAll(): Promise { + this.shuttingDown = true; + + // Wait for in-progress mounts to complete (with timeout) + const maxWait = 10000; // 10 seconds + const start = Date.now(); + while (this.mountingInProgress.size > 0 && Date.now() - start < maxWait) { + await new Promise((resolve) => setTimeout(resolve, 100)); + } + logger.info(`[ZipManager] Unmounting all ZIPs (${this.mountedZips.size})...`); const promises = Array.from(this.mountedZips.keys()).map((id) => this.unmount(id)); await Promise.all(promises); + // Wait for all pending close operations from LRU cache disposal + if (this.pendingCloses.size > 0) { + logger.info( + `[ZipManager] Waiting for ${this.pendingCloses.size} pending close operations...` + ); + await Promise.all(Array.from(this.pendingCloses)); + } + logger.info(`[ZipManager] ✓ All ZIPs unmounted`); } } diff --git a/backend/src/middleware/auth.ts b/backend/src/middleware/auth.ts index e4bb8d8..18151ff 100644 --- a/backend/src/middleware/auth.ts +++ b/backend/src/middleware/auth.ts @@ -23,8 +23,8 @@ declare global { * Authenticate user from JWT token * Requires valid token in Authorization header */ -export const authenticate = async (req: Request, res: Response, next: NextFunction) => { - try { +export const authenticate = asyncHandler( + async (req: Request, res: Response, next: NextFunction) => { // Get token from Authorization header const authHeader = req.headers.authorization; if (!authHeader || !authHeader.startsWith('Bearer ')) { @@ -38,17 +38,15 @@ export const authenticate = async (req: Request, res: Response, next: NextFuncti req.user = user; next(); - } catch (error) { - next(error); } -}; +); /** * Optional authentication - allows guest access if enabled * If token is provided, verifies it. Otherwise, sets guest user (if guest access enabled) */ -export const optionalAuth = async (req: Request, res: Response, next: NextFunction) => { - try { +export const optionalAuth = asyncHandler( + async (req: Request, res: Response, next: NextFunction) => { const authHeader = req.headers.authorization; if (authHeader && authHeader.startsWith('Bearer ')) { @@ -62,7 +60,8 @@ export const optionalAuth = async (req: Request, res: Response, next: NextFuncti throw new AppError(401, 'Authentication required', true, 'AUTH_REQUIRED'); } - // Set guest user with read-only permissions + // Guest users get no permissions - they can only browse, not play + // (authentication is required for games.play permission) req.user = { id: 0, username: 'guest', @@ -73,10 +72,8 @@ export const optionalAuth = async (req: Request, res: Response, next: NextFuncti } next(); - } catch (error) { - next(error); } -}; +); /** * Soft authentication - populates req.user if token exists, otherwise leaves undefined @@ -139,7 +136,15 @@ export const sharedAccessAuth = asyncHandler( ); } + // Grant games.play permission for valid shared access tokens req.sharedAccess = payload; + req.user = { + id: 0, + username: 'shared-guest', + email: '', + role: 'guest', + permissions: ['games.read', 'playlists.read', 'games.play'], + }; return next(); } catch (error) { if (error instanceof AppError) throw error; @@ -161,13 +166,14 @@ export const sharedAccessAuth = asyncHandler( throw new AppError(401, 'Authentication required', true, 'AUTH_REQUIRED'); } - // Set guest user + // Guest users without a shared access token can only browse, not play + // (games.play is only granted when a valid shared access token is verified above) req.user = { id: 0, username: 'guest', email: '', role: 'guest', - permissions: ['games.read', 'playlists.read', 'games.play'], + permissions: ['games.read', 'playlists.read'], }; next(); @@ -180,8 +186,8 @@ export const sharedAccessAuth = asyncHandler( */ export const validateSharedGameAccess = (gameIdParam: string = 'id') => { return asyncHandler(async (req: Request, res: Response, next: NextFunction) => { - // If regular user auth, skip validation - if (req.user) { + // If regular user auth (non-zero ID), skip validation + if (req.user && req.user.id !== 0) { return next(); } diff --git a/backend/src/middleware/errorHandler.ts b/backend/src/middleware/errorHandler.ts index 82019d9..4440105 100644 --- a/backend/src/middleware/errorHandler.ts +++ b/backend/src/middleware/errorHandler.ts @@ -1,4 +1,5 @@ import { Request, Response, NextFunction } from 'express'; +import { ZodError } from 'zod'; import { logger } from '../utils/logger'; export class AppError extends Error { @@ -43,7 +44,7 @@ function sanitizeBody(body: any): any { } export function errorHandler( - err: Error | AppError, + err: Error | AppError | ZodError, req: Request, res: Response, _next: NextFunction @@ -65,6 +66,22 @@ export function errorHandler( }); } + // Handle Zod validation errors + if (err instanceof ZodError) { + logger.error(`[ValidationError] Zod validation failed`, { + path: req.path, + method: req.method, + errors: err.errors, + }); + + return res.status(400).json({ + error: { + message: `Validation error: ${err.errors.map((e) => e.message).join(', ')}`, + statusCode: 400, + }, + }); + } + // Unhandled errors - Log full details with sanitization logger.error(`[UnhandledError] ${err.message}`, { name: err.name, diff --git a/backend/src/middleware/performanceTracking.ts b/backend/src/middleware/performanceTracking.ts index e186d31..7781227 100644 --- a/backend/src/middleware/performanceTracking.ts +++ b/backend/src/middleware/performanceTracking.ts @@ -40,7 +40,7 @@ function normalizePath(path: string): string { .replace(/\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi, '/:id') // Replace numeric IDs .replace(/\/\d+/g, '/:id') - // Replace alphanumeric IDs (game IDs, share tokens) - .replace(/\/[a-zA-Z0-9_-]{8,}/g, '/:id') + // Replace hex IDs (like game data IDs) - must contain at least one digit + .replace(/\/[a-f0-9]{8,}/gi, '/:id') ); } diff --git a/backend/src/proxy/config.ts b/backend/src/proxy/config.ts deleted file mode 100644 index 23b9bd4..0000000 --- a/backend/src/proxy/config.ts +++ /dev/null @@ -1,124 +0,0 @@ -import path from 'path'; -import fs from 'fs/promises'; -import { logger } from '../utils/logger'; - -export interface ServerSettings { - // Proxy server port (default: 22500) - proxyPort: number; - - // GameZip server port (default: 22501) - gameZipPort: number; - - // Legacy paths - legacyHTDOCSPath: string; - legacyCGIBINPath: string; - - // Override paths for file resolution - legacyOverridePaths: string[]; - - // External fallback URLs - infinityServerURL?: string; - externalFilePaths: string[]; - - // MAD4FP mode settings - mad4fpEnabled: boolean; - mad4fpPaths: string[]; - - // Performance settings - chunkSize: number; - maxConcurrentDownloads: number; - - // Feature flags - allowCrossDomain: boolean; - enableBrotli: boolean; - enableCGI: boolean; - enableHtaccess: boolean; -} - -export class ConfigManager { - private static settings: ServerSettings | null = null; - - /** - * Load configuration from proxySettings.json and environment - */ - static async loadConfig(flashpointPath: string): Promise { - try { - const proxySettingsPath = path.join(flashpointPath, 'Server', 'proxySettings.json'); - const data = await fs.readFile(proxySettingsPath, 'utf-8'); - const proxySettings = JSON.parse(data); - - // Build settings from proxySettings.json and defaults - this.settings = { - proxyPort: parseInt(process.env.PROXY_PORT || '22500'), - gameZipPort: parseInt(process.env.GAMEZIPSERVER_PORT || '22501'), - - legacyHTDOCSPath: path.join(flashpointPath, 'Legacy', 'htdocs'), - legacyCGIBINPath: path.join(flashpointPath, 'Legacy', 'cgi-bin'), - - legacyOverridePaths: proxySettings.legacyOverridePaths || [], - - // Use HTTPS for external sources (servers redirect HTTP to HTTPS anyway) - infinityServerURL: proxySettings.infinityServerURL - ? proxySettings.infinityServerURL.replace(/^http:\/\//, 'https://') - : 'https://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs/', - - externalFilePaths: (proxySettings.externalFilePaths || []).map((url: string) => - url.replace(/^http:\/\//, 'https://') - ), - - mad4fpEnabled: proxySettings.mad4fpEnabled || false, - mad4fpPaths: proxySettings.mad4fpPaths || [], - - chunkSize: 64 * 1024, // 64KB chunks for streaming - maxConcurrentDownloads: 10, - - allowCrossDomain: true, - enableBrotli: true, - enableCGI: process.env.ENABLE_CGI === 'true', - enableHtaccess: process.env.ENABLE_HTACCESS === 'true', - }; - - logger.info('[ProxyConfig] Configuration loaded successfully'); - logger.info(`[ProxyConfig] Proxy port: ${this.settings.proxyPort}`); - logger.info(`[ProxyConfig] GameZip port: ${this.settings.gameZipPort}`); - logger.info(`[ProxyConfig] HTDOCS path: ${this.settings.legacyHTDOCSPath}`); - logger.info(`[ProxyConfig] Override paths: ${this.settings.legacyOverridePaths.length}`); - logger.info(`[ProxyConfig] External sources: ${this.settings.externalFilePaths.length}`); - - return this.settings; - } catch (error) { - logger.error('[ProxyConfig] Failed to load proxySettings.json, using defaults', error); - - // Fallback to defaults - this.settings = { - proxyPort: 22500, - gameZipPort: 22501, - legacyHTDOCSPath: path.join(flashpointPath, 'Legacy', 'htdocs'), - legacyCGIBINPath: path.join(flashpointPath, 'Legacy', 'cgi-bin'), - legacyOverridePaths: [], - infinityServerURL: 'https://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs/', - externalFilePaths: [ - 'https://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs', - 'https://infinity.unstable.life/Flashpoint/Legacy/htdocs/', - ], - mad4fpEnabled: false, - mad4fpPaths: [], - chunkSize: 64 * 1024, - maxConcurrentDownloads: 10, - allowCrossDomain: true, - enableBrotli: true, - enableCGI: false, - enableHtaccess: false, - }; - - return this.settings; - } - } - - static getSettings(): ServerSettings { - if (!this.settings) { - throw new Error('Configuration not loaded. Call loadConfig() first.'); - } - return this.settings; - } -} diff --git a/backend/src/proxy/gamezipserver.ts b/backend/src/proxy/gamezipserver.ts deleted file mode 100644 index 96758fc..0000000 --- a/backend/src/proxy/gamezipserver.ts +++ /dev/null @@ -1,314 +0,0 @@ -import http from 'http'; -import { URL } from 'url'; -import { lookup as mimeLookup } from 'mime-types'; -import path from 'path'; -import { logger } from '../utils/logger'; -import { zipManager } from './zip-manager'; -import { ConfigManager } from './config'; - -/** - * GameZip Server - Serves files from mounted ZIP archives - * Replicates the GameZip server from FlashpointGameServer (port 22501) - */ -export class GameZipServer { - private server: http.Server | null = null; - private settings = ConfigManager.getSettings(); - - /** - * Start the GameZip server - */ - async start(): Promise { - const server = http.createServer(async (req, res) => { - try { - await this.handleRequest(req, res); - } catch (error) { - logger.error('[GameZipServer] Unhandled error:', error); - - if (!res.headersSent) { - res.writeHead(500); - res.end('Internal Server Error'); - } - } - }); - - // Configure server settings - server.timeout = 120000; // 2 minutes timeout - server.keepAliveTimeout = 65000; // 65 seconds keep-alive - - // Start listening - await new Promise((resolve, reject) => { - server.listen(this.settings.gameZipPort, () => { - logger.info(`[GameZipServer] ========================================`); - logger.info(`[GameZipServer] GameZip Server started`); - logger.info(`[GameZipServer] Port: ${this.settings.gameZipPort}`); - logger.info(`[GameZipServer] ========================================`); - resolve(); - }); - - server.on('error', (error: NodeJS.ErrnoException) => { - if (error.code === 'EADDRINUSE') { - logger.error(`[GameZipServer] Port ${this.settings.gameZipPort} is already in use`); - } else { - logger.error('[GameZipServer] Failed to start:', error); - } - reject(error); - }); - }); - - this.server = server; - return server; - } - - /** - * Handle incoming HTTP request - */ - private async handleRequest(req: http.IncomingMessage, res: http.ServerResponse): Promise { - const method = req.method; - const url = req.url || '/'; - - logger.info(`[GameZipServer] ${method} ${url}`); - - // Handle different request types - if (method === 'OPTIONS') { - this.handleOptionsRequest(req, res); - return; - } - - if (method === 'POST' && url.startsWith('/mount/')) { - await this.handleMount(req, res); - return; - } - - if (method === 'DELETE' && url.startsWith('/mount/')) { - await this.handleUnmount(req, res); - return; - } - - if (method === 'GET' && url === '/mounts') { - this.handleListMounts(req, res); - return; - } - - if (method === 'GET' || method === 'HEAD') { - await this.handleFileRequest(req, res); - return; - } - - // Unknown request - this.sendError(res, 404, 'Not Found'); - } - - /** - * Handle CORS preflight - */ - private handleOptionsRequest(req: http.IncomingMessage, res: http.ServerResponse): void { - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, DELETE, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); - res.setHeader('Access-Control-Max-Age', '86400'); - } - - res.writeHead(204); - res.end(); - } - - /** - * Handle POST /mount/:id - Mount a ZIP file - * Body: { zipPath: string } - */ - private async handleMount(req: http.IncomingMessage, res: http.ServerResponse): Promise { - // Extract ID from URL - const id = req.url?.split('/mount/')[1]; - if (!id) { - this.sendError(res, 400, 'Missing mount ID'); - return; - } - - // Parse request body - const body = await this.readBody(req); - let zipPath: string; - - try { - const data = JSON.parse(body); - zipPath = data.zipPath; - - if (!zipPath) { - this.sendError(res, 400, 'Missing zipPath in request body'); - return; - } - } catch (error) { - this.sendError(res, 400, 'Invalid JSON'); - return; - } - - // Mount the ZIP - try { - await zipManager.mount(id, zipPath); - - res.setHeader('Content-Type', 'application/json'); - res.writeHead(200); - res.end(JSON.stringify({ success: true, id, zipPath })); - } catch (error) { - logger.error(`[GameZipServer] Mount failed:`, error); - this.sendError(res, 500, error instanceof Error ? error.message : 'Mount failed'); - } - } - - /** - * Handle DELETE /mount/:id - Unmount a ZIP file - */ - private async handleUnmount(req: http.IncomingMessage, res: http.ServerResponse): Promise { - const id = req.url?.split('/mount/')[1]; - if (!id) { - this.sendError(res, 400, 'Missing mount ID'); - return; - } - - const success = await zipManager.unmount(id); - - res.setHeader('Content-Type', 'application/json'); - res.writeHead(success ? 200 : 404); - res.end(JSON.stringify({ success, id })); - } - - /** - * Handle GET /mounts - List all mounted ZIPs - */ - private handleListMounts(req: http.IncomingMessage, res: http.ServerResponse): void { - const mounts = zipManager.getMountedZips(); - - res.setHeader('Content-Type', 'application/json'); - res.writeHead(200); - res.end(JSON.stringify({ mounts })); - } - - /** - * Handle GET /* - Serve file from mounted ZIPs - * Supports proxy-style requests: GET http://domain.com/path HTTP/1.1 - */ - private async handleFileRequest( - req: http.IncomingMessage, - res: http.ServerResponse - ): Promise { - if (!req.url) { - this.sendError(res, 400, 'Bad Request: No URL'); - return; - } - - let hostname: string; - let urlPath: string; - - // Parse URL (same logic as main proxy server) - if (req.url.startsWith('http://') || req.url.startsWith('https://')) { - // Proxy-style: GET http://domain.com/path HTTP/1.1 - const targetUrl = new URL(req.url); - hostname = targetUrl.hostname; - urlPath = targetUrl.pathname + targetUrl.search; - } else if (req.url.startsWith('/http://') || req.url.startsWith('/https://')) { - // Path-based: GET /http://domain.com/path HTTP/1.1 - const urlWithoutSlash = req.url.substring(1); - const targetUrl = new URL(urlWithoutSlash); - hostname = targetUrl.hostname; - urlPath = targetUrl.pathname + targetUrl.search; - } else { - // Regular: GET /path HTTP/1.1 - hostname = req.headers.host || 'localhost'; - urlPath = req.url; - } - - // Build relative path - const relPath = path.posix.join(hostname, urlPath); - - logger.info(`[GameZipServer] Looking for: ${relPath}`); - - // Search mounted ZIPs - const result = await zipManager.findFile(relPath); - - if (!result) { - logger.debug(`[GameZipServer] File not found in any mounted ZIP: ${relPath}`); - this.sendError(res, 404, 'File not found in mounted ZIPs'); - return; - } - - // Determine content type - const ext = path.extname(urlPath).toLowerCase(); - const contentType = mimeLookup(ext) || 'application/octet-stream'; - - // Send file - logger.info( - `[GameZipServer] ✓ Serving from ZIP ${result.mountId}: ${relPath} (${result.data.length} bytes)` - ); - - // CORS headers - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); - } - - res.setHeader('Content-Type', contentType); - res.setHeader('Content-Length', result.data.length); - res.setHeader('Cache-Control', 'public, max-age=86400'); // Cache for 24 hours - res.setHeader('X-Source', `gamezipserver:${result.mountId}`); - res.setHeader('Connection', 'keep-alive'); - - res.writeHead(200); - res.end(result.data); - } - - /** - * Read request body - */ - private readBody(req: http.IncomingMessage): Promise { - return new Promise((resolve, reject) => { - let body = ''; - req.on('data', (chunk) => { - body += chunk.toString(); - }); - req.on('end', () => { - resolve(body); - }); - req.on('error', reject); - }); - } - - /** - * Send error response - */ - private sendError(res: http.ServerResponse, statusCode: number, message: string): void { - logger.warn(`[GameZipServer] Sending error ${statusCode}: ${message}`); - - // Set CORS headers even for error responses - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, DELETE, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); - } - - res.setHeader('Content-Type', 'text/plain'); - res.writeHead(statusCode); - res.end(message); - } - - /** - * Stop the server - */ - async stop(): Promise { - if (this.server) { - logger.info('[GameZipServer] Stopping server...'); - - await new Promise((resolve, reject) => { - this.server!.close((err) => { - if (err) reject(err); - else resolve(); - }); - }); - - logger.info('[GameZipServer] Server stopped'); - } - - // Unmount all ZIPs - await zipManager.unmountAll(); - } -} diff --git a/backend/src/proxy/http-proxy-server.ts b/backend/src/proxy/http-proxy-server.ts deleted file mode 100644 index c91d57d..0000000 --- a/backend/src/proxy/http-proxy-server.ts +++ /dev/null @@ -1,100 +0,0 @@ -import http from 'http'; -import { logger } from '../utils/logger'; -import { ConfigManager } from './config'; -import { ProxyRequestHandler } from './proxy-request-handler'; - -export interface ProxyServerOptions { - proxyPort?: number; - legacyHTDOCSPath: string; - gameDataPath?: string; - externalFilePaths?: string[]; - allowCrossDomain?: boolean; - chunkSize?: number; -} - -/** - * Create and start the Flashpoint Proxy Server - * This is the HTTP proxy implementation that replicates FlashpointGameServer - */ -export async function createHTTPProxyServer(options: ProxyServerOptions): Promise { - // Load configuration - const flashpointPath = options.legacyHTDOCSPath.replace(/[\\/]Legacy[\\/]htdocs.*$/, ''); - await ConfigManager.loadConfig(flashpointPath); - - const settings = ConfigManager.getSettings(); - - // Override settings from options if provided - if (options.proxyPort) { - settings.proxyPort = options.proxyPort; - } - if (options.allowCrossDomain !== undefined) { - settings.allowCrossDomain = options.allowCrossDomain; - } - if (options.chunkSize) { - settings.chunkSize = options.chunkSize; - } - - // Create request handler - const requestHandler = new ProxyRequestHandler(); - - // Create HTTP server - const server = http.createServer(async (req, res) => { - try { - // Handle CORS preflight - if (req.method === 'OPTIONS') { - requestHandler.handleOptionsRequest(req, res); - return; - } - - // Handle proxy request - await requestHandler.handleRequest(req, res); - } catch (error) { - logger.error('[HTTPProxyServer] Unhandled error:', error); - - if (!res.headersSent) { - res.writeHead(500); - res.end('Internal Server Error'); - } - } - }); - - // Configure server settings - server.timeout = 120000; // 2 minutes timeout - server.keepAliveTimeout = 65000; // 65 seconds keep-alive - - // Start listening - await new Promise((resolve, reject) => { - server.listen(settings.proxyPort, () => { - logger.info(`[HTTPProxyServer] ========================================`); - logger.info(`[HTTPProxyServer] Flashpoint Proxy Server started`); - logger.info(`[HTTPProxyServer] Port: ${settings.proxyPort}`); - logger.info(`[HTTPProxyServer] HTDOCS: ${settings.legacyHTDOCSPath}`); - logger.info(`[HTTPProxyServer] CGI-BIN: ${settings.legacyCGIBINPath}`); - logger.info(`[HTTPProxyServer] External sources: ${settings.externalFilePaths.length}`); - logger.info(`[HTTPProxyServer] ========================================`); - resolve(); - }); - - server.on('error', (error: NodeJS.ErrnoException) => { - if (error.code === 'EADDRINUSE') { - logger.error(`[HTTPProxyServer] Port ${settings.proxyPort} is already in use`); - logger.error('[HTTPProxyServer] Please check if another instance is running'); - } else { - logger.error('[HTTPProxyServer] Failed to start:', error); - } - reject(error); - }); - }); - - return server; -} - -/** - * Create and start the GameZip server - * Serves files from mounted ZIP archives (port 22501) - */ -export async function createGameZipServer(): Promise { - const { GameZipServer } = await import('./gamezipserver'); - const gameZipServer = new GameZipServer(); - return await gameZipServer.start(); -} diff --git a/backend/src/proxy/legacy-server.ts b/backend/src/proxy/legacy-server.ts deleted file mode 100644 index b3e9107..0000000 --- a/backend/src/proxy/legacy-server.ts +++ /dev/null @@ -1,262 +0,0 @@ -import path from 'path'; -import fs from 'fs/promises'; -import { lookup as mimeLookup } from 'mime-types'; -import { logger } from '../utils/logger'; -import { ConfigManager } from './config'; -import axios from 'axios'; -import zlib from 'zlib'; -import { promisify } from 'util'; - -const brotliDecompress = promisify(zlib.brotliDecompress); - -// Script file extensions -const SCRIPT_EXTENSIONS = ['php', 'php5', 'phtml', 'pl']; - -// Index file extensions (in priority order) -const INDEX_EXTENSIONS = ['html', 'htm', 'php', 'swf']; - -export interface LegacyFileResponse { - data: Buffer; - contentType: string; - source: string; -} - -export class LegacyServer { - private settings = ConfigManager.getSettings(); - - /** - * Main file serving method - replicates ServeLegacy() from Go implementation - * @param hostname The hostname from the request (e.g., "www.example.com") - * @param urlPath The path from the request (e.g., "/path/file.swf") - */ - async serveLegacy(hostname: string, urlPath: string): Promise { - // Strip port from hostname - const host = hostname.split(':')[0]; - - // Build relative path: {host}/{path} - const relPath = path.posix.join(host, urlPath); - - logger.info(`[LegacyServer] Serving: ${host}${urlPath}`); - logger.info(`[LegacyServer] Relative path: ${relPath}`); - - // Step 1: Build all path candidates - const pathCandidates = this.buildPathCandidates(relPath, urlPath); - - // Step 2: Try each candidate in order - for (const candidate of pathCandidates) { - logger.debug(`[LegacyServer] Trying: ${candidate.path} (${candidate.type})`); - - try { - const stats = await fs.stat(candidate.path); - - if (stats.isFile()) { - logger.info(`[LegacyServer] ✓ Found file: ${candidate.path}`); - return await this.serveFile(candidate.path, candidate.type === 'cgi-bin'); - } - } catch (error) { - // File doesn't exist, continue to next candidate - continue; - } - } - - // Step 3: No local file found, try external sources - logger.info(`[LegacyServer] File not found locally, trying external sources...`); - return await this.tryExternalSources(relPath); - } - - /** - * Build all path candidates in priority order - * Replicates the path resolution algorithm from Go's ServeLegacy() - */ - private buildPathCandidates( - relPath: string, - urlPath: string - ): Array<{ path: string; type: string }> { - const candidates: Array<{ path: string; type: string }> = []; - - // 1. Exact file paths (with and without query string) - candidates.push({ - path: path.join(this.settings.legacyHTDOCSPath, relPath), - type: 'exact', - }); - - // If URL has query string, try without it - const pathWithoutQuery = relPath.split('?')[0]; - if (pathWithoutQuery !== relPath) { - candidates.push({ - path: path.join(this.settings.legacyHTDOCSPath, pathWithoutQuery), - type: 'exact-no-query', - }); - } - - // 2. Override paths (exact files in override directories) - for (const override of this.settings.legacyOverridePaths) { - candidates.push({ - path: path.join(this.settings.legacyHTDOCSPath, override, relPath), - type: `override:${override}`, - }); - - if (pathWithoutQuery !== relPath) { - candidates.push({ - path: path.join(this.settings.legacyHTDOCSPath, override, pathWithoutQuery), - type: `override:${override}-no-query`, - }); - } - } - - // 3. CGI-BIN paths (for script files) - if (this.isScriptUrl(urlPath)) { - candidates.push({ - path: path.join(this.settings.legacyCGIBINPath, relPath), - type: 'cgi-bin', - }); - - if (pathWithoutQuery !== relPath) { - candidates.push({ - path: path.join(this.settings.legacyCGIBINPath, pathWithoutQuery), - type: 'cgi-bin-no-query', - }); - } - } - - // 4. Index files (for directory requests) - for (const ext of INDEX_EXTENSIONS) { - candidates.push({ - path: path.join(this.settings.legacyHTDOCSPath, relPath, `index.${ext}`), - type: `index:${ext}`, - }); - - // Index files in override paths - for (const override of this.settings.legacyOverridePaths) { - candidates.push({ - path: path.join(this.settings.legacyHTDOCSPath, override, relPath, `index.${ext}`), - type: `index-override:${override}:${ext}`, - }); - } - } - - return candidates; - } - - /** - * Check if URL is a script file - */ - private isScriptUrl(urlPath: string): boolean { - const ext = path.extname(urlPath).toLowerCase().substring(1); - return SCRIPT_EXTENSIONS.includes(ext); - } - - /** - * Serve a file from disk - * @param filePath Absolute path to file - * @param isCGI Whether to execute as CGI script - */ - private async serveFile(filePath: string, isCGI: boolean = false): Promise { - // Read file - let data = await fs.readFile(filePath); - - // Handle Brotli compression - if (this.settings.enableBrotli && filePath.endsWith('.br')) { - logger.debug(`[LegacyServer] Decompressing Brotli file: ${filePath}`); - data = await brotliDecompress(data); - filePath = filePath.substring(0, filePath.length - 3); // Remove .br extension - } - - // Determine content type - const contentType = mimeLookup(filePath) || 'application/octet-stream'; - - // TODO: Execute CGI scripts if enabled - if (isCGI && this.settings.enableCGI) { - logger.warn('[LegacyServer] CGI execution not yet implemented'); - } - - return { - data, - contentType, - source: 'local-htdocs', - }; - } - - /** - * Try external sources (Infinity Server, MAD4FP, etc.) - */ - private async tryExternalSources(relPath: string): Promise { - // Try Infinity Server first - if (this.settings.infinityServerURL) { - logger.info(`[LegacyServer] Trying Infinity Server...`); - const result = await this.tryExternalSource(this.settings.infinityServerURL, relPath); - if (result) { - return { ...result, source: 'infinity-server' }; - } - } - - // Try other external paths - for (const externalPath of this.settings.externalFilePaths) { - logger.info(`[LegacyServer] Trying: ${externalPath}`); - const result = await this.tryExternalSource(externalPath, relPath); - if (result) { - return { ...result, source: externalPath }; - } - } - - // Try MAD4FP if enabled - if (this.settings.mad4fpEnabled) { - for (const mad4fpPath of this.settings.mad4fpPaths) { - logger.info(`[LegacyServer] Trying MAD4FP: ${mad4fpPath}`); - const result = await this.tryExternalSource(mad4fpPath, relPath); - if (result) { - return { ...result, source: `mad4fp:${mad4fpPath}` }; - } - } - } - - throw new Error('File not found in any source'); - } - - /** - * Download from external source - */ - private async tryExternalSource( - baseUrl: string, - relPath: string - ): Promise<{ data: Buffer; contentType: string } | null> { - const normalizedBase = baseUrl.endsWith('/') ? baseUrl : `${baseUrl}/`; - const fullUrl = `${normalizedBase}${relPath}`; - - logger.info(`[LegacyServer] Downloading: ${fullUrl}`); - - try { - const response = await axios.get(fullUrl, { - responseType: 'arraybuffer', - timeout: 45000, - maxRedirects: 5, - validateStatus: (status) => status === 200, - headers: { - 'User-Agent': 'Flashpoint-Proxy/1.0', - }, - }); - - const data = Buffer.from(response.data); - const contentType = response.headers['content-type'] || 'application/octet-stream'; - - logger.info(`[LegacyServer] ✓ Downloaded: ${data.length} bytes (${contentType})`); - - return { data, contentType }; - } catch (error) { - // Log but don't throw - allow trying next source - if (axios.isAxiosError(error)) { - if (error.response?.status === 404) { - logger.debug(`[LegacyServer] ✗ Not found (404)`); - } else if (error.response?.status === 403) { - logger.warn(`[LegacyServer] ✗ Forbidden (403)`); - } else if (error.code === 'ECONNABORTED' || error.code === 'ETIMEDOUT') { - logger.warn(`[LegacyServer] ✗ Timeout`); - } else { - logger.warn(`[LegacyServer] ✗ Error: ${error.message}`); - } - } - - return null; - } - } -} diff --git a/backend/src/proxy/mimeTypes.ts b/backend/src/proxy/mimeTypes.ts deleted file mode 100644 index 2193c87..0000000 --- a/backend/src/proxy/mimeTypes.ts +++ /dev/null @@ -1,278 +0,0 @@ -/** - * MIME type mappings from Flashpoint proxySettings.json - * Includes 199 custom MIME types for legacy web content - */ - -export const CUSTOM_MIME_TYPES: { [key: string]: string } = { - // Legacy multimedia formats - aab: 'application/x-authorware-bin', - aam: 'application/x-authorware-map', - aas: 'application/x-authorware-seg', - afl: 'video/animaflex', - aif: 'audio/aiff', - aifc: 'audio/aiff', - aiff: 'audio/aiff', - asd: 'application/astound', - asmx: 'text/xml', - asn: 'application/astound', - au: 'audio/basic', - aut: 'application/pbautomation', - aw3: 'application/x-awingsoft-winds3d', - axs: 'application/x-MindAvenueAXELStream', - - // 3D and modeling formats - blend: 'application/x-burster', - blendz: 'application/x-burster', - blz: 'video/blz', - bswrl: 'application/x-bscontact', - bub: 'application/photobubble', - bxwrl: 'application/x-blaxxuncc3d', - - // Director and Shockwave - ccn: 'application/x-cnc', - cct: 'application/x-director', - cnc: 'application/x-cnc', - cst: 'application/x-director', - cxt: 'application/x-director', - dcr: 'application/x-director', - dir: 'application/x-director', - dxr: 'application/x-director', - swa: 'application/x-director', - w3d: 'application/x-director', - - // Chemical and scientific formats - cdx: 'chemical/x-cdx', - chm: 'chemical/x-chemdraw', - cow: 'chemical/x-cow', - csm: 'chemical/x-csml', - csml: 'chemical/x-csml', - cub: 'chemical/x-gaussian-cube', - cube: 'chemical/x-gaussian-cube', - dx: 'chemical/x-jcamp-dx', - emb: 'chemical/x-pdb', - embl: 'chemical/x-pdb', - gau: 'chemical/x-gaussian-input', - jdx: 'chemical/x-jcamp-dx', - mol: 'chemical/x-mdl-molfile', - mop: 'chemical/x-mopac-input', - pdb: 'chemical/x-pdb', - rxn: 'chemical/x-mdl-rxnfile', - skc: 'chemical/x-mdl-tgf', - tgf: 'chemical/x-mdl-tgf', - xyz: 'chemical/x-xyz', - - // Image formats - cgm: 'image/cgm', - cit: 'image/cit', - dgn: 'image/dgn', - djvu: 'image/vnd.djvu', - fh4: 'image/x-freehand4', - fh5: 'image/x-freehand5', - fh7: 'image/x-freehand7', - fhc: 'image/x-freehand', - jp2: 'image/jp2', - jp2k: 'image/jp2', - mcf: 'image/vasa', - qdgx: 'image/x-qdgx', - rle: 'image/rle', - svg: 'image/svg+xml', - svgz: 'image/svg+xml', - twf: 'image/x-twf', - twfz: 'image/x-twf-zlib-compressed', - vec: 'image/vec', - - // Audio formats - it: 'audio/it', - itz: 'audio/x-zipped-it', - mdz: 'audio/x-zipped-mod', - med: 'audio/x-mod', - mid: 'audio/mid', - midi: 'audio/midi', - mod: 'audio/mod', - s3m: 'audio/s3m', - s3z: 'audio/x-mod', - sid: 'audio/x-sidtune', - wav: 'audio/wav', - xm: 'audio/xm', - xmz: 'audio/x-mod', - - // Video formats - mov: 'video/quicktime', - - // Application formats - class: 'application/java', - cmo: 'application/x-virtools', - co: 'application/x-cult3d-object', - css: 'text/css', - deepv: 'application/x-deepv', - dpg: 'application/vnd.dpgraph', - dpgraph: 'application/vnd.dpgraph', - dsn: 'application/x-altiadsn', - dvl: 'application/x-devalvrx', - elec: 'application/x-electrifier', - eva: 'application/x-eva', - evy: 'application/envoy', - - // Compressed formats - gz: 'application/x-gzip-compressed', - wrz: 'application/x-gzip-compressed', - x3dz: 'application/x-gzip-compressed', - - // Web formats - htm: 'text/html', - html: 'text/html', - js: 'text/javascript', - mjs: 'text/javascript', - json: 'application/json', - xml: 'application/xml', - - // Plugin formats - ips: 'application/x-ipscript', - ipx: 'application/x-ipix', - jar: 'application/java-archive', - mwc: 'application/vnd.dpgraph', - mwf: 'application/x-mwf', - nmo: 'application/x-virtools', - nms: 'application/x-virtools', - p3d: 'application/x-p3d', - pqf: 'application/x-cprplayer', - pqi: 'application/cprplayer', - pw3: 'application/x-pulse-player-32', - pwc: 'application/x-pulse-player', - pwn: 'application/x-pulse-download', - pws: 'application/x-pulse-stream', - sca: 'application/x-supercard', - scr: 'application/x-rasmol', - smp: 'application/studiom', - spl: 'application/futuresplash', - sts: 'application/x-squeak-source', - swf: 'application/x-shockwave-flash', - tbk: 'application/toolbook', - tcl: 'application/x-tcl', - thp: 'plugin/x-theorist', - tv: 'application/x-alambik-script', - tvb: 'application/x-alambik-script', - tvd: 'application/x-alambik-script', - tvs: 'application/x-alambik-script', - tvv: 'application/x-alambik-script', - unity3d: 'application/vnd.unity', - vmo: 'application/x-virtools', - vobj: 'application/x-netscape-vae-plugin-vae', - wasm: 'application/wasm', - web: 'application/vnd.xara', - xap: 'application/x-silverlight-app', - xar: 'application/vnd.xara', - xpg: 'text/x-xpg', - - // World/3D formats - d96: 'x-world/x-d96', - dae: 'model/x-bs-collada+xml', - mus: 'x-world/x-d96', - rbs: 'x-world/realibase', - svf: 'vector/x-svf', - svr: 'x-world/x-svr', - vrt: 'x-world/x-vrt', - wrl: 'model/vrml', - wvr: 'x-world/x-wvr', - x3db: 'model/x3d+binary', - xvr: 'x-world/x-xvr', -}; - -// Standard MIME types for common web files -const STANDARD_MIME_TYPES: { [key: string]: string } = { - // Text - txt: 'text/plain', - css: 'text/css', - csv: 'text/csv', - - // Images - png: 'image/png', - jpg: 'image/jpeg', - jpeg: 'image/jpeg', - gif: 'image/gif', - bmp: 'image/bmp', - ico: 'image/x-icon', - webp: 'image/webp', - - // Audio - mp3: 'audio/mpeg', - ogg: 'audio/ogg', - oga: 'audio/ogg', - - // Video - mp4: 'video/mp4', - webm: 'video/webm', - ogv: 'video/ogg', - avi: 'video/x-msvideo', - - // Fonts - woff: 'font/woff', - woff2: 'font/woff2', - ttf: 'font/ttf', - otf: 'font/otf', - eot: 'application/vnd.ms-fontobject', - - // Archives - zip: 'application/zip', - rar: 'application/x-rar-compressed', - '7z': 'application/x-7z-compressed', - tar: 'application/x-tar', - - // Documents - pdf: 'application/pdf', - doc: 'application/msword', - docx: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', - xls: 'application/vnd.ms-excel', - xlsx: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', - ppt: 'application/vnd.ms-powerpoint', - pptx: 'application/vnd.openxmlformats-officedocument.presentationml.presentation', - - // Binary - bin: 'application/octet-stream', - exe: 'application/octet-stream', - dll: 'application/octet-stream', -}; - -/** - * Get MIME type for a file extension - * Checks custom Flashpoint MIME types first, then standard types - */ -export function getMimeType(extension: string): string { - const ext = extension.toLowerCase(); - - // Check custom MIME types first (priority for legacy content) - if (CUSTOM_MIME_TYPES[ext]) { - return CUSTOM_MIME_TYPES[ext]; - } - - // Check standard MIME types - if (STANDARD_MIME_TYPES[ext]) { - return STANDARD_MIME_TYPES[ext]; - } - - // Default to octet-stream for unknown types - return 'application/octet-stream'; -} - -/** - * Check if a file extension is a gzipped type - */ -export function isGzippedType(extension: string): boolean { - const gzippedTypes = ['svgz']; - return gzippedTypes.includes(extension.toLowerCase()); -} - -/** - * Check if a file extension is a script type - */ -export function isScriptType(extension: string): boolean { - const scriptTypes = ['php', 'php5', 'phtml', 'pl']; - return scriptTypes.includes(extension.toLowerCase()); -} - -/** - * Get list of index file types - */ -export function getIndexFileTypes(): string[] { - return ['html', 'htm', 'php', 'php5', 'phtml', 'asp', 'aspx', 'jsp', 'jspx']; -} diff --git a/backend/src/proxy/proxy-request-handler.ts b/backend/src/proxy/proxy-request-handler.ts deleted file mode 100644 index a0663a3..0000000 --- a/backend/src/proxy/proxy-request-handler.ts +++ /dev/null @@ -1,180 +0,0 @@ -import { IncomingMessage, ServerResponse } from 'http'; -import { URL } from 'url'; -import { logger } from '../utils/logger'; -import { ConfigManager } from './config'; -import { LegacyServer } from './legacy-server'; -import axios from 'axios'; - -export class ProxyRequestHandler { - private legacyServer: LegacyServer; - private settings = ConfigManager.getSettings(); - - constructor() { - this.legacyServer = new LegacyServer(); - } - - /** - * Handle incoming HTTP proxy request - * Replicates the request processing flow from Go's goproxy - */ - async handleRequest(req: IncomingMessage, res: ServerResponse): Promise { - try { - logger.info(`[ProxyHandler] ${req.method} ${req.url}`); - - // Parse the request URL - // For proxy requests, req.url contains the full URL: http://example.com/path - // For regular requests, req.url contains just the path: /path - if (!req.url) { - this.sendError(res, 400, 'Bad Request: No URL'); - return; - } - - let targetUrl: URL; - let hostname: string; - let urlPath: string; - - // Check if this is a proxy-style request (starts with http://) - if (req.url.startsWith('http://') || req.url.startsWith('https://')) { - // Proxy-style request: GET http://example.com/path HTTP/1.1 - targetUrl = new URL(req.url); - hostname = targetUrl.hostname; - urlPath = targetUrl.pathname + targetUrl.search; - } else if (req.url.startsWith('/http://') || req.url.startsWith('/https://')) { - // Path-based proxy request: GET /http://example.com/path HTTP/1.1 - // Strip leading slash and parse as URL - const urlWithoutSlash = req.url.substring(1); - targetUrl = new URL(urlWithoutSlash); - hostname = targetUrl.hostname; - urlPath = targetUrl.pathname + targetUrl.search; - } else { - // Regular request: GET /path HTTP/1.1 with Host header - hostname = req.headers.host || 'localhost'; - urlPath = req.url; - targetUrl = new URL(`http://${hostname}${urlPath}`); - } - - logger.info(`[ProxyHandler] Parsed - Host: ${hostname}, Path: ${urlPath}`); - - // Step 1: Try GameZip server first (if enabled) - if (this.settings.gameZipPort) { - logger.debug(`[ProxyHandler] Trying GameZip server...`); - const gameZipResult = await this.tryGameZipServer(hostname, urlPath); - if (gameZipResult) { - logger.info(`[ProxyHandler] ✓ Served from GameZip`); - this.sendResponse(res, gameZipResult.data, gameZipResult.contentType, 'gamezipserver'); - return; - } - } - - // Step 2: Try legacy file serving (local htdocs + external fallbacks) - logger.debug(`[ProxyHandler] Trying legacy server...`); - const legacyResult = await this.legacyServer.serveLegacy(hostname, urlPath); - - logger.info(`[ProxyHandler] ✓ Served from ${legacyResult.source}`); - this.sendResponse(res, legacyResult.data, legacyResult.contentType, legacyResult.source); - } catch (error) { - logger.error('[ProxyHandler] Error handling request:', error); - - if (error instanceof Error && error.message.includes('not found')) { - this.sendError(res, 404, 'File not found in any source'); - } else { - this.sendError(res, 500, 'Internal server error'); - } - } - } - - /** - * Try to fetch from GameZip server - */ - private async tryGameZipServer( - hostname: string, - urlPath: string - ): Promise<{ data: Buffer; contentType: string } | null> { - try { - // Make proxy-style request to GameZip server - // The GameZip server also accepts proxy-style requests: GET http://hostname/path HTTP/1.1 - const gameZipServerUrl = `http://localhost:${this.settings.gameZipPort}`; - const targetUrl = `http://${hostname}${urlPath}`; - const requestUrl = `${gameZipServerUrl}/${targetUrl}`; - - const response = await axios.get(requestUrl, { - responseType: 'arraybuffer', - timeout: 5000, // Quick timeout for local server - validateStatus: (status) => status === 200, - proxy: false, // Don't use system proxy - }); - - const data = Buffer.from(response.data); - const contentType = response.headers['content-type'] || 'application/octet-stream'; - - return { data, contentType }; - } catch (error) { - // GameZip server not available or file not found - logger.debug(`[ProxyHandler] GameZip unavailable or file not found`); - return null; - } - } - - /** - * Send successful response - */ - private sendResponse( - res: ServerResponse, - data: Buffer, - contentType: string, - source: string - ): void { - // CORS headers - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); - } - - // Content headers - res.setHeader('Content-Type', contentType); - res.setHeader('Content-Length', data.length); - res.setHeader('Cache-Control', 'public, max-age=86400'); // Cache for 24 hours - res.setHeader('X-Source', source); // Debug header - - // Connection keep-alive - res.setHeader('Connection', 'keep-alive'); - - res.writeHead(200); - res.end(data); - } - - /** - * Send error response - */ - private sendError(res: ServerResponse, statusCode: number, message: string): void { - logger.warn(`[ProxyHandler] Sending error ${statusCode}: ${message}`); - - // IMPORTANT: Set CORS headers even for error responses - // Otherwise browsers will show CORS error instead of the actual error - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); - } - - res.setHeader('Content-Type', 'text/plain'); - res.writeHead(statusCode); - res.end(message); - } - - /** - * Handle OPTIONS request for CORS preflight - */ - handleOptionsRequest(req: IncomingMessage, res: ServerResponse): void { - if (this.settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); - res.setHeader('Access-Control-Max-Age', '86400'); - } - - res.writeHead(204); - res.end(); - } -} diff --git a/backend/src/proxy/server.ts b/backend/src/proxy/server.ts deleted file mode 100644 index 256d83b..0000000 --- a/backend/src/proxy/server.ts +++ /dev/null @@ -1,520 +0,0 @@ -import express, { Request, Response, NextFunction } from 'express'; -import axios from 'axios'; -import fs from 'fs'; -import path from 'path'; -import { logger } from '../utils/logger'; -import { getMimeType } from './mimeTypes'; -import { zipManager } from './zip-manager'; - -export interface ProxyConfig { - proxyPort: number; - legacyHTDOCSPath: string; - gameDataPath: string; - externalFilePaths: string[]; - allowCrossDomain: boolean; - chunkSize?: number; -} - -export class GameProxyServer { - private app: express.Application; - private config: ProxyConfig; - - constructor(proxyConfig: ProxyConfig) { - this.app = express(); - this.config = { - ...proxyConfig, - chunkSize: proxyConfig.chunkSize || 8192, // 8KB default chunk size - }; - this.setupMiddleware(); - this.setupRoutes(); - } - - private setupMiddleware(): void { - // CORS middleware - if (this.config.allowCrossDomain) { - this.app.use((req, res, next) => { - res.header('Access-Control-Allow-Origin', '*'); - res.header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS'); - res.header( - 'Access-Control-Allow-Headers', - 'Origin, X-Requested-With, Content-Type, Accept' - ); - - if (req.method === 'OPTIONS') { - res.sendStatus(200); - } else { - next(); - } - }); - } - - // Request logging - this.app.use((req, res, next) => { - logger.debug(`[Proxy] ${req.method} ${req.path}`); - next(); - }); - } - - private setupRoutes(): void { - // Main proxy route - handles all requests - this.app.get('*', async (req: Request, res: Response, next: NextFunction) => { - try { - await this.handleProxyRequest(req, res); - } catch (error) { - next(error); - } - }); - - // Error handler - this.app.use((err: Error, req: Request, res: Response, _next: NextFunction) => { - logger.error('[Proxy] Error:', err); - res.status(500).json({ - error: { - message: 'Proxy server error', - statusCode: 500, - }, - }); - }); - } - - private async handleProxyRequest(req: Request, res: Response): Promise { - let requestPath = req.path.substring(1); // Remove leading slash - - // Parse URL if the request path is a full URL (e.g., http://domain.com/path) - const parsedPath = this.parseProxyPath(requestPath); - requestPath = parsedPath; - - logger.debug(`[Proxy] Resolved path: ${requestPath}`); - - // Try to serve from local paths first - const localFile = await this.tryLocalPaths(requestPath); - - if (localFile) { - await this.serveFile(localFile, res); - return; - } - - // Try to serve from mounted ZIP files (mimics Flashpoint ZipServer) - const zipContent = await this.tryZipFiles(requestPath); - - if (zipContent) { - await this.serveZipContent(zipContent, requestPath, res); - return; - } - - // Try external fallback URLs - const externalContent = await this.tryExternalPaths(requestPath); - - if (externalContent) { - await this.serveExternalContent(externalContent, res); - return; - } - - // File not found - logger.warn(`[Proxy] File not found: ${requestPath}`); - res.status(404).json({ - error: { - message: 'File not found', - path: requestPath, - statusCode: 404, - }, - }); - } - - /** - * Parse proxy path - converts full URLs to local file paths - * Examples: - * http://farm.maxgames.com/game.swf -> farm.maxgames.com/game.swf - * https://example.com/path/file.html -> example.com/path/file.html - * http://www.example.com/Find%20My%20File.swf -> www.example.com/Find My File.swf - * regular/path/file.swf -> regular/path/file.swf (unchanged) - */ - private parseProxyPath(requestPath: string): string { - try { - // Check if it's a full URL (starts with http:// or https://) - if (requestPath.startsWith('http://') || requestPath.startsWith('https://')) { - const url = new URL(requestPath); - - // CRITICAL FIX: Decode the pathname to handle URL-encoded characters (spaces, etc.) - // The JavaScript URL class preserves encoding, but filesystem paths need actual characters - // This matches how PHP's web server automatically decodes paths for the Flashpoint Launcher - const decodedPathname = decodeURIComponent(url.pathname); - const decodedSearch = url.search ? decodeURIComponent(url.search) : ''; - - // Combine hostname and decoded pathname, removing leading slash from pathname - const localPath = url.hostname + decodedPathname + decodedSearch; - logger.debug(`[Proxy] Parsed URL: ${requestPath} -> ${localPath}`); - return localPath; - } - - // If not a URL, return as-is - return requestPath; - } catch (error) { - logger.warn(`[Proxy] Failed to parse URL: ${requestPath}, using as-is`); - return requestPath; - } - } - - private async tryLocalPaths(requestPath: string): Promise { - // Priority order: - // 1. Legacy htdocs path - // 2. Game data path - - const pathsToTry = [ - path.join(this.config.legacyHTDOCSPath, requestPath), - path.join(this.config.gameDataPath, requestPath), - ]; - - for (const filePath of pathsToTry) { - try { - if (fs.existsSync(filePath)) { - const stats = fs.statSync(filePath); - - if (stats.isFile()) { - return filePath; - } else if (stats.isDirectory()) { - // Try index files - const indexFiles = ['index.html', 'index.htm', 'index.php']; - for (const indexFile of indexFiles) { - const indexPath = path.join(filePath, indexFile); - if (fs.existsSync(indexPath) && fs.statSync(indexPath).isFile()) { - return indexPath; - } - } - } - } - } catch (error) { - logger.debug(`[Proxy] Error checking path ${filePath}:`, error); - } - } - - return null; - } - - /** - * Try to serve file from mounted ZIP archives in Data/Games - * This replicates the ZipServer functionality from FlashpointGameServer (port 22501) - */ - private async tryZipFiles(requestPath: string): Promise { - logger.debug(`[Proxy] Checking ZIP files for: ${requestPath}`); - - try { - // The zipManager.findFile() already searches all mounted ZIPs - // It tries multiple path variants: content/, htdocs/, etc. - const result = await zipManager.findFile(requestPath); - - if (result) { - logger.info( - `[Proxy] ✓ Found in ZIP ${result.mountId}: ${requestPath} (${result.data.length} bytes)` - ); - return result.data; - } - - logger.debug(`[Proxy] File not found in any mounted ZIP: ${requestPath}`); - return null; - } catch (error) { - logger.error(`[Proxy] Error searching ZIP files:`, error); - return null; - } - } - - /** - * Auto-discover and mount all ZIP files in Data/Games directory - * This is called on server startup to make game ZIPs available - */ - private async autoMountZipFiles(): Promise { - try { - const gamesPath = this.config.gameDataPath; - - if (!fs.existsSync(gamesPath)) { - logger.warn(`[Proxy] Games directory not found: ${gamesPath}`); - logger.warn(`[Proxy] ZIP file serving will be disabled`); - return; - } - - logger.info(`[Proxy] Auto-mounting ZIP files from: ${gamesPath}`); - - // Find all .zip files in Data/Games - const files = fs.readdirSync(gamesPath); - const zipFiles = files.filter((f) => f.toLowerCase().endsWith('.zip')); - - if (zipFiles.length === 0) { - logger.info(`[Proxy] No ZIP files found in ${gamesPath}`); - return; - } - - logger.info(`[Proxy] Found ${zipFiles.length} ZIP file(s) to mount`); - - // Mount each ZIP file - let mountedCount = 0; - let failedCount = 0; - - for (const zipFile of zipFiles) { - try { - const zipPath = path.join(gamesPath, zipFile); - // Use filename (without extension) as mount ID - const mountId = path.basename(zipFile, '.zip'); - - await zipManager.mount(mountId, zipPath); - mountedCount++; - } catch (error) { - logger.warn(`[Proxy] Failed to mount ${zipFile}:`, error); - failedCount++; - } - } - - logger.info(`[Proxy] ========================================`); - logger.info(`[Proxy] ZIP Auto-Mount Summary:`); - logger.info(`[Proxy] Total found: ${zipFiles.length}`); - logger.info(`[Proxy] Mounted: ${mountedCount}`); - if (failedCount > 0) { - logger.warn(`[Proxy] Failed: ${failedCount}`); - } - logger.info(`[Proxy] ========================================`); - } catch (error) { - logger.error(`[Proxy] Error during ZIP auto-mount:`, error); - } - } - - /** - * Serve content from ZIP file - */ - private async serveZipContent( - content: Buffer, - requestPath: string, - res: Response - ): Promise { - try { - // Determine MIME type from file extension - const ext = path.extname(requestPath).substring(1).toLowerCase(); - const mimeType = getMimeType(ext); - - res.setHeader('Content-Type', mimeType); - res.setHeader('Content-Length', content.length); - res.setHeader('X-Source', 'zip'); - res.setHeader('Cache-Control', 'public, max-age=86400'); // Cache for 24 hours - - res.send(content); - - logger.debug( - `[Proxy] Served from ZIP: ${requestPath} (${mimeType}, ${content.length} bytes)` - ); - } catch (error) { - logger.error('[Proxy] Error serving ZIP content:', error); - throw error; - } - } - - /** - * Encode URL for CDN access: - * - Ensure path segments are properly encoded - * - Preserve URL structure (protocol, domain, query params) - * - Handle already-encoded segments gracefully - */ - private encodeUrlForCDN(url: string): string { - try { - const urlObj = new URL(url); - - // Encode each path segment individually - // This prevents double-encoding while ensuring special characters are handled - const pathSegments = urlObj.pathname.split('/'); - const encodedSegments = pathSegments.map((segment) => { - if (!segment) return segment; // Preserve empty segments (leading/trailing slashes) - - try { - // Try to decode first - if it succeeds, the segment might be already encoded - const decoded = decodeURIComponent(segment); - // Re-encode to ensure consistency - return encodeURIComponent(decoded); - } catch { - // If decode fails, segment is either already properly encoded or has invalid encoding - // Try to encode it directly - return encodeURIComponent(segment); - } - }); - - urlObj.pathname = encodedSegments.join('/'); - - // Also encode query parameters properly - if (urlObj.search) { - const params = new URLSearchParams(urlObj.search); - urlObj.search = params.toString(); - } - - return urlObj.toString(); - } catch (error) { - // If URL parsing fails, return original URL - logger.warn( - `[Proxy] Failed to encode URL for CDN: ${error instanceof Error ? error.message : 'Unknown error'}` - ); - return url; - } - } - - private async tryExternalPaths(requestPath: string): Promise { - logger.info(`[Proxy] Trying external CDN for: ${requestPath}`); - - for (const baseUrl of this.config.externalFilePaths) { - try { - const url = `${baseUrl.trim()}/${requestPath}`; - // Encode URL like PHP router does - const encodedUrl = this.encodeUrlForCDN(url); - logger.info(`[Proxy] Attempting: ${encodedUrl}`); - - const response = await axios.get(encodedUrl, { - responseType: 'arraybuffer', - timeout: 30000, // 30 second timeout - maxRedirects: 5, - headers: { - // Match PHP router's Accept-Encoding header - 'Accept-Encoding': 'identity', - }, - // Note: Removed validateStatus to allow axios to follow redirects automatically - }); - - if (response.status === 200 && response.data) { - const buffer = Buffer.from(response.data); - - // Archive.org serves empty files instead of 404s - check for this - if (buffer.length === 0) { - logger.info(`[Proxy] ✗ Empty file returned (treated as 404): ${encodedUrl}`); - continue; - } - - logger.info( - `[Proxy] ✓ Successfully fetched from external: ${encodedUrl} (${buffer.length} bytes)` - ); - - // Cache the downloaded file locally for future use - await this.cacheDownloadedFile(requestPath, buffer); - - return buffer; - } - } catch (error) { - let errorMsg = 'Unknown error'; - if (error && typeof error === 'object') { - const err = error as { response?: { status?: number }; message?: string }; - errorMsg = err.response?.status - ? `HTTP ${err.response.status}` - : err.message || 'Unknown error'; - } - logger.info(`[Proxy] ✗ Failed (${errorMsg}): ${baseUrl.trim()}/${requestPath}`); - } - } - - logger.warn(`[Proxy] All CDN sources exhausted for: ${requestPath}`); - return null; - } - - /** - * Cache downloaded file locally for future use (like Flashpoint Launcher does) - */ - private async cacheDownloadedFile(requestPath: string, content: Buffer): Promise { - try { - // Determine the best local path (prefer Legacy/htdocs) - const localPath = path.join(this.config.legacyHTDOCSPath, requestPath); - const dir = path.dirname(localPath); - - // Create directory structure if it doesn't exist - if (!fs.existsSync(dir)) { - fs.mkdirSync(dir, { recursive: true }); - } - - // Write the file - fs.writeFileSync(localPath, content); - logger.info(`[Proxy] Cached downloaded file: ${localPath}`); - } catch (error) { - logger.warn(`[Proxy] Failed to cache file locally:`, error); - // Don't throw - caching is optional, we already have the content - } - } - - private async serveFile(filePath: string, res: Response): Promise { - try { - const ext = path.extname(filePath).substring(1).toLowerCase(); - const mimeType = getMimeType(ext); - - res.setHeader('Content-Type', mimeType); - - const stats = fs.statSync(filePath); - res.setHeader('Content-Length', stats.size); - - // Stream file with chunking for large files - const readStream = fs.createReadStream(filePath, { - highWaterMark: this.config.chunkSize, - }); - - readStream.on('error', (error) => { - logger.error('[Proxy] Stream error:', error); - if (!res.headersSent) { - res.status(500).json({ - error: { - message: 'Error streaming file', - statusCode: 500, - }, - }); - } - }); - - readStream.pipe(res); - - logger.debug(`[Proxy] Serving local file: ${filePath} (${mimeType})`); - } catch (error) { - logger.error('[Proxy] Error serving file:', error); - throw error; - } - } - - private async serveExternalContent(content: Buffer, res: Response): Promise { - try { - // Try to determine MIME type from content or default to octet-stream - const mimeType = 'application/octet-stream'; // Will be improved with actual detection - - res.setHeader('Content-Type', mimeType); - res.setHeader('Content-Length', content.length); - res.send(content); - } catch (error) { - logger.error('[Proxy] Error serving external content:', error); - throw error; - } - } - - public async start(): Promise { - // Auto-mount ZIP files from Data/Games before starting the server - await this.autoMountZipFiles(); - - return new Promise((resolve, reject) => { - const server = this.app.listen(this.config.proxyPort, () => { - logger.info(`[Proxy] Game proxy server listening on port ${this.config.proxyPort}`); - resolve(); - }); - - server.on('error', (error: NodeJS.ErrnoException) => { - if (error.code === 'EADDRINUSE') { - logger.error(`[Proxy] Port ${this.config.proxyPort} is already in use.`); - logger.error( - '[Proxy] Please check if another instance is running or change PROXY_PORT in .env' - ); - logger.error( - '[Proxy] You can find the process using: netstat -ano | findstr :' + - this.config.proxyPort - ); - } else { - logger.error('[Proxy] Failed to start proxy server:', error); - } - reject(error); - }); - }); - } - - public getApp(): express.Application { - return this.app; - } -} - -// Factory function to create and start proxy server -export async function createProxyServer(proxyConfig: ProxyConfig): Promise { - const server = new GameProxyServer(proxyConfig); - await server.start(); - return server; -} diff --git a/backend/src/proxy/zip-manager.ts b/backend/src/proxy/zip-manager.ts deleted file mode 100644 index 0ca6044..0000000 --- a/backend/src/proxy/zip-manager.ts +++ /dev/null @@ -1,210 +0,0 @@ -import StreamZip from 'node-stream-zip'; -import fs from 'fs/promises'; -import { logger } from '../utils/logger'; - -interface MountedZip { - id: string; - zipPath: string; - zip: StreamZip.StreamZipAsync; - mountTime: Date; -} - -export class ZipManager { - private mountedZips: Map = new Map(); - - /** - * Mount a ZIP file for serving - * @param id Unique identifier for this mount (e.g., game data ID) - * @param zipPath Absolute path to the ZIP file - */ - async mount(id: string, zipPath: string): Promise { - // Check if already mounted - if (this.mountedZips.has(id)) { - logger.debug(`[ZipManager] ZIP already mounted: ${id}`); - return; - } - - // Verify ZIP file exists - try { - await fs.access(zipPath); - } catch (error) { - throw new Error(`ZIP file not found: ${zipPath}`); - } - - logger.info(`[ZipManager] Mounting ZIP: ${id} -> ${zipPath}`); - - // Create ZIP reader - const zip = new StreamZip.async({ file: zipPath }); - - // Store mounted ZIP - this.mountedZips.set(id, { - id, - zipPath, - zip, - mountTime: new Date(), - }); - - logger.info(`[ZipManager] ✓ Mounted ZIP: ${id} (${await this.getFileCount(id)} files)`); - } - - /** - * Unmount a ZIP file - * @param id Unique identifier of the mounted ZIP - */ - async unmount(id: string): Promise { - const mounted = this.mountedZips.get(id); - if (!mounted) { - logger.warn(`[ZipManager] ZIP not mounted: ${id}`); - return false; - } - - logger.info(`[ZipManager] Unmounting ZIP: ${id}`); - - try { - await mounted.zip.close(); - this.mountedZips.delete(id); - logger.info(`[ZipManager] ✓ Unmounted ZIP: ${id}`); - return true; - } catch (error) { - logger.error(`[ZipManager] Error unmounting ZIP: ${id}`, error); - return false; - } - } - - /** - * Get a file from a mounted ZIP - * @param id ZIP mount ID - * @param filePath Path within the ZIP (e.g., "htdocs/www.example.com/file.swf") - */ - async getFile(id: string, filePath: string): Promise { - const mounted = this.mountedZips.get(id); - if (!mounted) { - logger.debug(`[ZipManager] ZIP not mounted: ${id}`); - return null; - } - - try { - // Normalize path (remove leading slash) - const normalizedPath = filePath.startsWith('/') ? filePath.substring(1) : filePath; - - logger.debug(`[ZipManager] Reading from ZIP ${id}: ${normalizedPath}`); - - // Get file from ZIP - const data = await mounted.zip.entryData(normalizedPath); - - logger.debug(`[ZipManager] ✓ Read ${data.length} bytes from ${id}:${normalizedPath}`); - - return data; - } catch (error) { - // File not found in ZIP - logger.debug(`[ZipManager] File not found in ${id}: ${filePath}`); - return null; - } - } - - /** - * Search for a file across all mounted ZIPs - * @param relPath Relative path (e.g., "www.example.com/path/file.swf") - * @returns Buffer and mount ID if found - */ - async findFile(relPath: string): Promise<{ data: Buffer; mountId: string } | null> { - // Try with different prefixes (different ZIP structures) - const pathsToTry = [ - `content/${relPath}`, // Most common: content/domain/path - `htdocs/${relPath}`, // Standard: htdocs/domain/path - relPath, // No prefix: domain/path - `Legacy/htdocs/${relPath}`, // Full path: Legacy/htdocs/domain/path - ]; - - // Search all mounted ZIPs - for (const [id, _mounted] of this.mountedZips) { - for (const pathVariant of pathsToTry) { - const data = await this.getFile(id, pathVariant); - if (data) { - logger.info(`[ZipManager] ✓ Found in ${id}: ${pathVariant}`); - return { data, mountId: id }; - } - } - } - - return null; - } - - /** - * Get list of mounted ZIPs - */ - getMountedZips(): Array<{ id: string; zipPath: string; mountTime: Date; fileCount: number }> { - const result: Array<{ id: string; zipPath: string; mountTime: Date; fileCount: number }> = []; - - for (const [id, mounted] of this.mountedZips) { - result.push({ - id, - zipPath: mounted.zipPath, - mountTime: mounted.mountTime, - fileCount: 0, // Will be populated async - }); - } - - return result; - } - - /** - * Get number of files in a mounted ZIP - */ - private async getFileCount(id: string): Promise { - const mounted = this.mountedZips.get(id); - if (!mounted) return 0; - - try { - const entries = await mounted.zip.entries(); - return Object.keys(entries).length; - } catch (error) { - return 0; - } - } - - /** - * List files in a mounted ZIP (for debugging) - */ - async listFiles(id: string, pattern?: string): Promise { - const mounted = this.mountedZips.get(id); - if (!mounted) return []; - - try { - const entries = await mounted.zip.entries(); - let files = Object.keys(entries); - - if (pattern) { - const regex = new RegExp(pattern, 'i'); - files = files.filter((f) => regex.test(f)); - } - - return files; - } catch (error) { - logger.error(`[ZipManager] Error listing files in ${id}:`, error); - return []; - } - } - - /** - * Check if a ZIP is mounted - */ - isMounted(id: string): boolean { - return this.mountedZips.has(id); - } - - /** - * Unmount all ZIPs (cleanup on shutdown) - */ - async unmountAll(): Promise { - logger.info(`[ZipManager] Unmounting all ZIPs (${this.mountedZips.size})...`); - - const promises = Array.from(this.mountedZips.keys()).map((id) => this.unmount(id)); - await Promise.all(promises); - - logger.info(`[ZipManager] ✓ All ZIPs unmounted`); - } -} - -// Singleton instance -export const zipManager = new ZipManager(); diff --git a/backend/src/routes/activities.ts b/backend/src/routes/activities.ts index 7ba85f4..8472e6a 100644 --- a/backend/src/routes/activities.ts +++ b/backend/src/routes/activities.ts @@ -72,7 +72,14 @@ router.get( } = queryResult.data; // Build filters object - const filters: any = {}; + const filters: { + userId?: number; + username?: string; + action?: string; + resource?: string; + startDate?: string; + endDate?: string; + } = {}; if (userId !== undefined) filters.userId = userId; if (username) filters.username = username; if (action) filters.action = action; @@ -96,6 +103,16 @@ router.get( requirePermission('activities.read'), asyncHandler(async (req, res) => { const timeRange = (req.query.timeRange as '24h' | '7d' | '30d') || DEFAULT_TIME_RANGE; + + // Validate timeRange + const validTimeRanges = ['24h', '7d', '30d'] as const; + if (req.query.timeRange && !validTimeRanges.includes(timeRange)) { + return res.status(400).json({ + success: false, + error: 'Invalid timeRange. Must be 24h, 7d, or 30d', + }); + } + const customRange = req.query.startDate && req.query.endDate ? { @@ -149,6 +166,15 @@ router.get( const limit = Math.min(parseInt(req.query.limit as string) || 10, MAX_ACTIONS_LIMIT); const timeRange = (req.query.timeRange as '24h' | '7d' | '30d') || DEFAULT_TIME_RANGE; + // Validate timeRange + const validTimeRanges = ['24h', '7d', '30d'] as const; + if (req.query.timeRange && !validTimeRanges.includes(timeRange)) { + return res.status(400).json({ + success: false, + error: 'Invalid timeRange. Must be 24h, 7d, or 30d', + }); + } + const result = await activityService.getTopActions(limit, timeRange); res.json({ @@ -179,6 +205,15 @@ router.get( }); } + // Validate timeRange + const validTimeRanges = ['24h', '7d', '30d'] as const; + if (req.query.timeRange && !validTimeRanges.includes(timeRange)) { + return res.status(400).json({ + success: false, + error: 'Invalid timeRange. Must be 24h, 7d, or 30d', + }); + } + const result = await activityService.getBreakdown(groupBy, limit, timeRange); res.json({ diff --git a/backend/src/routes/auth.ts b/backend/src/routes/auth.ts index 2a86eeb..aba4b43 100644 --- a/backend/src/routes/auth.ts +++ b/backend/src/routes/auth.ts @@ -5,8 +5,14 @@ import { ActivityService } from '../services/ActivityService'; import { UserDatabaseService } from '../services/UserDatabaseService'; import { AppError } from '../middleware/errorHandler'; import { asyncHandler } from '../middleware/asyncHandler'; +import { softAuth } from '../middleware/auth'; import { logActivity } from '../middleware/activityLogger'; import { logger } from '../utils/logger'; +import { + setRefreshTokenCookie, + clearRefreshTokenCookie, + getRefreshTokenFromCookie, +} from '../utils/cookies'; import { z } from 'zod'; const router = Router(); @@ -61,6 +67,29 @@ const registerLimiter = rateLimit({ }, }); +// Moderate limits on token refresh +const refreshLimiter = rateLimit({ + windowMs: 60 * 1000, // 1 minute + max: 30, // 30 refresh requests per minute per IP + message: { + error: { + message: 'Too many token refresh attempts from this IP, please try again later', + statusCode: 429, + }, + }, + standardHeaders: true, + legacyHeaders: false, + handler: (req, res) => { + logger.warn(`[Security] Rate limit exceeded for token refresh from IP: ${req.ip}`); + res.status(429).json({ + error: { + message: 'Too many token refresh attempts from this IP, please try again later', + statusCode: 429, + }, + }); + }, +}); + // Validation schemas const loginSchema = z.object({ username: z.string().min(3).max(50), @@ -73,10 +102,6 @@ const registerSchema = z.object({ password: z.string().min(6), }); -const refreshTokenSchema = z.object({ - refreshToken: z.string(), -}); - /** * GET /api/auth/setup-status * Check if the system needs initial setup (no users exist) @@ -104,7 +129,7 @@ router.post( const credentials = loginSchema.parse(req.body); const ipAddress = req.ip || ''; - logger.info(`[Auth] Login attempt for user: ${credentials.username} from ${ipAddress}`); + logger.debug(`[Auth] Login attempt for user: ${credentials.username} from ${ipAddress}`); const result = await authService.login(credentials, ipAddress); @@ -119,16 +144,32 @@ router.post( }); logger.info(`[Auth] Login successful for user: ${credentials.username}`); - res.json(result); + + // Set refresh token as HTTP-only cookie + setRefreshTokenCookie(res, result.tokens.refreshToken); + + res.json({ + user: result.user, + tokens: { + accessToken: result.tokens.accessToken, + expiresIn: result.tokens.expiresIn, + }, + }); } catch (error) { if (error instanceof z.ZodError) { - logger.warn(`[Auth] Validation error during login: ${error.errors[0].message}`); - throw new AppError(400, `Validation error: ${error.errors[0].message}`); + const username = + typeof req.body?.username === 'string' ? req.body.username.substring(0, 50) : 'unknown'; + logger.warn(`[Auth] Validation error during login for user: ${username}`); + throw new AppError( + 400, + `Validation error: ${error.errors.map((e) => e.message).join(', ')}` + ); } // Log failed login attempt const ipAddress = req.ip || ''; - const username = req.body.username || 'unknown'; + const username = + typeof req.body?.username === 'string' ? req.body.username.substring(0, 50) : 'unknown'; try { await activityService.log({ @@ -152,7 +193,7 @@ router.post( } // Log the error with full details before passing to error handler - logger.error(`[Auth] Login failed for user: ${username}`, { + logger.warn(`[Auth] Login failed for user: ${username}`, { error: error instanceof Error ? error.message : String(error), stack: error instanceof Error ? error.stack : undefined, }); @@ -194,14 +235,31 @@ router.post( }); logger.info(`[Auth] Registration successful for user: ${data.username}`); - res.status(201).json(result); + + // Set refresh token as HTTP-only cookie + setRefreshTokenCookie(res, result.tokens.refreshToken); + + res.status(201).json({ + user: result.user, + tokens: { + accessToken: result.tokens.accessToken, + expiresIn: result.tokens.expiresIn, + }, + }); } catch (error) { if (error instanceof z.ZodError) { - logger.warn(`[Auth] Validation error during registration: ${error.errors[0].message}`); - throw new AppError(400, `Validation error: ${error.errors[0].message}`); + const username = + typeof req.body?.username === 'string' ? req.body.username.substring(0, 50) : 'unknown'; + logger.warn(`[Auth] Validation error during registration for user: ${username}`); + throw new AppError( + 400, + `Validation error: ${error.errors.map((e) => e.message).join(', ')}` + ); } - logger.error(`[Auth] Registration failed for user: ${req.body.username}`, { + const username = + typeof req.body?.username === 'string' ? req.body.username.substring(0, 50) : 'unknown'; + logger.error(`[Auth] Registration failed for user: ${username}`, { error: error instanceof Error ? error.message : String(error), stack: error instanceof Error ? error.stack : undefined, }); @@ -217,11 +275,16 @@ router.post( */ router.post( '/logout', + softAuth, logActivity('auth.logout', 'auth'), asyncHandler(async (req, res) => { - const { refreshToken } = refreshTokenSchema.parse(req.body); + const refreshToken = getRefreshTokenFromCookie(req.cookies); - await authService.logout(refreshToken); + if (refreshToken) { + await authService.logout(refreshToken); + } + + clearRefreshTokenCookie(res); res.json({ success: true, message: 'Logged out successfully' }); }) @@ -233,12 +296,23 @@ router.post( */ router.post( '/refresh', + refreshLimiter, asyncHandler(async (req, res) => { - const { refreshToken } = refreshTokenSchema.parse(req.body); + const refreshToken = getRefreshTokenFromCookie(req.cookies); + + if (!refreshToken) { + throw new AppError(401, 'No refresh token provided'); + } const tokens = await authService.refreshToken(refreshToken); - res.json(tokens); + // Set new refresh token cookie (token rotation) + setRefreshTokenCookie(res, tokens.refreshToken); + + res.json({ + accessToken: tokens.accessToken, + expiresIn: tokens.expiresIn, + }); }) ); diff --git a/backend/src/routes/cache.ts b/backend/src/routes/cache.ts index 2cf8910..7ea7441 100644 --- a/backend/src/routes/cache.ts +++ b/backend/src/routes/cache.ts @@ -47,23 +47,34 @@ router.post( asyncHandler(async (req, res) => { const { cacheType } = req.body; + // Validate cacheType + const validCacheTypes = ['gameSearch', 'permissions', 'all']; + if (cacheType && !validCacheTypes.includes(cacheType)) { + return res.status(400).json({ + success: false, + error: 'Invalid cacheType. Must be one of: gameSearch, permissions, all', + }); + } + const results: Record = {}; - if (!cacheType || cacheType === 'gameSearch') { - GameSearchCache.clearCache(); - results.gameSearch = true; - } - - if (!cacheType || cacheType === 'permissions') { - PermissionCache.clearAll(); - results.permissions = true; - } - - if (!cacheType || cacheType === 'all') { - GameSearchCache.clearCache(); - PermissionCache.clearAll(); - results.gameSearch = true; - results.permissions = true; + // Use switch to fix mutually exclusive cache clearing logic + switch (cacheType) { + case 'gameSearch': + GameSearchCache.clearCache(); + results.gameSearch = true; + break; + case 'permissions': + PermissionCache.clearAll(); + results.permissions = true; + break; + default: + // Clear all caches when cacheType is 'all' or undefined + GameSearchCache.clearCache(); + PermissionCache.clearAll(); + results.gameSearch = true; + results.permissions = true; + break; } res.json({ diff --git a/backend/src/routes/community-playlists.ts b/backend/src/routes/community-playlists.ts index 991f897..c5391be 100644 --- a/backend/src/routes/community-playlists.ts +++ b/backend/src/routes/community-playlists.ts @@ -1,5 +1,9 @@ import { Router } from 'express'; -import { CommunityPlaylistService } from '../services/CommunityPlaylistService'; +import { z } from 'zod'; +import { + CommunityPlaylistService, + ALLOWED_DOWNLOAD_DOMAINS, +} from '../services/CommunityPlaylistService'; import { AppError } from '../middleware/errorHandler'; import { asyncHandler } from '../middleware/asyncHandler'; import { optionalAuth } from '../middleware/auth'; @@ -8,6 +12,11 @@ import { logActivity } from '../middleware/activityLogger'; const router = Router(); const communityPlaylistService = new CommunityPlaylistService(); +// Validation schema +const downloadSchema = z.object({ + downloadUrl: z.string().url().max(2000), +}); + // GET /api/community-playlists - Fetch list of community playlists from wiki router.get( '/', @@ -39,10 +48,25 @@ router.post( gameCount: res.locals.gameCount || 0, })), asyncHandler(async (req, res) => { - const { downloadUrl } = req.body; + // Validate request body + const { downloadUrl } = downloadSchema.parse(req.body); - if (!downloadUrl) { - throw new AppError(400, 'downloadUrl is required'); + try { + const parsedUrl = new URL(downloadUrl); + const isAllowed = ALLOWED_DOWNLOAD_DOMAINS.some( + (domain) => parsedUrl.hostname === domain || parsedUrl.hostname.endsWith('.' + domain) + ); + if (!isAllowed) { + throw new AppError(400, 'Download URL domain not allowed'); + } + if (parsedUrl.protocol !== 'https:') { + throw new AppError(400, 'Only HTTPS URLs are allowed'); + } + } catch (error) { + if (error instanceof AppError) { + throw error; + } + throw new AppError(400, 'Invalid URL'); } const result = await communityPlaylistService.downloadPlaylist(downloadUrl); diff --git a/backend/src/routes/database.ts b/backend/src/routes/database.ts index 53db96c..dea1ffb 100644 --- a/backend/src/routes/database.ts +++ b/backend/src/routes/database.ts @@ -1,9 +1,12 @@ import { Router, Request, Response } from 'express'; +import fs from 'fs'; import { DatabaseService } from '../services/DatabaseService'; import { authenticate } from '../middleware/auth'; import { requirePermission } from '../middleware/rbac'; import { logActivity } from '../middleware/activityLogger'; +import { asyncHandler } from '../middleware/asyncHandler'; import { logger } from '../utils/logger'; +import { config } from '../config'; const router = Router(); @@ -28,7 +31,7 @@ router.post( authenticate, requirePermission('settings.update'), logActivity('database.reload', 'database'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { logger.info('Manual database reload requested'); @@ -45,7 +48,7 @@ router.post( error: error instanceof Error ? error.message : 'Unknown error', }); } - } + }) ); /** @@ -68,11 +71,8 @@ router.get( authenticate, requirePermission('settings.update'), logActivity('database.status', 'database'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { - const fs = require('fs'); - const { config } = require('../config'); - const isConnected = DatabaseService.isConnected(); const stats = fs.statSync(config.flashpointDbPath); @@ -91,7 +91,7 @@ router.get( error: error instanceof Error ? error.message : 'Unknown error', }); } - } + }) ); export default router; diff --git a/backend/src/routes/downloads.ts b/backend/src/routes/downloads.ts index fb42f32..e425fab 100644 --- a/backend/src/routes/downloads.ts +++ b/backend/src/routes/downloads.ts @@ -1,4 +1,5 @@ import { Router, Request, Response } from 'express'; +import { z } from 'zod'; import { DownloadManager } from '../services/DownloadManager'; import { PreferencesService } from '../services/PreferencesService'; import { GameDatabaseUpdater } from '../services/GameDatabaseUpdater'; @@ -7,8 +8,16 @@ import { ActivityService } from '../services/ActivityService'; import { authenticate } from '../middleware/auth'; import { requirePermission } from '../middleware/rbac'; import { logActivity } from '../middleware/activityLogger'; +import { asyncHandler } from '../middleware/asyncHandler'; import { logger } from '../utils/logger'; +/** + * Zod schema for POST /:id/download request body + */ +const downloadBodySchema = z.object({ + gameDataId: z.number().int().positive().optional(), +}); + const router = Router(); const gameService = new GameService(); const activityService = new ActivityService(); @@ -40,131 +49,133 @@ router.post( gameTitle: res.locals.gameTitle, sources: res.locals.sources, })), - async (req: Request, res: Response) => { - try { - const gameId = req.params.id; - const { gameDataId: requestedGameDataId } = req.body; + asyncHandler(async (req: Request, res: Response) => { + const gameId = req.params.id; - logger.info('Download request received', { gameId, requestedGameDataId }); - - // Get game from database - const game = await gameService.getGameById(gameId); - if (!game) { - return res.status(404).json({ - success: false, - error: 'Game not found', - }); - } - - // Determine which game data to download - let gameDataId = requestedGameDataId; - if (!gameDataId) { - // Query game_data table to find the game data entry for this game - const foundGameDataId = await gameService.getGameDataId(gameId); - if (!foundGameDataId) { - return res.status(404).json({ - success: false, - error: 'Game has no game data to download', - }); - } - gameDataId = foundGameDataId; - } - - // Check if already downloaded - const isDownloaded = await GameDatabaseUpdater.isDownloaded(gameDataId); - if (isDownloaded) { - return res.status(409).json({ - success: false, - error: 'Game data already downloaded', - }); - } - - // Get game data info - const gameData = await GameDatabaseUpdater.getGameData(gameDataId); - if (!gameData) { - return res.status(404).json({ - success: false, - error: 'Game data not found', - }); - } - - // Get data sources from preferences - const sources = await PreferencesService.getGameDataSources(); - if (sources.length === 0) { - return res.status(500).json({ - success: false, - error: - 'No game data sources configured. Please configure gameDataSources in preferences.json', - }); - } - - // Store metadata for activity logging - res.locals.gameDataId = gameDataId; - res.locals.gameTitle = game.title; - res.locals.sources = sources.map((s) => s.name); - - // Capture metadata for background error logging - const downloadMetadata = { - gameId, - gameDataId, - gameTitle: game.title, - sha256: gameData.sha256, - sources: sources.map((s) => s.name), - userId: req.user?.id, - username: req.user?.username, - ipAddress: req.ip, - }; - - // Start download (non-blocking) - // We don't wait for it to complete - client uses SSE endpoint for progress - DownloadManager.downloadGameData( - gameDataId, - sources, - undefined, // Progress callback handled by SSE endpoint - undefined, // Details callback handled by SSE endpoint - undefined // No abort signal for now - ).catch(async (error) => { - const errorMessage = error instanceof Error ? error.message : 'Unknown error'; - logger.error('Background download failed', { - gameId, - gameDataId, - error: errorMessage, - }); - - // Log download failure activity - await activityService.log({ - userId: downloadMetadata.userId, - username: downloadMetadata.username, - action: 'games.download.failed', - resource: 'games', - resourceId: downloadMetadata.gameId, - details: { - gameDataId: downloadMetadata.gameDataId, - gameTitle: downloadMetadata.gameTitle, - sha256: downloadMetadata.sha256, - sources: downloadMetadata.sources, - error: errorMessage, - }, - ipAddress: downloadMetadata.ipAddress, - userAgent: req.get('user-agent'), - }); - }); - - // Return success immediately - res.json({ - success: true, - message: 'Download started', - gameDataId, - sha256: gameData.sha256, - }); - } catch (error) { - logger.error('Download endpoint error:', error); - res.status(500).json({ + // Validate request body + const bodyResult = downloadBodySchema.safeParse(req.body); + if (!bodyResult.success) { + return res.status(400).json({ success: false, - error: error instanceof Error ? error.message : 'Unknown error', + error: `Invalid request body: ${bodyResult.error.issues[0].message}`, }); } - } + + const { gameDataId: requestedGameDataId } = bodyResult.data; + + logger.info('Download request received', { gameId, requestedGameDataId }); + + // Get game from database + const game = await gameService.getGameById(gameId); + if (!game) { + return res.status(404).json({ + success: false, + error: 'Game not found', + }); + } + + // Determine which game data to download + let gameDataId = requestedGameDataId; + if (!gameDataId) { + // Query game_data table to find the game data entry for this game + const foundGameDataId = await gameService.getGameDataId(gameId); + if (!foundGameDataId) { + return res.status(404).json({ + success: false, + error: 'Game has no game data to download', + }); + } + gameDataId = foundGameDataId; + } + + // Check if already downloaded + const isDownloaded = await GameDatabaseUpdater.isDownloaded(gameDataId); + if (isDownloaded) { + return res.status(409).json({ + success: false, + error: 'Game data already downloaded', + }); + } + + // Get game data info + const gameData = await GameDatabaseUpdater.getGameData(gameDataId); + if (!gameData) { + return res.status(404).json({ + success: false, + error: 'Game data not found', + }); + } + + // Get data sources from preferences + const sources = await PreferencesService.getGameDataSources(); + if (sources.length === 0) { + return res.status(500).json({ + success: false, + error: + 'No game data sources configured. Please configure gameDataSources in preferences.json', + }); + } + + // Store metadata for activity logging + res.locals.gameDataId = gameDataId; + res.locals.gameTitle = game.title; + res.locals.sources = sources.map((s) => s.name); + + // Capture metadata for background error logging + const downloadMetadata = { + gameId, + gameDataId, + gameTitle: game.title, + sha256: gameData.sha256, + sources: sources.map((s) => s.name), + userId: req.user?.id, + username: req.user?.username, + ipAddress: req.ip, + }; + + // Start download (non-blocking) + // We don't wait for it to complete - client uses SSE endpoint for progress + DownloadManager.downloadGameData( + gameDataId, + sources, + undefined, // Progress callback handled by SSE endpoint + undefined, // Details callback handled by SSE endpoint + undefined // No abort signal for now + ).catch(async (error) => { + const errorMessage = error instanceof Error ? error.message : 'Unknown error'; + logger.error('Background download failed', { + gameId, + gameDataId, + error: errorMessage, + }); + + // Log download failure activity + await activityService.log({ + userId: downloadMetadata.userId, + username: downloadMetadata.username, + action: 'games.download.failed', + resource: 'games', + resourceId: downloadMetadata.gameId, + details: { + gameDataId: downloadMetadata.gameDataId, + gameTitle: downloadMetadata.gameTitle, + sha256: downloadMetadata.sha256, + sources: downloadMetadata.sources, + error: errorMessage, + }, + ipAddress: downloadMetadata.ipAddress, + userAgent: req.get('user-agent'), + }); + }); + + // Return success immediately + res.json({ + success: true, + message: 'Download started', + gameDataId, + sha256: gameData.sha256, + }); + }) ); /** @@ -188,65 +199,73 @@ router.get( '/:id/download/progress', authenticate, requirePermission('games.download'), - async (req: Request, res: Response) => { - try { - const gameId = req.params.id; + asyncHandler(async (req: Request, res: Response) => { + const gameId = req.params.id; - // Set SSE headers - res.setHeader('Content-Type', 'text/event-stream'); - res.setHeader('Cache-Control', 'no-cache'); - res.setHeader('Connection', 'keep-alive'); - res.setHeader('X-Accel-Buffering', 'no'); // Disable nginx buffering + // Set SSE headers + res.setHeader('Content-Type', 'text/event-stream'); + res.setHeader('Cache-Control', 'no-cache'); + res.setHeader('Connection', 'keep-alive'); + res.setHeader('X-Accel-Buffering', 'no'); // Disable nginx buffering - logger.info('SSE connection established', { gameId }); + logger.info('SSE connection established', { gameId }); - // Get game to find gameDataId - const game = await gameService.getGameById(gameId); - if (!game) { - res.write( - `data: ${JSON.stringify({ - status: 'error', - error: 'Game not found', - })}\n\n` - ); - res.end(); - return; - } - - const gameDataId = await gameService.getGameDataId(gameId); - if (!gameDataId) { - res.write( - `data: ${JSON.stringify({ - status: 'error', - error: 'Game has no game data', - })}\n\n` - ); - res.end(); - return; - } - - // Send initial status + // Get game to find gameDataId + const game = await gameService.getGameById(gameId); + if (!game) { res.write( `data: ${JSON.stringify({ - percent: 0, - status: 'waiting', - details: 'Preparing download...', + status: 'error', + error: 'Game not found', })}\n\n` ); + res.end(); + return; + } - // Poll for download progress - // Note: This is a simplified implementation. In production, you'd want - // a proper event emitter system to push updates rather than polling. - const pollInterval = setInterval(async () => { - try { - // Check if download is still active - const isActive = DownloadManager.isDownloadActive(gameDataId); + const gameDataId = await gameService.getGameDataId(gameId); + if (!gameDataId) { + res.write( + `data: ${JSON.stringify({ + status: 'error', + error: 'Game has no game data', + })}\n\n` + ); + res.end(); + return; + } - if (!isActive) { - // Check if download completed successfully - const isDownloaded = await GameDatabaseUpdater.isDownloaded(gameDataId); + // Send initial status + res.write( + `data: ${JSON.stringify({ + percent: 0, + status: 'waiting', + details: 'Preparing download...', + })}\n\n` + ); - if (isDownloaded) { + // Poll for download progress + // Note: This is a simplified implementation. In production, you'd want + // a proper event emitter system to push updates rather than polling. + let closed = false; + + const pollInterval = setInterval(async () => { + if (closed) return; + + try { + // Check if download is still active + const isActive = DownloadManager.isDownloadActive(gameDataId); + + if (closed) return; // Re-check after await + + if (!isActive) { + // Check if download completed successfully + const isDownloaded = await GameDatabaseUpdater.isDownloaded(gameDataId); + + if (closed) return; // Re-check after await + + if (isDownloaded) { + if (!closed) { res.write( `data: ${JSON.stringify({ percent: 100, @@ -254,24 +273,29 @@ router.get( details: 'Download completed successfully', })}\n\n` ); + } - // Log download completion activity - const gameData = await GameDatabaseUpdater.getGameData(gameDataId); - await activityService.log({ - userId: req.user?.id, - username: req.user?.username, - action: 'games.download.complete', - resource: 'games', - resourceId: gameId, - details: { - gameDataId, - gameTitle: game.title, - sha256: gameData?.sha256, - }, - ipAddress: req.ip, - userAgent: req.get('user-agent'), - }); - } else { + // Log download completion activity + const gameData = await GameDatabaseUpdater.getGameData(gameDataId); + + if (closed) return; // Re-check after await + + await activityService.log({ + userId: req.user?.id, + username: req.user?.username, + action: 'games.download.complete', + resource: 'games', + resourceId: gameId, + details: { + gameDataId, + gameTitle: game.title, + sha256: gameData?.sha256, + }, + ipAddress: req.ip, + userAgent: req.get('user-agent'), + }); + } else { + if (!closed) { res.write( `data: ${JSON.stringify({ percent: 0, @@ -280,11 +304,15 @@ router.get( })}\n\n` ); } + } - clearInterval(pollInterval); + clearInterval(pollInterval); + if (!closed) { res.end(); - } else { - // Send heartbeat + } + } else { + // Send heartbeat + if (!closed) { res.write( `data: ${JSON.stringify({ percent: 0, @@ -293,27 +321,25 @@ router.get( })}\n\n` ); } - } catch (error) { + } + } catch (error) { + if (!closed) { logger.error('Error in SSE poll', { error }); - clearInterval(pollInterval); + } + clearInterval(pollInterval); + if (!closed) { res.end(); } - }, 1000); // Poll every second + } + }, 1000); // Poll every second - // Clean up on client disconnect - req.on('close', () => { - logger.info('SSE connection closed', { gameId }); - clearInterval(pollInterval); - res.end(); - }); - } catch (error) { - logger.error('SSE endpoint error:', error); - res.status(500).json({ - success: false, - error: error instanceof Error ? error.message : 'Unknown error', - }); - } - } + // Clean up on client disconnect + req.on('close', () => { + closed = true; + logger.info('SSE connection closed', { gameId }); + clearInterval(pollInterval); + }); + }) ); /** @@ -334,52 +360,44 @@ router.delete( authenticate, requirePermission('games.download'), logActivity('games.download_cancel', 'games'), - async (req: Request, res: Response) => { - try { - const gameId = req.params.id; + asyncHandler(async (req: Request, res: Response) => { + const gameId = req.params.id; - logger.info('Download cancel request', { gameId }); + logger.info('Download cancel request', { gameId }); - // Get game to find gameDataId - const game = await gameService.getGameById(gameId); - if (!game) { - return res.status(404).json({ - success: false, - error: 'Game not found', - }); - } - - const gameDataId = await gameService.getGameDataId(gameId); - if (!gameDataId) { - return res.status(404).json({ - success: false, - error: 'Game has no game data', - }); - } - - // Attempt to cancel download - const cancelled = DownloadManager.cancelDownload(gameDataId); - - if (!cancelled) { - return res.status(404).json({ - success: false, - error: 'No active download found for this game', - }); - } - - res.json({ - success: true, - cancelled: true, - message: 'Download cancelled successfully', - }); - } catch (error) { - logger.error('Cancel download error:', error); - res.status(500).json({ + // Get game to find gameDataId + const game = await gameService.getGameById(gameId); + if (!game) { + return res.status(404).json({ success: false, - error: error instanceof Error ? error.message : 'Unknown error', + error: 'Game not found', }); } - } + + const gameDataId = await gameService.getGameDataId(gameId); + if (!gameDataId) { + return res.status(404).json({ + success: false, + error: 'Game has no game data', + }); + } + + // Attempt to cancel download + const cancelled = DownloadManager.cancelDownload(gameDataId); + + if (!cancelled) { + return res.status(404).json({ + success: false, + error: 'No active download found for this game', + }); + } + + res.json({ + success: true, + cancelled: true, + message: 'Download cancelled successfully', + }); + }) ); export default router; diff --git a/backend/src/routes/errors.ts b/backend/src/routes/errors.ts index bd7d5b6..d796e28 100644 --- a/backend/src/routes/errors.ts +++ b/backend/src/routes/errors.ts @@ -3,11 +3,28 @@ import { promises as fs } from 'fs'; import fsSync from 'fs'; import path from 'path'; import winston from 'winston'; +import { z } from 'zod'; import { authenticate } from '../middleware/auth'; import { requirePermission } from '../middleware/rbac'; +import { rateLimitStrict } from '../middleware/rateLimiter'; +import { asyncHandler } from '../middleware/asyncHandler'; import { logger } from '../utils/logger'; import { config } from '../config'; +/** + * Zod schema for error report validation + * Enforces strict types and size limits on client-submitted error reports + */ +const errorReportSchema = z.object({ + type: z.enum(['client_error', 'network_error', 'api_error', 'route_error']), + message: z.string().max(2000), + stack: z.string().max(5000).optional(), + url: z.string().max(2000), + timestamp: z.string().optional(), + userAgent: z.string().optional(), + context: z.record(z.string(), z.unknown()).optional(), +}); + const router = Router(); // Client error logs go to the same directory as backend logs @@ -75,17 +92,6 @@ interface ErrorReport { type AuthenticatedRequest = Request & { user?: { id: number } }; -/** - * Ensure logs directory exists - */ -async function ensureLogDir(): Promise { - try { - await fs.mkdir(LOG_DIR, { recursive: true }); - } catch (error) { - logger.error('Failed to create logs directory:', error); - } -} - /** * Log error report to file in OTEL-compatible JSON format */ @@ -110,8 +116,6 @@ async function logError(report: ErrorReport): Promise { */ async function getRecentErrors(limit: number = 100): Promise { try { - await ensureLogDir(); - try { const content = await fs.readFile(ERROR_LOG_FILE, 'utf8'); const lines = content.trim().split('\n'); @@ -164,54 +168,60 @@ async function getRecentErrors(limit: number = 100): Promise { * Report a client-side error * Public endpoint (no authentication required, but rate-limited) */ -router.post('/report', async (req: Request, res: Response) => { - try { - const report: ErrorReport = req.body; +router.post( + '/report', + rateLimitStrict, + asyncHandler(async (req: Request, res: Response) => { + try { + // Validate request body with Zod schema + const parseResult = errorReportSchema.safeParse(req.body); + if (!parseResult.success) { + return res.status(400).json({ + success: false, + error: { + code: 'VALIDATION_ERROR', + message: `Invalid error report: ${parseResult.error.issues[0].message}`, + }, + }); + } - // Validate required fields - if (!report.type || !report.message || !report.url) { - return res.status(400).json({ + const validatedData = parseResult.data; + + // Build report from validated data + const report: ErrorReport = { + type: validatedData.type, + message: validatedData.message, + stack: validatedData.stack, + url: validatedData.url, + timestamp: validatedData.timestamp || new Date().toISOString(), + userAgent: validatedData.userAgent || req.headers['user-agent'] || 'Unknown', + context: validatedData.context, + }; + + // Add user ID if authenticated + const authReq = req as AuthenticatedRequest; + if (authReq.user) { + report.userId = authReq.user.id; + } + + // Log to file + await logError(report); + + logger.info(`[Client Error] ${report.type} - ${report.message} (${report.url})`); + + res.json({ success: true }); + } catch (error) { + logger.error('Error in /errors/report:', error); + res.status(500).json({ success: false, error: { - code: 'VALIDATION_ERROR', - message: 'Missing required fields: type, message, url', + code: 'INTERNAL_ERROR', + message: 'Failed to log error report', }, }); } - - // Add user ID if authenticated - const authReq = req as AuthenticatedRequest; - if (authReq.user) { - report.userId = authReq.user.id; - } - - // Add timestamp if not provided - if (!report.timestamp) { - report.timestamp = new Date().toISOString(); - } - - // Add user agent if not provided - if (!report.userAgent) { - report.userAgent = req.headers['user-agent'] || 'Unknown'; - } - - // Log to file - await logError(report); - - logger.info(`[Client Error] ${report.type} - ${report.message} (${report.url})`); - - res.json({ success: true }); - } catch (error) { - logger.error('Error in /errors/report:', error); - res.status(500).json({ - success: false, - error: { - code: 'INTERNAL_ERROR', - message: 'Failed to log error report', - }, - }); - } -}); + }) +); /** * GET /api/errors/recent @@ -222,9 +232,9 @@ router.get( '/recent', authenticate, requirePermission('settings.update'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { - const limit = parseInt(req.query.limit as string) || 100; + const limit = Math.min(parseInt(req.query.limit as string) || 100, 1000); const errors = await getRecentErrors(limit); res.json({ @@ -245,7 +255,7 @@ router.get( }, }); } - } + }) ); /** @@ -257,7 +267,7 @@ router.get( '/stats', authenticate, requirePermission('settings.update'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { const errors = await getRecentErrors(1000); @@ -310,7 +320,7 @@ router.get( }, }); } - } + }) ); export default router; diff --git a/backend/src/routes/favorites.ts b/backend/src/routes/favorites.ts index f59baa1..dd85b57 100644 --- a/backend/src/routes/favorites.ts +++ b/backend/src/routes/favorites.ts @@ -24,11 +24,11 @@ const toggleFavoriteSchema = z.object({ }); const batchAddSchema = z.object({ - gameIds: z.array(z.string()).min(1), + gameIds: z.array(z.string()).min(1).max(500), }); const batchRemoveSchema = z.object({ - gameIds: z.array(z.string()).min(1), + gameIds: z.array(z.string()).min(1).max(500), }); /** @@ -44,8 +44,12 @@ router.get( throw new AppError(401, 'Authentication required'); } - const limit = req.query.limit ? parseInt(req.query.limit as string, 10) : undefined; - const offset = req.query.offset ? parseInt(req.query.offset as string, 10) : undefined; + const limit = req.query.limit + ? Math.min(parseInt(req.query.limit as string, 10) || 50, 100) + : undefined; + const offset = req.query.offset + ? Math.max(0, parseInt(req.query.offset as string, 10) || 0) + : undefined; const favorites = favoritesService.getUserFavorites(req.user.id, limit, offset); res.json(favorites); @@ -83,10 +87,22 @@ router.get( throw new AppError(401, 'Authentication required'); } - const limit = req.query.limit ? parseInt(req.query.limit as string, 10) : undefined; - const offset = req.query.offset ? parseInt(req.query.offset as string, 10) : undefined; - const sortBy = (req.query.sortBy as 'title' | 'dateAdded') || 'dateAdded'; - const sortOrder = (req.query.sortOrder as 'asc' | 'desc') || 'desc'; + const limit = req.query.limit + ? Math.min(parseInt(req.query.limit as string, 10) || 50, 100) + : undefined; + const offset = req.query.offset + ? Math.max(0, parseInt(req.query.offset as string, 10) || 0) + : undefined; + + // Validate sortBy + const allowedSortBy = ['title', 'dateAdded']; + const sortBy = + req.query.sortBy && allowedSortBy.includes(req.query.sortBy as string) + ? (req.query.sortBy as 'title' | 'dateAdded') + : 'dateAdded'; + + // Validate sortOrder + const sortOrder = req.query.sortOrder === 'asc' ? 'asc' : 'desc'; const games = await favoritesService.getUserFavoriteGames( req.user.id, diff --git a/backend/src/routes/game-files.ts b/backend/src/routes/game-files.ts deleted file mode 100644 index 0a9eea9..0000000 --- a/backend/src/routes/game-files.ts +++ /dev/null @@ -1,55 +0,0 @@ -import { Router, Request, Response } from 'express'; -import { GameFileService } from '../services/GameFileService'; -import { AppError } from '../middleware/errorHandler'; -import { asyncHandler } from '../middleware/asyncHandler'; -import { logger } from '../utils/logger'; - -const router = Router(); -const gameFileService = new GameFileService(); - -/** - * GET /game-files/* - Proxy game file requests with automatic fallback - * - * Examples: - * - /game-files/www.example.com/path/file.swf - * - /game-files/cdn.games.com/assets/game.swf - */ -router.get( - '/*', - asyncHandler(async (req: Request, res: Response) => { - try { - // Extract path after /game-files/ - const urlPath = req.path.substring(1); // Remove leading slash - - if (!urlPath) { - throw new AppError(400, 'File path is required'); - } - - logger.info(`[GameFilesRoute] Request: ${urlPath}`); - - // Get the file (tries Game Server first, then external sources) - const result = await gameFileService.getGameFile(urlPath); - - logger.info(`[GameFilesRoute] Serving from ${result.source}: ${result.data.length} bytes`); - - // Set headers - res.setHeader('Content-Type', result.contentType); - res.setHeader('Content-Length', result.data.length); - res.setHeader('Cache-Control', 'public, max-age=86400'); // Cache for 24 hours - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('X-Source', result.source); // Debug header - - // Send file - res.send(result.data); - } catch (error) { - if (error instanceof Error && error.message.includes('not found')) { - throw new AppError(404, 'Game file not found in any source'); - } else { - logger.error('[GameFilesRoute] Error:', error); - throw error; - } - } - }) -); - -export default router; diff --git a/backend/src/routes/game-proxy.ts b/backend/src/routes/game-proxy.ts new file mode 100644 index 0000000..9756ffb --- /dev/null +++ b/backend/src/routes/game-proxy.ts @@ -0,0 +1,46 @@ +import { Router, Request, Response } from 'express'; +import { IncomingMessage, ServerResponse } from 'http'; +import { ProxyRequestHandler } from '../game/proxy-request-handler'; +import { logger } from '../utils/logger'; +import { asyncHandler } from '../middleware/asyncHandler'; + +const router = Router(); + +let proxyHandler: ProxyRequestHandler | null = null; + +function getHandler(): ProxyRequestHandler { + if (!proxyHandler) { + proxyHandler = new ProxyRequestHandler(); + } + return proxyHandler; +} + +// Handle all game content requests: /game-proxy/* +// Express strips the mount path, so req.url is everything after /game-proxy +// e.g., /game-proxy/http://domain.com/file.swf → req.url = /http://domain.com/file.swf +router.all( + '/*', + asyncHandler(async (req: Request, res: Response) => { + const handler = getHandler(); + + // OPTIONS requests for CORS preflight + if (req.method === 'OPTIONS') { + // Cast Express types to Node HTTP types for compatibility + // Double cast needed: Express.Request doesn't directly extend IncomingMessage + handler.handleOptionsRequest( + req as unknown as IncomingMessage, + res as unknown as ServerResponse + ); + return; + } + + // Delegate to ProxyRequestHandler + // Double cast needed: Express.Request doesn't directly extend IncomingMessage + await handler.handleRequest( + req as unknown as IncomingMessage, + res as unknown as ServerResponse + ); + }) +); + +export default router; diff --git a/backend/src/routes/game-zip.ts b/backend/src/routes/game-zip.ts new file mode 100644 index 0000000..c57d97f --- /dev/null +++ b/backend/src/routes/game-zip.ts @@ -0,0 +1,108 @@ +import { Router, Request, Response, json, NextFunction } from 'express'; +import { IncomingMessage, ServerResponse } from 'http'; +import { z } from 'zod'; +import { gameZipServer } from '../game/gamezipserver'; +import { logger } from '../utils/logger'; +import { authenticate } from '../middleware/auth'; +import { requirePermission } from '../middleware/rbac'; +import { config } from '../config'; +import { asyncHandler } from '../middleware/asyncHandler'; + +/** + * Zod schema for POST /mount/:id request body + */ +const mountBodySchema = z.object({ + zipPath: z.string().min(1).max(500), + gameId: z.string().optional(), + dateAdded: z.string().optional(), + sha256: z.string().optional(), +}); + +const router = Router(); + +// Restrictive CORS middleware for admin routes +const adminCors = (req: Request, res: Response, next: NextFunction) => { + res.setHeader('Access-Control-Allow-Origin', config.domain); + res.setHeader('Access-Control-Allow-Methods', 'GET, POST, DELETE, OPTIONS'); + res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization'); + res.setHeader('Access-Control-Allow-Credentials', 'true'); + if (req.method === 'OPTIONS') { + res.status(204).end(); + return; + } + next(); +}; + +// Mount a ZIP: POST /game-zip/mount/:id +router.post( + '/mount/:id', + adminCors, + authenticate, + requirePermission('settings.update'), + json(), + asyncHandler(async (req: Request, res: Response) => { + // Validate request body + const bodyResult = mountBodySchema.safeParse(req.body); + if (!bodyResult.success) { + return res.status(400).json({ + success: false, + error: `Invalid request body: ${bodyResult.error.issues[0].message}`, + }); + } + + const { zipPath, gameId, dateAdded, sha256 } = bodyResult.data; + + const result = await gameZipServer.mountZip({ + id: req.params.id, + zipPath, + gameId, + dateAdded, + sha256, + }); + res.status(result.statusCode).json(result); + }) +); + +// Unmount a ZIP: DELETE /game-zip/mount/:id +router.delete( + '/mount/:id', + adminCors, + authenticate, + requirePermission('settings.update'), + asyncHandler(async (req: Request, res: Response) => { + const success = await gameZipServer.unmountZip(req.params.id); + res.status(success ? 200 : 404).json({ success, id: req.params.id }); + }) +); + +// List mounted ZIPs: GET /game-zip/mounts +router.get( + '/mounts', + adminCors, + authenticate, + requirePermission('settings.update'), + (_req: Request, res: Response) => { + res.json({ mounts: gameZipServer.listMounts() }); + } +); + +// Serve files from mounted ZIPs: GET /game-zip/* +// Handles proxy-style URLs like /game-zip/http://domain.com/path +router.get('/*', async (req: Request, res: Response) => { + try { + // Delegate to gameZipServer's file handler + await gameZipServer.handleFileRequest( + req as unknown as IncomingMessage, + res as unknown as ServerResponse + ); + } catch (error) { + logger.error('[GameZip] File request error:', error); + if (!res.headersSent) { + res.status(500).send('Internal Server Error'); + } else if (!res.writableEnded) { + res.end(); + } + } +}); + +export default router; diff --git a/backend/src/routes/games.ts b/backend/src/routes/games.ts index 2d79044..e8cf3c2 100644 --- a/backend/src/routes/games.ts +++ b/backend/src/routes/games.ts @@ -7,7 +7,7 @@ import { asyncHandler } from '../middleware/asyncHandler'; import { sharedAccessAuth, validateSharedGameAccess } from '../middleware/auth'; import { logActivity } from '../middleware/activityLogger'; import { rateLimitStandard } from '../middleware/rateLimiter'; -import { config } from '../config'; +import { logger } from '../utils/logger'; import { z } from 'zod'; const router = Router(); @@ -101,6 +101,7 @@ router.get( limit: query.limit, showBroken: query.showBroken, showExtreme: query.showExtreme, + fields: 'list', // Only return essential columns for browse/search views }); // Store result count for activity logging @@ -123,7 +124,7 @@ router.get( router.get( '/most-played', asyncHandler(async (req, res) => { - const limit = Math.min(parseInt(req.query.limit as string) || 20, 50); + const limit = Math.max(1, Math.min(parseInt(req.query.limit as string) || 20, 50)); const games = await gameService.getMostPlayedGames(limit); res.json({ @@ -134,6 +135,23 @@ router.get( }) ); +// GET /api/games/random - Get random game +// IMPORTANT: Must be defined BEFORE /:id to avoid being shadowed by the wildcard +router.get( + '/random', + asyncHandler(async (req, res) => { + const librarySchema = z.enum(['arcade', 'theatre']).optional(); + const library = librarySchema.parse(req.query.library); + const game = await gameService.getRandomGame(library); + + if (!game) { + throw new AppError(404, 'No games found'); + } + + res.json(game); + }) +); + // GET /api/games/:id - Get single game by ID router.get( '/:id', @@ -163,28 +181,13 @@ router.get( '/:id/related', validateSharedGameAccess('id'), asyncHandler(async (req, res) => { - const limit = Math.min(parseInt(req.query.limit as string) || 10, 50); + const limit = Math.max(1, Math.min(parseInt(req.query.limit as string) || 10, 50)); const relatedGames = await gameService.getRelatedGames(req.params.id, limit); res.json(relatedGames); }) ); -// GET /api/games/random - Get random game -router.get( - '/random', - asyncHandler(async (req, res) => { - const library = req.query.library as string | undefined; - const game = await gameService.getRandomGame(library); - - if (!game) { - throw new AppError(404, 'No games found'); - } - - res.json(game); - }) -); - // GET /api/games/:id/launch - Get game launch data router.get( '/:id/launch', @@ -201,18 +204,16 @@ router.get( throw new AppError(404, 'Game not found'); } - // Auto-mount ZIP file if game has game data - // This ensures files are available from the GameZip server - // Check if game has data by checking presentOnDisk field (not null means has game_data entries) - if (game.presentOnDisk !== null) { - await gameDataService.mountGameZip(game.id); - } + // Mount the game ZIP (awaited so local ZIPs are ready before we return the URL). + // For downloads, mountGameZip returns quickly with { mounted: false, downloading: true }. + const mountResult = await gameDataService.mountGameZip(game.id); + const downloading = mountResult.downloading || false; // Extract launch command (usually the SWF file path for Flash games) let launchCommand = game.launchCommand || ''; - // If no launch command but has game data, query game_data table - if (!launchCommand && game.presentOnDisk !== null) { + // If no launch command, try to get path from game_data table + if (!launchCommand) { const gameDataPath = await gameService.getGameDataPath(game.id); if (gameDataPath) { launchCommand = gameDataPath; @@ -225,15 +226,12 @@ router.get( let contentUrl = ''; if (launchCommand) { - // Use the configured external game service URL - // In Docker: This is /game-proxy (routed through nginx) - // In local dev: This is http://localhost:22500 (direct access) - const proxyUrl = config.gameServiceExternalUrl; + // Game content is served via /game-proxy route (integrated in backend) + const proxyUrl = '/game-proxy'; // Check if launch command is an absolute URL if (launchCommand.startsWith('http://') || launchCommand.startsWith('https://')) { // Use the full URL as-is with the proxy - // The HTTP proxy server handles proxy-style requests: GET http://domain.com/path HTTP/1.1 contentUrl = `${proxyUrl}/${launchCommand}`; } else { // For relative paths, construct full URL with domain from source @@ -246,8 +244,8 @@ router.get( const domain = sourceUrl.hostname; // Construct full URL: http://domain/path - const path = launchCommand.startsWith('/') ? launchCommand : `/${launchCommand}`; - fullUrl = `http://${domain}${path}`; + const urlPath = launchCommand.startsWith('/') ? launchCommand : `/${launchCommand}`; + fullUrl = `http://${domain}${urlPath}`; } catch { // If source is not a valid URL, assume launchCommand is already domain/path fullUrl = launchCommand.startsWith('http') ? launchCommand : `http://${launchCommand}`; @@ -280,6 +278,7 @@ router.get( applicationPath: game.applicationPath, playMode: game.playMode, canPlayInBrowser, + downloading, }); }) ); diff --git a/backend/src/routes/github.ts b/backend/src/routes/github.ts index 6293b86..5d16d1b 100644 --- a/backend/src/routes/github.ts +++ b/backend/src/routes/github.ts @@ -1,5 +1,6 @@ import { Router, Request, Response } from 'express'; -import { logger } from '../utils/logger.js'; +import { logger } from '../utils/logger'; +import { asyncHandler } from '../middleware/asyncHandler'; const router = Router(); @@ -8,8 +9,9 @@ const router = Router(); * Proxy endpoint for fetching GitHub star count * Avoids CORS issues when calling GitHub API from browser */ -router.get('/stars', async (_req: Request, res: Response) => { - try { +router.get( + '/stars', + asyncHandler(async (_req: Request, res: Response) => { const repoUrl = 'https://api.github.com/repos/darkraise/flashpoint-web'; const response = await fetch(repoUrl, { @@ -31,15 +33,7 @@ router.get('/stars', async (_req: Request, res: Response) => { stars: data.stargazers_count, }, }); - } catch (error) { - logger.error('Failed to fetch GitHub stars:', error); - res.status(500).json({ - success: false, - error: { - message: 'Failed to fetch GitHub star count', - }, - }); - } -}); + }) +); export default router; diff --git a/backend/src/routes/health.ts b/backend/src/routes/health.ts index 96967e7..96f510e 100644 --- a/backend/src/routes/health.ts +++ b/backend/src/routes/health.ts @@ -13,6 +13,7 @@ const router = Router(); * GET /api/health * Basic health check - publicly accessible * Returns minimal status information + * NOTE: A simpler /health endpoint also exists for load balancer checks (registered before middleware) */ router.get('/', (req, res) => { const isHealthy = DatabaseService.isConnected() && UserDatabaseService.isConnected(); @@ -138,8 +139,8 @@ router.get( }, services: { gameService: { - proxyUrl: config.gameServerUrl || 'http://localhost:22500', - gameZipPort: config.gameServerHttpPort || 22501, + status: 'integrated', + routes: ['/game-proxy/*', '/game-zip/*'], }, }, healthCheckDuration: `${healthCheckTime}ms`, diff --git a/backend/src/routes/index.ts b/backend/src/routes/index.ts index e63a8b7..9b6cbb6 100644 --- a/backend/src/routes/index.ts +++ b/backend/src/routes/index.ts @@ -5,7 +5,6 @@ import tagsRouter from './tags'; import playlistsRouter from './playlists'; import communityPlaylistsRouter from './community-playlists'; import proxyRouter from './proxy'; -import gameFilesRouter from './game-files'; import updatesRouter from './updates'; import statisticsRouter from './statistics'; import downloadsRouter from './downloads'; @@ -85,9 +84,6 @@ export function setupRoutes(app: Express): void { app.use('/api/statistics', statisticsRouter); app.use('/api/database', databaseRouter); - // Game files proxy with automatic fallback - app.use('/game-files', gameFilesRouter); - app.use('/proxy', proxyRouter); // 404 handler diff --git a/backend/src/routes/jobs.ts b/backend/src/routes/jobs.ts index cee2eff..c826a66 100644 --- a/backend/src/routes/jobs.ts +++ b/backend/src/routes/jobs.ts @@ -2,6 +2,7 @@ import { Router, Request, Response } from 'express'; import { authenticate } from '../middleware/auth'; import { requirePermission } from '../middleware/rbac'; import { logActivity } from '../middleware/activityLogger'; +import { asyncHandler } from '../middleware/asyncHandler'; import { JobScheduler } from '../services/JobScheduler'; import { JobExecutionService } from '../services/JobExecutionService'; import { CachedSystemSettingsService } from '../services/CachedSystemSettingsService'; @@ -18,7 +19,7 @@ router.get( '/', authenticate, requirePermission('settings.update'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { const jobs = JobScheduler.getAllJobsEnriched(); res.json(jobs); @@ -26,7 +27,28 @@ router.get( logger.error('Failed to get jobs:', error); res.status(500).json({ error: { message: 'Failed to retrieve jobs' } }); } - } + }) +); + +// =================================== +// GET ALL EXECUTION LOGS +// =================================== +router.get( + '/logs/all', + authenticate, + requirePermission('settings.update'), + asyncHandler(async (req: Request, res: Response) => { + try { + const limit = Math.min(parseInt(req.query.limit as string) || 100, 200); + const offset = Math.max(0, parseInt(req.query.offset as string, 10) || 0); + + const result = executionService.getAllLogs(limit, offset); + res.json({ ...result, limit, offset }); + } catch (error) { + logger.error('Failed to get all logs:', error); + res.status(500).json({ error: { message: 'Failed to retrieve logs' } }); + } + }) ); // =================================== @@ -36,7 +58,7 @@ router.get( '/:jobId', authenticate, requirePermission('settings.update'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { const { jobId } = req.params; const job = JobScheduler.getJobStatusEnriched(jobId); @@ -50,7 +72,7 @@ router.get( logger.error(`Failed to get job ${req.params.jobId}:`, error); res.status(500).json({ error: { message: 'Failed to retrieve job' } }); } - } + }) ); // =================================== @@ -61,7 +83,7 @@ router.patch( authenticate, requirePermission('settings.update'), logActivity('jobs.update', 'job'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { const { jobId } = req.params; const { enabled } = req.body; @@ -103,7 +125,7 @@ router.patch( error: { message: error instanceof Error ? error.message : 'Failed to update job' }, }); } - } + }) ); // =================================== @@ -114,7 +136,7 @@ router.post( authenticate, requirePermission('settings.update'), logActivity('jobs.start', 'job'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { const { jobId } = req.params; JobScheduler.startJob(jobId); @@ -126,7 +148,7 @@ router.post( message: error instanceof Error ? error.message : 'Failed to start job', }); } - } + }) ); // =================================== @@ -137,7 +159,7 @@ router.post( authenticate, requirePermission('settings.update'), logActivity('jobs.stop', 'job'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { const { jobId } = req.params; JobScheduler.stopJob(jobId); @@ -149,7 +171,7 @@ router.post( message: error instanceof Error ? error.message : 'Failed to stop job', }); } - } + }) ); // =================================== @@ -160,7 +182,7 @@ router.post( authenticate, requirePermission('settings.update'), logActivity('jobs.trigger', 'job'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { const { jobId } = req.params; const userId = req.user!.id.toString(); @@ -180,7 +202,7 @@ router.post( logger.error(`Failed to trigger job ${req.params.jobId}:`, error); res.status(500).json({ success: false, message }); } - } + }) ); // =================================== @@ -190,11 +212,11 @@ router.get( '/:jobId/logs', authenticate, requirePermission('settings.update'), - async (req: Request, res: Response) => { + asyncHandler(async (req: Request, res: Response) => { try { const { jobId } = req.params; const limit = Math.min(parseInt(req.query.limit as string) || 50, 100); - const offset = parseInt(req.query.offset as string) || 0; + const offset = Math.max(0, parseInt(req.query.offset as string, 10) || 0); const result = executionService.getJobLogs(jobId, limit, offset); res.json({ ...result, limit, offset }); @@ -202,28 +224,7 @@ router.get( logger.error(`Failed to get logs for job ${req.params.jobId}:`, error); res.status(500).json({ error: { message: 'Failed to retrieve job logs' } }); } - } -); - -// =================================== -// GET ALL EXECUTION LOGS -// =================================== -router.get( - '/logs/all', - authenticate, - requirePermission('settings.update'), - async (req: Request, res: Response) => { - try { - const limit = Math.min(parseInt(req.query.limit as string) || 100, 200); - const offset = parseInt(req.query.offset as string) || 0; - - const result = executionService.getAllLogs(limit, offset); - res.json({ ...result, limit, offset }); - } catch (error) { - logger.error('Failed to get all logs:', error); - res.status(500).json({ error: { message: 'Failed to retrieve logs' } }); - } - } + }) ); export default router; diff --git a/backend/src/routes/play-tracking.ts b/backend/src/routes/play-tracking.ts index 49ea2eb..4e40c35 100644 --- a/backend/src/routes/play-tracking.ts +++ b/backend/src/routes/play-tracking.ts @@ -39,6 +39,7 @@ const LIMITS = { router.use(requireFeature('enableStatistics')); // All routes require authentication and games.play permission +// Note: authenticate middleware guarantees req.user is populated router.use(authenticate); router.use(requirePermission('games.play')); @@ -56,11 +57,7 @@ router.post( asyncHandler(async (req: Request, res: Response) => { const { gameId, gameTitle } = startPlaySessionSchema.parse(req.body); - if (!req.user) { - throw new AppError(401, 'Authentication required'); - } - - const sessionId = await playTrackingService.startPlaySession(req.user.id, gameId, gameTitle); + const sessionId = await playTrackingService.startPlaySession(req.user!.id, gameId, gameTitle); // Store sessionId for activity logging res.locals.sessionId = sessionId; @@ -82,7 +79,7 @@ router.post( asyncHandler(async (req: Request, res: Response) => { const { sessionId } = endPlaySessionSchema.parse(req.body); - await playTrackingService.endPlaySession(sessionId); + await playTrackingService.endPlaySession(sessionId, req.user!.id); res.json({ success: true, @@ -98,11 +95,7 @@ router.post( router.get( '/stats', asyncHandler(async (req: Request, res: Response) => { - if (!req.user) { - throw new AppError(401, 'Authentication required'); - } - - const stats = await playTrackingService.getUserStats(req.user.id); + const stats = await playTrackingService.getUserStats(req.user!.id); res.json(stats); }) @@ -115,17 +108,13 @@ router.get( router.get( '/game-stats', asyncHandler(async (req: Request, res: Response) => { - if (!req.user) { - throw new AppError(401, 'Authentication required'); - } - const limit = Math.min( parseInt(req.query.limit as string) || LIMITS.GAME_STATS_DEFAULT, LIMITS.GAME_STATS_MAX ); - const offset = parseInt(req.query.offset as string) || 0; + const offset = Math.max(0, parseInt(req.query.offset as string) || 0); - const stats = await playTrackingService.getUserGameStats(req.user.id, limit, offset); + const stats = await playTrackingService.getUserGameStats(req.user!.id, limit, offset); res.json({ data: stats, @@ -142,17 +131,13 @@ router.get( router.get( '/history', asyncHandler(async (req: Request, res: Response) => { - if (!req.user) { - throw new AppError(401, 'Authentication required'); - } - const limit = Math.min( parseInt(req.query.limit as string) || LIMITS.HISTORY_DEFAULT, LIMITS.HISTORY_MAX ); - const offset = parseInt(req.query.offset as string) || 0; + const offset = Math.max(0, parseInt(req.query.offset as string) || 0); - const history = await playTrackingService.getUserPlayHistory(req.user.id, limit, offset); + const history = await playTrackingService.getUserPlayHistory(req.user!.id, limit, offset); res.json({ data: history, @@ -169,16 +154,12 @@ router.get( router.get( '/top-games', asyncHandler(async (req: Request, res: Response) => { - if (!req.user) { - throw new AppError(401, 'Authentication required'); - } - const limit = Math.min( parseInt(req.query.limit as string) || LIMITS.TOP_GAMES_DEFAULT, LIMITS.TOP_GAMES_MAX ); - const topGames = await playTrackingService.getTopGames(req.user.id, limit); + const topGames = await playTrackingService.getTopGames(req.user!.id, limit); res.json(topGames); }) @@ -191,16 +172,12 @@ router.get( router.get( '/activity-over-time', asyncHandler(async (req: Request, res: Response) => { - if (!req.user) { - throw new AppError(401, 'Authentication required'); - } - const days = Math.min( parseInt(req.query.days as string) || LIMITS.ACTIVITY_DAYS_DEFAULT, LIMITS.ACTIVITY_DAYS_MAX ); - const activity = await playTrackingService.getPlayActivityOverTime(req.user.id, days); + const activity = await playTrackingService.getPlayActivityOverTime(req.user!.id, days); res.json(activity); }) @@ -213,16 +190,12 @@ router.get( router.get( '/games-distribution', asyncHandler(async (req: Request, res: Response) => { - if (!req.user) { - throw new AppError(401, 'Authentication required'); - } - const limit = Math.min( parseInt(req.query.limit as string) || LIMITS.GAMES_DISTRIBUTION_DEFAULT, LIMITS.GAMES_DISTRIBUTION_MAX ); - const distribution = await playTrackingService.getGamesDistribution(req.user.id, limit); + const distribution = await playTrackingService.getGamesDistribution(req.user!.id, limit); res.json(distribution); }) diff --git a/backend/src/routes/playlists.ts b/backend/src/routes/playlists.ts index e603f6f..411d00c 100644 --- a/backend/src/routes/playlists.ts +++ b/backend/src/routes/playlists.ts @@ -10,6 +10,7 @@ import { authenticate } from '../middleware/auth'; import { requirePermission } from '../middleware/rbac'; import { requireFeature } from '../middleware/featureFlags'; import { logActivity } from '../middleware/activityLogger'; +import { z } from 'zod'; const router = Router(); const playlistService = new PlaylistService(); @@ -21,6 +22,16 @@ const playlistService = new PlaylistService(); // Admins with settings.update permission will bypass this check (via global softAuth) router.use(requireFeature('enablePlaylists')); +// Validation schemas +const createPlaylistSchema = z.object({ + title: z.string().min(1).max(200), + description: z.string().max(2000).optional(), +}); + +const addGamesToPlaylistSchema = z.object({ + gameIds: z.array(z.string()).min(1), +}); + // GET /api/playlists - List all playlists // No additional auth needed - softAuth already runs at router level router.get( @@ -53,13 +64,9 @@ router.post( requirePermission('playlists.create'), logActivity('playlists.create', 'playlists'), asyncHandler(async (req, res) => { - const data: CreatePlaylistDto = req.body; + const data = createPlaylistSchema.parse(req.body); - if (!data.title) { - throw new AppError(400, 'Playlist title is required'); - } - - const playlist = await playlistService.createPlaylist(data); + const playlist = await playlistService.createPlaylist(data as CreatePlaylistDto); res.status(201).json(playlist); }) ); @@ -71,13 +78,12 @@ router.post( requirePermission('playlists.update'), logActivity('playlists.add_games', 'playlists'), asyncHandler(async (req, res) => { - const data: AddGamesToPlaylistDto = req.body; + const data = addGamesToPlaylistSchema.parse(req.body); - if (!data.gameIds || !Array.isArray(data.gameIds)) { - throw new AppError(400, 'gameIds array is required'); - } - - const playlist = await playlistService.addGamesToPlaylist(req.params.id, data); + const playlist = await playlistService.addGamesToPlaylist( + req.params.id, + data as AddGamesToPlaylistDto + ); if (!playlist) { throw new AppError(404, 'Playlist not found'); @@ -94,13 +100,12 @@ router.delete( requirePermission('playlists.update'), logActivity('playlists.remove_games', 'playlists'), asyncHandler(async (req, res) => { - const data: AddGamesToPlaylistDto = req.body; + const data = addGamesToPlaylistSchema.parse(req.body); - if (!data.gameIds || !Array.isArray(data.gameIds)) { - throw new AppError(400, 'gameIds array is required'); - } - - const playlist = await playlistService.removeGamesFromPlaylist(req.params.id, data); + const playlist = await playlistService.removeGamesFromPlaylist( + req.params.id, + data as AddGamesToPlaylistDto + ); if (!playlist) { throw new AppError(404, 'Playlist not found'); diff --git a/backend/src/routes/proxy.ts b/backend/src/routes/proxy.ts index 1528ce8..bbbb720 100644 --- a/backend/src/routes/proxy.ts +++ b/backend/src/routes/proxy.ts @@ -1,5 +1,4 @@ import { Router, Response, NextFunction } from 'express'; -import { createProxyMiddleware } from 'http-proxy-middleware'; import { config, getExternalImageUrls } from '../config'; import { asyncHandler } from '../middleware/asyncHandler'; import { logger } from '../utils/logger'; @@ -55,6 +54,7 @@ async function serveFileWithFallback( // Try local file first if (fs.existsSync(localPath)) { logger.debug(`[Proxy] Serving local file: ${localPath}`); + res.setHeader('X-Content-Type-Options', 'nosniff'); return res.sendFile(localPath); } @@ -93,11 +93,26 @@ async function serveFileWithFallback( `[Proxy] Failed to cache image locally: ${cacheError instanceof Error ? cacheError.message : 'Unknown error'}` ); } - })(); + })().catch((err) => { + logger.warn('[Proxy] Cache write failed:', err); + }); + + // Determine content type from file extension (not from CDN response) + const ext = path.extname(relativePath).slice(1).toLowerCase(); + const mimeMap: Record = { + jpg: 'image/jpeg', + jpeg: 'image/jpeg', + png: 'image/png', + gif: 'image/gif', + webp: 'image/webp', + svg: 'image/svg+xml', + ico: 'image/x-icon', + bmp: 'image/bmp', + }; + const contentType = mimeMap[ext] || 'application/octet-stream'; - // Set appropriate content type - const contentType = response.headers['content-type'] || 'image/jpeg'; res.setHeader('Content-Type', contentType); + res.setHeader('X-Content-Type-Options', 'nosniff'); res.setHeader('Cache-Control', 'public, max-age=86400'); // Cache for 1 day return res.send(imageBuffer); @@ -123,32 +138,8 @@ async function serveFileWithFallback( } } -// Proxy to Flashpoint Game Server for game content -router.use( - '/game', - createProxyMiddleware({ - target: config.gameServerUrl, - changeOrigin: true, - pathRewrite: { - '^/proxy/game': '', // Remove /proxy/game prefix - }, - onProxyReq: (proxyReq, req, _res) => { - // Add headers if needed - if (req.ip) { - proxyReq.setHeader('X-Forwarded-For', req.ip); - } - }, - onError: (err, req, res) => { - logger.error('Proxy error:', err); - res.status(500).json({ - error: { - message: 'Failed to proxy request to game server', - statusCode: 500, - }, - }); - }, - }) -); +// Note: Game content is now served directly via /game-proxy and /game-zip routes +// (registered in server.ts before middleware). No proxy to external game-service needed. // Serve images with CDN fallback router.get( diff --git a/backend/src/routes/roles.ts b/backend/src/routes/roles.ts index bc58d31..f08d0f0 100644 --- a/backend/src/routes/roles.ts +++ b/backend/src/routes/roles.ts @@ -68,6 +68,10 @@ router.get( logActivity('roles.view', 'roles'), asyncHandler(async (req, res) => { const id = parseInt(req.params.id); + if (isNaN(id)) { + throw new AppError(400, 'Invalid role ID'); + } + const role = await roleService.getRoleById(id); if (!role) { @@ -112,6 +116,10 @@ router.patch( logActivity('roles.update', 'roles'), asyncHandler(async (req, res) => { const id = parseInt(req.params.id); + if (isNaN(id)) { + throw new AppError(400, 'Invalid role ID'); + } + const data = updateRoleSchema.parse(req.body); const role = await roleService.updateRole(id, data.name, data.description, data.priority); @@ -131,6 +139,10 @@ router.put( logActivity('roles.update_permissions', 'roles'), asyncHandler(async (req, res) => { const id = parseInt(req.params.id); + if (isNaN(id)) { + throw new AppError(400, 'Invalid role ID'); + } + const data = updatePermissionsSchema.parse(req.body); await roleService.updateRolePermissions(id, data.permissionIds); @@ -151,6 +163,9 @@ router.delete( logActivity('roles.delete', 'roles'), asyncHandler(async (req, res) => { const id = parseInt(req.params.id); + if (isNaN(id)) { + throw new AppError(400, 'Invalid role ID'); + } await roleService.deleteRole(id); diff --git a/backend/src/routes/shared-playlists.ts b/backend/src/routes/shared-playlists.ts index 9996e73..7280201 100644 --- a/backend/src/routes/shared-playlists.ts +++ b/backend/src/routes/shared-playlists.ts @@ -1,4 +1,5 @@ import { Router } from 'express'; +import { z } from 'zod'; import { validateSharedPlaylist } from '../middleware/validateSharedPlaylist'; import { asyncHandler } from '../middleware/asyncHandler'; import { authenticate } from '../middleware/auth'; @@ -14,6 +15,11 @@ const router = Router(); const playlistService = new UserPlaylistService(); const userService = new UserService(); +// Validation schema +const clonePlaylistSchema = z.object({ + newTitle: z.string().min(1).max(255).optional(), +}); + /** * GET /api/playlists/shared/:shareToken * Get shared playlist metadata (anonymous access) @@ -28,7 +34,16 @@ router.get( const playlist = req.sharedPlaylist!; // Sanitize response - don't expose sensitive fields - const response: any = { + const response: { + id: number; + title: string; + description: string | null; + icon: string | null; + gameCount: number; + createdAt: string; + updatedAt: string; + ownerUsername?: string; + } = { id: playlist.id, title: playlist.title, description: playlist.description, @@ -144,7 +159,8 @@ router.post( requirePermission('playlists.create'), validateSharedPlaylist, asyncHandler(async (req, res) => { - const { newTitle } = req.body; + // Validate request body + const { newTitle } = clonePlaylistSchema.parse(req.body); const clonedPlaylist = playlistService.cloneSharedPlaylist( req.params.shareToken, diff --git a/backend/src/routes/system-settings.ts b/backend/src/routes/system-settings.ts index d6c06f5..995c249 100644 --- a/backend/src/routes/system-settings.ts +++ b/backend/src/routes/system-settings.ts @@ -7,6 +7,7 @@ import { PermissionCache } from '../services/PermissionCache'; import { authenticate } from '../middleware/auth'; import { requirePermission } from '../middleware/rbac'; import { logActivity } from '../middleware/activityLogger'; +import { asyncHandler } from '../middleware/asyncHandler'; import { JobScheduler } from '../services/JobScheduler'; import { logger } from '../utils/logger'; import { CategorySettings } from '../types/settings'; @@ -44,24 +45,18 @@ router.get( authenticate, requirePermission('settings.update'), logActivity('settings.list', 'system_settings'), - async (req: Request, res: Response) => { - try { - const allSettings = systemSettings.getAll(); - res.json(allSettings); - } catch (error) { - logger.error('Failed to get all settings:', error); - res.status(500).json({ - error: { message: 'Failed to retrieve settings' }, - }); - } - } + asyncHandler(async (req: Request, res: Response) => { + const allSettings = systemSettings.getAll(); + res.json(allSettings); + }) ); // =================================== // GET PUBLIC SETTINGS (No Auth Required) // =================================== -router.get('/public', async (req: Request, res: Response) => { - try { +router.get( + '/public', + asyncHandler(async (req: Request, res: Response) => { const publicSettings: Record< string, Record @@ -91,14 +86,121 @@ router.get('/public', async (req: Request, res: Response) => { } res.json(publicSettings); - } catch (error) { - logger.error('Failed to get public settings:', error); - res.status(500).json({ - error: { message: 'Failed to retrieve public settings' }, + }) +); + +// =================================== +// GET CACHE STATISTICS (Admin Only) +// =================================== +router.get( + '/_cache/stats', + authenticate, + requirePermission('settings.update'), + asyncHandler(async (req: Request, res: Response) => { + const stats = systemSettings.getCacheStats(); + res.json(stats); + }) +); + +// =================================== +// CLEAR CACHE (Admin Only) +// =================================== +router.post( + '/_cache/clear', + authenticate, + requirePermission('settings.update'), + logActivity('settings.cache_clear', 'system_settings'), + asyncHandler(async (req: Request, res: Response) => { + const { category } = req.body; + + if (category) { + systemSettings.clearCategoryCache(category); + } else { + systemSettings.clearCache(); + } + + res.json({ message: 'Cache cleared successfully' }); + }) +); + +// =================================== +// GET PERMISSION CACHE STATISTICS (Admin Only) +// =================================== +router.get( + '/_cache/permissions/stats', + authenticate, + requirePermission('settings.update'), + asyncHandler(async (req: Request, res: Response) => { + const stats = PermissionCache.getStats(); + res.json({ + ...stats, + description: { + userCacheSize: 'Number of cached user permissions', + roleCacheSize: 'Number of cached role permissions', + totalSize: 'Total cached entries', + ttl: { + userPermissions: '5 minutes', + rolePermissions: '10 minutes', + }, + }, }); - } + }) +); + +// Validation schema for permission cache clear +const clearPermissionCacheSchema = z.object({ + type: z.enum(['all', 'user', 'role', 'users', 'roles']), + id: z.number().int().positive().optional(), }); +// =================================== +// CLEAR PERMISSION CACHE (Admin Only) +// =================================== +router.post( + '/_cache/permissions/clear', + authenticate, + requirePermission('settings.update'), + logActivity('settings.permission_cache_clear', 'system_settings'), + asyncHandler(async (req: Request, res: Response) => { + const validation = clearPermissionCacheSchema.safeParse(req.body); + + if (!validation.success) { + return res.status(400).json({ + error: { + message: 'Invalid request body', + details: validation.error.errors, + }, + }); + } + + const { type, id } = validation.data; + + if (type === 'user' && id) { + PermissionCache.invalidateUser(Number(id)); + res.json({ message: `User ${id} permission cache cleared successfully` }); + } else if (type === 'role' && id) { + PermissionCache.invalidateRole(Number(id)); + res.json({ message: `Role ${id} permission cache cleared successfully` }); + } else if (type === 'users') { + PermissionCache.invalidateAllUsers(); + res.json({ message: 'All user permission caches cleared successfully' }); + } else if (type === 'roles') { + PermissionCache.invalidateAllRoles(); + res.json({ message: 'All role permission caches cleared successfully' }); + } else if (type === 'all') { + PermissionCache.clearAll(); + res.json({ message: 'All permission caches cleared successfully' }); + } else { + res.status(400).json({ + error: { + message: + 'Invalid request. Specify type: "user", "role", "users", "roles", or "all". For "user" or "role", provide an id.', + }, + }); + } + }) +); + // =================================== // GET CATEGORY SETTINGS (Admin Only) // =================================== @@ -106,20 +208,13 @@ router.get( '/:category', authenticate, requirePermission('settings.read'), - async (req: Request, res: Response) => { - try { - const { category } = req.params; - const categorySettings = systemSettings.getCategory(category); + asyncHandler(async (req: Request, res: Response) => { + const { category } = req.params; + const categorySettings = systemSettings.getCategory(category); - // Return settings (empty object if category has no settings yet) - res.json(categorySettings); - } catch (error) { - logger.error(`Failed to get ${req.params.category} settings:`, error); - res.status(500).json({ - error: { message: 'Failed to retrieve settings' }, - }); - } - } + // Return settings (empty object if category has no settings yet) + res.json(categorySettings); + }) ); // =================================== @@ -134,47 +229,32 @@ router.patch( authenticate, requirePermission('settings.update'), logActivity('settings.update', 'system_settings'), - async (req: Request, res: Response) => { - try { - const { category } = req.params; - const validation = updateCategorySchema.safeParse(req.body); + asyncHandler(async (req: Request, res: Response) => { + const { category } = req.params; + const validation = updateCategorySchema.safeParse(req.body); - if (!validation.success) { - return res.status(400).json({ - error: { - message: 'Invalid request body', - details: validation.error.errors, - }, - }); - } - - const settings = validation.data; - const userId = req.user!.id; - - // Update settings (validation happens in service) - systemSettings.updateCategory(category, settings, userId); - - // Update job scheduler if jobs settings changed - updateJobScheduler(category, settings); - - // Return updated settings - const updated = systemSettings.getCategory(category); - res.json(updated); - } catch (error) { - logger.error(`Failed to update ${req.params.category} settings:`, error); - - // Check if it's a validation error - if (error instanceof Error && error.message.includes('must be')) { - return res.status(400).json({ - error: { message: error.message }, - }); - } - - res.status(500).json({ - error: { message: 'Failed to update settings' }, + if (!validation.success) { + return res.status(400).json({ + error: { + message: 'Invalid request body', + details: validation.error.errors, + }, }); } - } + + const settings = validation.data; + const userId = req.user!.id; + + // Update settings (validation happens in service) + systemSettings.updateCategory(category, settings, userId); + + // Update job scheduler if jobs settings changed + updateJobScheduler(category, settings); + + // Return updated settings + const updated = systemSettings.getCategory(category); + res.json(updated); + }) ); // =================================== @@ -184,26 +264,19 @@ router.get( '/:category/:key', authenticate, requirePermission('settings.read'), - async (req: Request, res: Response) => { - try { - const { category, key } = req.params; - const fullKey = `${category}.${key}`; - const value = systemSettings.get(fullKey); + asyncHandler(async (req: Request, res: Response) => { + const { category, key } = req.params; + const fullKey = `${category}.${key}`; + const value = systemSettings.get(fullKey); - if (value === null) { - return res.status(404).json({ - error: { message: `Setting '${fullKey}' not found` }, - }); - } - - res.json({ value }); - } catch (error) { - logger.error(`Failed to get setting ${req.params.category}.${req.params.key}:`, error); - res.status(500).json({ - error: { message: 'Failed to retrieve setting' }, + if (value === null) { + return res.status(404).json({ + error: { message: `Setting '${fullKey}' not found` }, }); } - } + + res.json({ value }); + }) ); // =================================== @@ -218,168 +291,30 @@ router.patch( authenticate, requirePermission('settings.update'), logActivity('settings.update', 'system_settings'), - async (req: Request, res: Response) => { - try { - const { category, key } = req.params; - const fullKey = `${category}.${key}`; - const validation = updateSettingSchema.safeParse(req.body); + asyncHandler(async (req: Request, res: Response) => { + const { category, key } = req.params; + const fullKey = `${category}.${key}`; + const validation = updateSettingSchema.safeParse(req.body); - if (!validation.success) { - return res.status(400).json({ - error: { - message: 'Invalid request body', - details: validation.error.errors, - }, - }); - } - - const { value } = validation.data; - const userId = req.user!.id; - - // Update setting (validation happens in service) - systemSettings.set(fullKey, value, userId); - - // Return updated value - const updated = systemSettings.get(fullKey); - res.json({ value: updated }); - } catch (error) { - logger.error(`Failed to update setting ${req.params.category}.${req.params.key}:`, error); - - // Check if it's a validation error - if (error instanceof Error && error.message.includes('must be')) { - return res.status(400).json({ - error: { message: error.message }, - }); - } - - res.status(500).json({ - error: { message: 'Failed to update setting' }, - }); - } - } -); - -// =================================== -// GET CACHE STATISTICS (Admin Only) -// =================================== -router.get( - '/_cache/stats', - authenticate, - requirePermission('settings.update'), - async (req: Request, res: Response) => { - try { - const stats = systemSettings.getCacheStats(); - res.json(stats); - } catch (error) { - logger.error('Failed to get cache stats:', error); - res.status(500).json({ - error: { message: 'Failed to retrieve cache statistics' }, - }); - } - } -); - -// =================================== -// CLEAR CACHE (Admin Only) -// =================================== -router.post( - '/_cache/clear', - authenticate, - requirePermission('settings.update'), - logActivity('settings.cache_clear', 'system_settings'), - async (req: Request, res: Response) => { - try { - const { category } = req.body; - - if (category) { - systemSettings.clearCategoryCache(category); - } else { - systemSettings.clearCache(); - } - - res.json({ message: 'Cache cleared successfully' }); - } catch (error) { - logger.error('Failed to clear cache:', error); - res.status(500).json({ - error: { message: 'Failed to clear cache' }, - }); - } - } -); - -// =================================== -// GET PERMISSION CACHE STATISTICS (Admin Only) -// =================================== -router.get( - '/_cache/permissions/stats', - authenticate, - requirePermission('settings.update'), - async (req: Request, res: Response) => { - try { - const stats = PermissionCache.getStats(); - res.json({ - ...stats, - description: { - userCacheSize: 'Number of cached user permissions', - roleCacheSize: 'Number of cached role permissions', - totalSize: 'Total cached entries', - ttl: { - userPermissions: '5 minutes', - rolePermissions: '10 minutes', - }, + if (!validation.success) { + return res.status(400).json({ + error: { + message: 'Invalid request body', + details: validation.error.errors, }, }); - } catch (error) { - logger.error('Failed to get permission cache stats:', error); - res.status(500).json({ - error: { message: 'Failed to retrieve permission cache statistics' }, - }); } - } -); -// =================================== -// CLEAR PERMISSION CACHE (Admin Only) -// =================================== -router.post( - '/_cache/permissions/clear', - authenticate, - requirePermission('settings.update'), - logActivity('settings.permission_cache_clear', 'system_settings'), - async (req: Request, res: Response) => { - try { - const { type, id } = req.body; + const { value } = validation.data; + const userId = req.user!.id; - if (type === 'user' && id) { - PermissionCache.invalidateUser(Number(id)); - res.json({ message: `User ${id} permission cache cleared successfully` }); - } else if (type === 'role' && id) { - PermissionCache.invalidateRole(Number(id)); - res.json({ message: `Role ${id} permission cache cleared successfully` }); - } else if (type === 'users') { - PermissionCache.invalidateAllUsers(); - res.json({ message: 'All user permission caches cleared successfully' }); - } else if (type === 'roles') { - PermissionCache.invalidateAllRoles(); - res.json({ message: 'All role permission caches cleared successfully' }); - } else if (type === 'all') { - PermissionCache.clearAll(); - res.json({ message: 'All permission caches cleared successfully' }); - } else { - res.status(400).json({ - error: { - message: - 'Invalid request. Specify type: "user", "role", "users", "roles", or "all". For "user" or "role", provide an id.', - }, - }); - } - } catch (error) { - logger.error('Failed to clear permission cache:', error); - res.status(500).json({ - error: { message: 'Failed to clear permission cache' }, - }); - } - } + // Update setting (validation happens in service) + systemSettings.set(fullKey, value, userId); + + // Return updated value + const updated = systemSettings.get(fullKey); + res.json({ value: updated }); + }) ); export default router; diff --git a/backend/src/routes/updates.ts b/backend/src/routes/updates.ts index 39cfd63..c65c5a9 100644 --- a/backend/src/routes/updates.ts +++ b/backend/src/routes/updates.ts @@ -7,9 +7,11 @@ import { authenticate } from '../middleware/auth'; import { requirePermission } from '../middleware/rbac'; import { logActivity } from '../middleware/activityLogger'; import { asyncHandler } from '../middleware/asyncHandler'; +import { rateLimitStandard } from '../middleware/rateLimiter'; import { logger } from '../utils/logger'; const router = Router(); +router.use(rateLimitStandard); const metadataUpdateService = new MetadataUpdateService(); const metadataSyncService = new MetadataSyncService(); diff --git a/backend/src/routes/user-playlists.ts b/backend/src/routes/user-playlists.ts index ea9f197..2fe62e2 100644 --- a/backend/src/routes/user-playlists.ts +++ b/backend/src/routes/user-playlists.ts @@ -36,15 +36,15 @@ const updatePlaylistSchema = z.object({ }); const addGamesSchema = z.object({ - gameIds: z.array(z.string()).min(1), + gameIds: z.array(z.string()).min(1).max(100), }); const removeGamesSchema = z.object({ - gameIds: z.array(z.string()).min(1), + gameIds: z.array(z.string()).min(1).max(100), }); const reorderGamesSchema = z.object({ - gameIdOrder: z.array(z.string()).min(1), + gameIdOrder: z.array(z.string()).min(1).max(100), }); const copyFlashpointPlaylistSchema = z.object({ diff --git a/backend/src/routes/users.ts b/backend/src/routes/users.ts index 372d028..1cc7016 100644 --- a/backend/src/routes/users.ts +++ b/backend/src/routes/users.ts @@ -26,7 +26,7 @@ const updateUserSchema = z.object({ }); const changePasswordSchema = z.object({ - currentPassword: z.string().min(6), + currentPassword: z.string().min(6).optional(), newPassword: z.string().min(6), }); @@ -40,7 +40,7 @@ const updateThemeSettingsSchema = z.object({ primaryColor: z.string().min(1), }); -const updateUserSettingsSchema = z.record(z.string(), z.string()); +const updateUserSettingsSchema = z.record(z.string().max(50), z.string().max(500)); /** * GET /api/users @@ -52,8 +52,8 @@ router.get( requirePermission('users.read'), logActivity('users.list', 'users'), asyncHandler(async (req, res) => { - const page = parseInt(req.query.page as string) || 1; - const limit = parseInt(req.query.limit as string) || 50; + const page = Math.max(1, parseInt(req.query.page as string) || 1); + const limit = Math.min(parseInt(req.query.limit as string) || 50, 100); const result = await userService.getUsers(page, limit); @@ -61,117 +61,6 @@ router.get( }) ); -/** - * GET /api/users/:id - * Get single user by ID - */ -router.get( - '/:id', - authenticate, - requirePermission('users.read'), - logActivity('users.view', 'users'), - asyncHandler(async (req, res) => { - const id = parseInt(req.params.id); - const user = await userService.getUserById(id); - - if (!user) { - throw new AppError(404, 'User not found'); - } - - res.json(user); - }) -); - -/** - * POST /api/users - * Create new user (admin only) - */ -router.post( - '/', - authenticate, - requirePermission('users.create'), - logActivity('users.create', 'users'), - asyncHandler(async (req, res) => { - const data = createUserSchema.parse(req.body); - - const user = await userService.createUser({ - username: data.username, - email: data.email, - password: data.password, - roleId: data.roleId, - isActive: data.isActive ?? true, - }); - - res.status(201).json(user); - }) -); - -/** - * PATCH /api/users/:id - * Update existing user - */ -router.patch( - '/:id', - authenticate, - requirePermission('users.update'), - logActivity('users.update.profile', 'users', (req) => ({ - userId: req.params.id, - fieldsUpdated: Object.keys(req.body), - })), - asyncHandler(async (req, res) => { - const id = parseInt(req.params.id); - const data = updateUserSchema.parse(req.body); - - const user = await userService.updateUser(id, data); - - res.json(user); - }) -); - -/** - * DELETE /api/users/:id - * Delete user (prevents deleting last admin) - */ -router.delete( - '/:id', - authenticate, - requirePermission('users.delete'), - logActivity('users.delete', 'users'), - asyncHandler(async (req, res) => { - const id = parseInt(req.params.id); - - await userService.deleteUser(id); - - res.json({ success: true, message: 'User deleted successfully' }); - }) -); - -/** - * POST /api/users/:id/change-password - * Change user password - * Users can change their own password, admins can change any password - */ -router.post( - '/:id/change-password', - authenticate, - asyncHandler(async (req, res) => { - const id = parseInt(req.params.id); - const data = changePasswordSchema.parse(req.body); - - // Check if user is changing their own password or is an admin - const isOwnPassword = req.user?.id === id; - const isAdmin = req.user?.permissions.includes('users.update'); - - if (!isOwnPassword && !isAdmin) { - throw new AppError(403, 'Insufficient permissions'); - } - - await userService.changePassword(id, data.currentPassword, data.newPassword); - - res.json({ success: true, message: 'Password changed successfully' }); - }) -); - /** * GET /api/users/me/theme (DEPRECATED - redirects to new endpoint) * Legacy endpoint for backward compatibility @@ -289,4 +178,137 @@ router.patch( }) ); +/** + * GET /api/users/:id + * Get single user by ID + */ +router.get( + '/:id', + authenticate, + requirePermission('users.read'), + logActivity('users.view', 'users'), + asyncHandler(async (req, res) => { + const id = parseInt(req.params.id); + if (isNaN(id)) { + throw new AppError(400, 'Invalid user ID'); + } + + const user = await userService.getUserById(id); + + if (!user) { + throw new AppError(404, 'User not found'); + } + + res.json(user); + }) +); + +/** + * POST /api/users + * Create new user (admin only) + */ +router.post( + '/', + authenticate, + requirePermission('users.create'), + logActivity('users.create', 'users'), + asyncHandler(async (req, res) => { + const data = createUserSchema.parse(req.body); + + const user = await userService.createUser({ + username: data.username, + email: data.email, + password: data.password, + roleId: data.roleId, + isActive: data.isActive ?? true, + }); + + res.status(201).json(user); + }) +); + +/** + * PATCH /api/users/:id + * Update existing user + */ +router.patch( + '/:id', + authenticate, + requirePermission('users.update'), + logActivity('users.update.profile', 'users', (req) => ({ + userId: req.params.id, + fieldsUpdated: Object.keys(req.body), + })), + asyncHandler(async (req, res) => { + const id = parseInt(req.params.id); + if (isNaN(id)) { + throw new AppError(400, 'Invalid user ID'); + } + + const data = updateUserSchema.parse(req.body); + + const user = await userService.updateUser(id, data); + + res.json(user); + }) +); + +/** + * DELETE /api/users/:id + * Delete user (prevents deleting last admin) + */ +router.delete( + '/:id', + authenticate, + requirePermission('users.delete'), + logActivity('users.delete', 'users'), + asyncHandler(async (req, res) => { + const id = parseInt(req.params.id); + if (isNaN(id)) { + throw new AppError(400, 'Invalid user ID'); + } + + await userService.deleteUser(id); + + res.json({ success: true, message: 'User deleted successfully' }); + }) +); + +/** + * POST /api/users/:id/change-password + * Change user password + * Users can change their own password, admins can change any password + */ +router.post( + '/:id/change-password', + authenticate, + asyncHandler(async (req, res) => { + const id = parseInt(req.params.id); + if (isNaN(id)) { + throw new AppError(400, 'Invalid user ID'); + } + + const data = changePasswordSchema.parse(req.body); + + // Check if user is changing their own password or is an admin + const isOwnPassword = req.user?.id === id; + const isAdmin = req.user?.permissions.includes('users.update'); + + if (!isOwnPassword && !isAdmin) { + throw new AppError(403, 'Insufficient permissions'); + } + + const isAdminReset = isAdmin && !isOwnPassword; + + // Require current password for non-admin password changes + if (!isAdminReset && !data.currentPassword) { + throw new AppError(400, 'Current password is required'); + } + + await userService.changePassword(id, data.currentPassword || '', data.newPassword, isAdminReset); + + res.json({ success: true, message: 'Password changed successfully' }); + }) +); + export default router; diff --git a/backend/src/server.ts b/backend/src/server.ts index e1c658f..884f925 100644 --- a/backend/src/server.ts +++ b/backend/src/server.ts @@ -5,6 +5,7 @@ import express, { Express } from 'express'; import cors from 'cors'; import helmet from 'helmet'; import compression from 'compression'; +import cookieParser from 'cookie-parser'; import { config } from './config'; import { logger, getLoggingStatus, verifyFileLogging } from './utils/logger'; import { errorHandler } from './middleware/errorHandler'; @@ -25,6 +26,12 @@ import { RuffleUpdateJob } from './jobs/RuffleUpdateJob'; import { CachedSystemSettingsService } from './services/CachedSystemSettingsService'; import { PermissionCache } from './services/PermissionCache'; import { RuffleService } from './services/RuffleService'; +import { ConfigManager } from './game/config'; +import { PreferencesService } from './game/services/PreferencesService'; +import { zipManager } from './game/zip-manager'; +import { gameZipServer } from './game/gamezipserver'; +import gameProxyRouter from './routes/game-proxy'; +import gameZipRouter from './routes/game-zip'; /** * Clean up old activity logs based on retention settings @@ -57,6 +64,28 @@ async function cleanupActivityLogs(): Promise { async function startServer() { const app: Express = express(); + // Trust proxy - Required for Docker/reverse proxy deployments + // Enables Express to correctly read X-Forwarded-* headers from reverse proxies + // Without this, req.ip returns the proxy IP instead of the client IP, causing all rate limiters to share a single counter + app.set('trust proxy', 1); + + // --- Game content routes (BEFORE all middleware) --- + // These bypass auth, maintenance mode, restrictive CORS, and body parsing. + // Only a permissive CORS middleware is applied to game routes. + const gameCors = (_req: express.Request, res: express.Response, next: express.NextFunction) => { + res.setHeader('Access-Control-Allow-Origin', '*'); + res.setHeader('Access-Control-Allow-Methods', 'GET, POST, DELETE, OPTIONS, HEAD'); + res.setHeader('Access-Control-Allow-Headers', '*'); + if (_req.method === 'OPTIONS') { + res.status(204).end(); + return; + } + next(); + }; + app.use('/game-proxy', gameCors, gameProxyRouter); + app.use('/game-zip', gameCors, gameZipRouter); + // --- End game content routes --- + // Security middleware app.use( helmet({ @@ -98,6 +127,9 @@ async function startServer() { app.use(express.json({ limit: '1mb' })); app.use(express.urlencoded({ extended: true, limit: '1mb' })); + // Cookie parsing + app.use(cookieParser()); + // Compression app.use(compression()); @@ -107,7 +139,16 @@ async function startServer() { // Request logging app.use((req, res, next) => { - logger.info(`${req.method} ${req.path}`); + // Skip logging for high-frequency paths + if ( + req.path === '/health' || + req.path.startsWith('/game-proxy/') || + req.path.startsWith('/game-zip/') + ) { + next(); + return; + } + logger.debug(`${req.method} ${req.path}`); next(); }); @@ -217,58 +258,31 @@ async function startServer() { // Non-fatal error - continue server startup } - // Note: Game proxy server functionality has been moved to game-service - // The backend now proxies requests to game-service instead of serving them directly - logger.info('📡 Game content will be proxied to game-service'); - logger.info(` Proxy Server: ${config.gameServerUrl || 'http://localhost:22500'}`); - logger.info(` GameZip Server: http://localhost:${config.gameServerHttpPort}`); + // Initialize game service configuration (integrated into backend) + try { + await ConfigManager.loadConfig(config.flashpointPath); + PreferencesService.initialize(config.flashpointPath); + logger.info('🎮 Game service configuration loaded (integrated)'); + } catch (error) { + logger.warn('⚠️ Failed to load game service config (non-fatal):', error); + } - // Health check with game-service connectivity verification - app.get('/health', async (req, res) => { - let gameServiceStatus: 'ok' | 'error' | 'unknown' = 'unknown'; - let gameServiceError: string | undefined; - - // Check game-service connectivity - try { - const controller = new AbortController(); - const timeoutId = setTimeout(() => controller.abort(), 3000); // 3 second timeout - - const gameServiceUrl = config.gameServerUrl || 'http://localhost:22500'; - const response = await fetch(`${gameServiceUrl}/health`, { - signal: controller.signal, - }); - clearTimeout(timeoutId); - - if (response.ok) { - gameServiceStatus = 'ok'; - } else { - gameServiceStatus = 'error'; - gameServiceError = `HTTP ${response.status}`; - } - } catch (error) { - gameServiceStatus = 'error'; - gameServiceError = - error instanceof Error - ? error.name === 'AbortError' - ? 'Connection timeout' - : error.message - : 'Unknown error'; - } - - const isHealthy = DatabaseService.isConnected() && gameServiceStatus === 'ok'; + // Health check - game service is now integrated (no external dependency) + // NOTE: This endpoint is registered before middleware for load balancer access. + // It performs basic health checks. For detailed health status, use /api/health (requires auth). + app.get('/health', async (_req, res) => { + const isHealthy = DatabaseService.isConnected(); res.status(isHealthy ? 200 : 503).json({ status: isHealthy ? 'ok' : 'degraded', timestamp: new Date().toISOString(), - flashpointPath: config.flashpointPath, services: { database: { status: DatabaseService.isConnected() ? 'ok' : 'error', }, gameService: { - status: gameServiceStatus, - url: config.gameServerUrl || 'http://localhost:22500', - error: gameServiceError, + status: 'ok', + note: 'integrated', }, }, }); @@ -304,10 +318,16 @@ async function startServer() { const server = app.listen(config.port, config.host, () => { logger.info(`🚀 Flashpoint Web API server running on http://${config.host}:${config.port}`); logger.info(`📁 Flashpoint path: ${config.flashpointPath}`); - logger.info(`🎮 Game Service: ${config.gameServerUrl || 'http://localhost:22500'}`); + logger.info(`🎮 Game content: /game-proxy/* and /game-zip/* (integrated)`); logger.info(`🌍 Environment: ${config.nodeEnv}`); }); + // Prevent connection exhaustion - critical for handling game file requests + server.keepAliveTimeout = 65000; // 65s - slightly above common load balancer timeout + server.headersTimeout = 66000; // Must be > keepAliveTimeout + server.timeout = 120000; // 2 min max for any request (including game file streaming) + // Note: Connection limiting is handled by the reverse proxy (nginx) + // Start cleanup scheduler for abandoned play sessions (every 6 hours) const playTrackingService = new PlayTrackingService(); const playSessionCleanupInterval = setInterval( @@ -318,6 +338,7 @@ async function startServer() { }, 6 * 60 * 60 * 1000 ); // 6 hours + playSessionCleanupInterval.unref(); // Run initial play session cleanup playTrackingService.cleanupAbandonedSessions().catch((error) => { @@ -334,6 +355,7 @@ async function startServer() { }, 6 * 60 * 60 * 1000 ); // 6 hours + loginAttemptsCleanupInterval.unref(); // Run initial login attempts cleanup authService.cleanupOldLoginAttempts().catch((error) => { @@ -349,6 +371,7 @@ async function startServer() { }, 24 * 60 * 60 * 1000 ); // 24 hours + activityLogsCleanupInterval.unref(); // Run initial activity logs cleanup cleanupActivityLogs().catch((error) => { @@ -356,7 +379,7 @@ async function startServer() { }); // Graceful shutdown - const shutdown = () => { + const shutdown = async () => { logger.info('Shutting down gracefully...'); // Stop scheduled jobs @@ -370,6 +393,32 @@ async function startServer() { clearInterval(loginAttemptsCleanupInterval); clearInterval(activityLogsCleanupInterval); + // Dispose game zip server (stops download cleanup interval) + try { + gameZipServer.dispose(); + } catch (err) { + logger.error('Failed to dispose game zip server during shutdown:', err); + } + + // Clean up mounted ZIPs (await to release file handles) + try { + await zipManager.unmountAll(); + } catch (err) { + logger.error('Failed to unmount ZIPs during shutdown:', err); + } + + // Close database connections + try { + DatabaseService.close(); + } catch (err) { + logger.error('Failed to close database during shutdown:', err); + } + try { + UserDatabaseService.close(); + } catch (err) { + logger.error('Failed to close user database during shutdown:', err); + } + // Close server server.close(() => { logger.info('Server closed'); @@ -380,7 +429,7 @@ async function startServer() { setTimeout(() => { logger.error('Could not close connections in time, forcefully shutting down'); process.exit(1); - }, 10000); + }, 10000).unref(); }; process.on('SIGTERM', shutdown); diff --git a/backend/src/services/ActivityService.ts b/backend/src/services/ActivityService.ts index 3c7e344..352b66e 100644 --- a/backend/src/services/ActivityService.ts +++ b/backend/src/services/ActivityService.ts @@ -13,6 +13,8 @@ import { AUTH_ACTIONS, categorizeAction } from '../utils/activityUtils'; /** * Generate SQL IN clause from AUTH_ACTIONS constant. * Keeps the constant as the single source of truth for auth action strings. + * SAFETY: This is safe because AUTH_ACTIONS is a hardcoded constant array + * (not user input). The join operation builds a static SQL fragment at module load time. */ const AUTH_ACTIONS_SQL = `'${AUTH_ACTIONS.join("', '")}'`; diff --git a/backend/src/services/AuthService.ts b/backend/src/services/AuthService.ts index 459255d..3e17e8a 100644 --- a/backend/src/services/AuthService.ts +++ b/backend/src/services/AuthService.ts @@ -8,6 +8,25 @@ import { hashPassword, verifyPassword } from '../utils/password'; import { AppError } from '../middleware/errorHandler'; import { LoginCredentials, RegisterData, AuthTokens, AuthUser } from '../types/auth'; +/** + * User data for token generation + */ +interface UserTokenData { + id: number; + username: string; + role_name: string; +} + +/** + * Auth settings shape from system_settings + */ +interface AuthSettings { + guest_access_enabled: number; + user_registration_enabled: number; + max_login_attempts: number; + lockout_duration_minutes: number; +} + export class AuthService { private systemSettings: SystemSettingsService; private cachedSystemSettings: CachedSystemSettingsService; @@ -98,53 +117,55 @@ export class AuthService { * The method automatically detects which scenario applies based on whether users exist */ async register(data: RegisterData): Promise<{ user: AuthUser; tokens: AuthTokens }> { - // Check if this is initial setup (no users exist) - const isInitialSetup = UserDatabaseService.needsInitialSetup(); - - // Validation: Regular registration requires registration to be enabled - // Initial setup bypasses this check (first user is always allowed) - if (!isInitialSetup) { - const settings = this.getAuthSettings(); - if (!settings.user_registration_enabled) { - throw new AppError(403, 'User registration is currently disabled'); - } - } - - // Check if username exists - const existingUser = UserDatabaseService.get('SELECT id FROM users WHERE username = ?', [ - data.username, - ]); - if (existingUser) { - throw new AppError(409, 'Username already exists'); - } - - // Check if email exists - const existingEmail = UserDatabaseService.get('SELECT id FROM users WHERE email = ?', [ - data.email, - ]); - if (existingEmail) { - throw new AppError(409, 'Email already exists'); - } - - // Hash password + // Hash password before transaction const passwordHash = await hashPassword(data.password); - // First user becomes admin (role_id = 1), subsequent users get 'user' role (role_id = 2) - const roleId = isInitialSetup ? 1 : 2; - const result = UserDatabaseService.run( - 'INSERT INTO users (username, email, password_hash, role_id) VALUES (?, ?, ?, ?)', - [data.username, data.email, passwordHash, roleId] - ); + // Wrap setup check + uniqueness checks + insert in transaction to prevent race condition + const db = UserDatabaseService.getDatabase(); + const registerTransaction = db.transaction(() => { + // Check username uniqueness inside transaction + const existingUser = db.prepare('SELECT id FROM users WHERE username = ?').get(data.username); + if (existingUser) { + throw new AppError(409, 'Username already exists'); + } - if (isInitialSetup) { - logger.info('╔═══════════════════════════════════════════════════════════════╗'); - logger.info('║ INITIAL ADMIN ACCOUNT CREATED ║'); - logger.info(`║ Username: ${data.username.padEnd(48)} ║`); - logger.info('║ The first user has been granted administrator privileges. ║'); - logger.info('╚═══════════════════════════════════════════════════════════════╝'); - } + // Check email uniqueness inside transaction + const existingEmail = db.prepare('SELECT id FROM users WHERE email = ?').get(data.email); + if (existingEmail) { + throw new AppError(409, 'Email already exists'); + } - const userId = result.lastInsertRowid as number; + // Check if this is initial setup (no users exist) + const count = db.prepare('SELECT COUNT(*) as count FROM users').get() as { count: number }; + const isInitialSetup = count.count === 0; + + // Validation: Regular registration requires registration to be enabled + // Initial setup bypasses this check (first user is always allowed) + if (!isInitialSetup) { + const settings = this.getAuthSettings(); + if (!settings.user_registration_enabled) { + throw new AppError(403, 'User registration is currently disabled'); + } + } + + // First user becomes admin (role_id = 1), subsequent users get 'user' role (role_id = 2) + const roleId = isInitialSetup ? 1 : 2; + const result = db + .prepare('INSERT INTO users (username, email, password_hash, role_id) VALUES (?, ?, ?, ?)') + .run(data.username, data.email, passwordHash, roleId); + + if (isInitialSetup) { + logger.info('╔═══════════════════════════════════════════════════════════════╗'); + logger.info('║ INITIAL ADMIN ACCOUNT CREATED ║'); + logger.info(`║ Username: ${data.username.padEnd(48)} ║`); + logger.info('║ The first user has been granted administrator privileges. ║'); + logger.info('╚═══════════════════════════════════════════════════════════════╝'); + } + + return result.lastInsertRowid as number; + }); + + const userId = registerTransaction(); // Apply default theme and primary color from system settings const appSettings = this.cachedSystemSettings.getCategory('app'); @@ -319,7 +340,7 @@ export class AuthService { /** * Generate access and refresh tokens */ - private async generateTokens(user: any): Promise { + private async generateTokens(user: UserTokenData): Promise { const accessToken = generateAccessToken({ userId: user.id, username: user.username, @@ -406,7 +427,7 @@ export class AuthService { /** * Get auth settings from system_settings */ - private getAuthSettings(): any { + private getAuthSettings(): AuthSettings { try { const authSettings = this.systemSettings.getCategory('auth'); @@ -424,8 +445,12 @@ export class AuthService { ? 1 : 0 : 1, - max_login_attempts: authSettings.maxLoginAttempts || 5, - lockout_duration_minutes: authSettings.lockoutDurationMinutes || 15, + max_login_attempts: + typeof authSettings.maxLoginAttempts === 'number' ? authSettings.maxLoginAttempts : 5, + lockout_duration_minutes: + typeof authSettings.lockoutDurationMinutes === 'number' + ? authSettings.lockoutDurationMinutes + : 15, }; } catch (error) { logger.error('Failed to get auth settings, using defaults:', error); diff --git a/backend/src/services/AuthSettingsService.ts b/backend/src/services/AuthSettingsService.ts index 2b5d8b6..8a191d4 100644 --- a/backend/src/services/AuthSettingsService.ts +++ b/backend/src/services/AuthSettingsService.ts @@ -57,7 +57,7 @@ export class AuthSettingsService { */ updateSettings(data: Partial, updatedBy: number): AuthSettings { // Filter out non-setting fields (updatedAt, updatedBy) - const { updatedAt: _updatedAt, updatedBy: _, ...settingsToUpdate } = data; + const { updatedAt: _updatedAt, updatedBy: _updatedBy, ...settingsToUpdate } = data; if (Object.keys(settingsToUpdate).length === 0) { return this.getSettings(); diff --git a/backend/src/services/CachedSystemSettingsService.ts b/backend/src/services/CachedSystemSettingsService.ts index 0e6f021..c9efbae 100644 --- a/backend/src/services/CachedSystemSettingsService.ts +++ b/backend/src/services/CachedSystemSettingsService.ts @@ -50,7 +50,7 @@ export class CachedSystemSettingsService extends SystemSettingsService { this.cacheTTL = cacheTTL; // Start cache cleanup interval (every 5 minutes) - this.cleanupIntervalId = setInterval(() => this.cleanup(), 5 * 60 * 1000); + this.cleanupIntervalId = setInterval(() => this.cleanup(), 5 * 60 * 1000).unref(); } /** diff --git a/backend/src/services/CommunityPlaylistService.ts b/backend/src/services/CommunityPlaylistService.ts index 4b4b6de..b8979f9 100644 --- a/backend/src/services/CommunityPlaylistService.ts +++ b/backend/src/services/CommunityPlaylistService.ts @@ -10,10 +10,13 @@ const PLAYLISTS_WIKI_URL = 'https://flashpointarchive.org/datahub/Playlists'; /** * Allowed domains for community playlist downloads (SSRF protection) + * Consolidated list used by both service and route validation */ -const ALLOWED_DOWNLOAD_DOMAINS = [ +export const ALLOWED_DOWNLOAD_DOMAINS = [ 'flashpointarchive.org', 'www.flashpointarchive.org', + 'download.flashpointarchive.org', + 'fpfss.unstable.life', 'github.com', 'raw.githubusercontent.com', 'gist.githubusercontent.com', @@ -38,9 +41,17 @@ export interface CommunityPlaylistsResponse { lastFetched: string; } +export interface PlaylistData { + id: string; + title: string; + description?: string; + games: Array<{ gameId: string; [key: string]: unknown }>; + [key: string]: unknown; +} + export interface DownloadResult { success: boolean; - playlist?: any; + playlist?: PlaylistData; error?: string; conflict?: boolean; } @@ -212,7 +223,8 @@ export class CommunityPlaylistService { // Process all headers and tables in sequence $('h2, h3, table.wikitable').each((i, elem) => { - const tagName = (elem as any).name; + const tagName = elem.type === 'tag' ? elem.name : null; + if (!tagName) return; if (tagName === 'h2') { // This is a main category header @@ -312,26 +324,37 @@ export class CommunityPlaylistService { /** * Validate that the downloaded playlist has the required structure */ - private validatePlaylistStructure(data: any): boolean { + private validatePlaylistStructure(data: unknown): boolean { + if (!data || typeof data !== 'object') return false; + const obj = data as Record; + // UUID validation regex for path traversal protection + const UUID_REGEX = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; + // Must have required fields - if (!data.id || typeof data.id !== 'string') { + if (!obj.id || typeof obj.id !== 'string') { logger.warn('[CommunityPlaylist] Validation failed: missing or invalid id'); return false; } - if (!data.title || typeof data.title !== 'string') { + // Validate ID format to prevent path traversal + if (!UUID_REGEX.test(obj.id)) { + logger.warn('[CommunityPlaylist] Validation failed: id is not a valid UUID format'); + return false; + } + + if (!obj.title || typeof obj.title !== 'string') { logger.warn('[CommunityPlaylist] Validation failed: missing or invalid title'); return false; } // Games array must exist and be valid - if (!Array.isArray(data.games)) { + if (!Array.isArray(obj.games)) { logger.warn('[CommunityPlaylist] Validation failed: games is not an array'); return false; } // Each game entry must have gameId - for (const game of data.games) { + for (const game of obj.games) { if (typeof game !== 'object' || !game.gameId) { logger.warn('[CommunityPlaylist] Validation failed: game entry missing gameId'); return false; diff --git a/backend/src/services/DatabaseService.ts b/backend/src/services/DatabaseService.ts index e15f9b8..24ff90d 100644 --- a/backend/src/services/DatabaseService.ts +++ b/backend/src/services/DatabaseService.ts @@ -5,19 +5,24 @@ import { measureQueryPerformance } from '../utils/queryPerformance'; import fs from 'fs'; import path from 'path'; -// Import GameSearchCache for cache invalidation +// Import GameSearchCache and GameService for cache invalidation on DB reload // Using dynamic import to avoid circular dependency -// Type is defined here to maintain type safety while avoiding circular import interface GameSearchCacheType { clearCache: () => void; } +interface GameServiceType { + clearFlashSwfCache: () => void; +} let GameSearchCache: GameSearchCacheType | null = null; +let GameServiceStatic: GameServiceType | null = null; try { // eslint-disable-next-line @typescript-eslint/no-require-imports GameSearchCache = require('./GameSearchCache').GameSearchCache as GameSearchCacheType; + // eslint-disable-next-line @typescript-eslint/no-require-imports + GameServiceStatic = require('./GameService').GameService as GameServiceType; } catch { - // GameSearchCache may not be available in all contexts - logger.debug('GameSearchCache not available for cache invalidation'); + // May not be available in all contexts + logger.debug('GameSearchCache/GameService not available for cache invalidation'); } export class DatabaseService { @@ -310,30 +315,38 @@ export class DatabaseService { logger.info('Source database file changed, syncing...'); - // Close existing database connection - if (this.db) { - this.db.close(); - this.db = null; - } - // If using local copy, update it from source if (this.isUsingLocalCopy && config.enableLocalDbCopy) { await this.copyDatabaseToLocal(); } - // Reopen database connection - this.db = this.openDatabase(this.activeDbPath); + // Open new connection FIRST + const newDb = this.openDatabase(this.activeDbPath); + + // Atomic swap: replace old connection with new one + const oldDb = this.db; + this.db = newDb; + + // Close old connection after swap + if (oldDb) { + try { + oldDb.close(); + } catch (closeError) { + logger.warn('Failed to close old database connection:', closeError); + } + } // Update modification time this.lastModifiedTime = stats.mtimeMs; - // Invalidate game search cache since database changed + // Invalidate caches since database changed if (GameSearchCache) { GameSearchCache.clearCache(); - logger.info('Game search cache invalidated due to database reload'); } - - logger.info('Database connection reopened successfully'); + if (GameServiceStatic) { + GameServiceStatic.clearFlashSwfCache(); + } + logger.info('Database reloaded, caches invalidated'); } catch (error) { logger.error('Failed to sync and reload database:', error); // Try to re-initialize if sync failed @@ -369,26 +382,36 @@ export class DatabaseService { if (this.isUsingLocalCopy && config.enableLocalDbCopy) { logger.info('Force syncing database from source...'); - // Close existing connection - if (this.db) { - this.db.close(); - this.db = null; - } - // Copy from source await this.copyDatabaseToLocal(); - // Reopen connection - this.db = this.openDatabase(this.activeDbPath); + // Open new connection FIRST + const newDb = this.openDatabase(this.activeDbPath); + + // Atomic swap: replace old connection with new one + const oldDb = this.db; + this.db = newDb; + + // Close old connection after swap + if (oldDb) { + try { + oldDb.close(); + } catch (closeError) { + logger.warn('Failed to close old database connection:', closeError); + } + } // Update modification time const stats = fs.statSync(this.sourceDbPath); this.lastModifiedTime = stats.mtimeMs; - // Invalidate cache + // Invalidate caches if (GameSearchCache) { GameSearchCache.clearCache(); } + if (GameServiceStatic) { + GameServiceStatic.clearFlashSwfCache(); + } logger.info('Force sync completed'); } else { @@ -533,17 +556,11 @@ export class DatabaseService { } // Save database changes to disk - // Note: With better-sqlite3, changes are automatically written to disk + // Note: With better-sqlite3 in DELETE journal mode, changes are automatically written to disk // This method exists for compatibility but is essentially a no-op static save(): void { - try { - const db = this.getDatabase(); - // Force a checkpoint in WAL mode to ensure all changes are written - db.pragma('wal_checkpoint(TRUNCATE)'); - logger.info('Database changes flushed to disk'); - } catch (error) { - logger.error('Failed to save database:', error); - throw error; - } + // No action needed in DELETE journal mode + // Changes are automatically flushed to disk + logger.debug('Database save called (no-op in DELETE journal mode)'); } } diff --git a/backend/src/services/DomainService.ts b/backend/src/services/DomainService.ts index c85329b..f7fc872 100644 --- a/backend/src/services/DomainService.ts +++ b/backend/src/services/DomainService.ts @@ -85,6 +85,8 @@ export class DomainService { /** * Validate and normalize a hostname + * Note: Route-level Zod validation performs comprehensive checks. + * This method provides basic sanitization only. */ private validateHostname(hostname: string): string { const trimmed = hostname.trim().toLowerCase(); @@ -93,21 +95,6 @@ export class DomainService { throw new AppError(400, 'Hostname cannot be empty'); } - // Reject protocol prefixes - if (/^https?:\/\//i.test(trimmed)) { - throw new AppError(400, 'Hostname must not include a protocol (http:// or https://)'); - } - - // Reject path, query, or fragment - if (/[/?#]/.test(trimmed)) { - throw new AppError(400, 'Hostname must not include a path, query string, or fragment'); - } - - // Basic hostname validation: allow alphanumeric, dots, hyphens, colons (for port) - if (!/^[a-z0-9._-]+(:[0-9]+)?$/i.test(trimmed)) { - throw new AppError(400, 'Invalid hostname format'); - } - return trimmed; } diff --git a/backend/src/services/DownloadManager.ts b/backend/src/services/DownloadManager.ts index b445175..e0f17d4 100644 --- a/backend/src/services/DownloadManager.ts +++ b/backend/src/services/DownloadManager.ts @@ -75,10 +75,9 @@ export class DownloadManager { this.activeDownloads.set(gameDataId, controller); // Chain external abort signal if provided + const onExternalAbort = () => controller.abort(); if (abortSignal) { - abortSignal.addEventListener('abort', () => { - controller.abort(); - }); + abortSignal.addEventListener('abort', onExternalAbort); } try { @@ -215,6 +214,9 @@ export class DownloadManager { ); } finally { // Clean up + if (abortSignal) { + abortSignal.removeEventListener('abort', onExternalAbort); + } this.activeDownloads.delete(gameDataId); } } @@ -248,6 +250,18 @@ export class DownloadManager { const writer = fs.createWriteStream(destPath); return new Promise((resolve, reject) => { + const onAbort = () => { + response.data.destroy(); + writer.destroy(); + reject(new Error('Download cancelled')); + }; + + abortSignal.addEventListener('abort', onAbort); + + const cleanup = () => { + abortSignal.removeEventListener('abort', onAbort); + }; + response.data.on('data', (chunk: Buffer) => { downloadedBytes += chunk.length; onProgress?.(downloadedBytes, totalBytes); @@ -258,24 +272,21 @@ export class DownloadManager { }); writer.on('finish', () => { + cleanup(); resolve(); }); writer.on('error', (error) => { + cleanup(); reject(new Error(`Failed to write file: ${error.message}`)); }); response.data.on('error', (error: Error) => { + cleanup(); writer.destroy(); reject(new Error(`Download stream error: ${error.message}`)); }); - abortSignal.addEventListener('abort', () => { - response.data.destroy(); - writer.destroy(); - reject(new Error('Download cancelled')); - }); - response.data.pipe(writer); }); } diff --git a/backend/src/services/FavoritesService.ts b/backend/src/services/FavoritesService.ts index 0c56b4a..f431f91 100644 --- a/backend/src/services/FavoritesService.ts +++ b/backend/src/services/FavoritesService.ts @@ -136,13 +136,13 @@ export class FavoritesService { if (limit !== undefined) { // Validate limit is a positive integer - const safeLimit = Math.max(1, Math.min(parseInt(String(limit), 10) || 50, 1000)); + const safeLimit = Math.max(1, Math.min(limit ?? 50, 1000)); query += ` LIMIT ?`; params.push(safeLimit); if (offset !== undefined) { // Validate offset is a non-negative integer - const safeOffset = Math.max(0, parseInt(String(offset), 10) || 0); + const safeOffset = Math.max(0, offset ?? 0); query += ` OFFSET ?`; params.push(safeOffset); } @@ -220,10 +220,13 @@ export class FavoritesService { const gameIds = favorites.map((f) => f.game_id); const games = await this.gameService.getGamesByIds(gameIds); + // Build Map for O(1) lookup (was O(N²) with .find() in loop) + const gamesMap = new Map(games.map((g) => [g.id, g])); + // Merge game data with addedAt timestamps const favoriteGames: FavoriteGame[] = favorites .map((fav) => { - const game = games.find((g) => g.id === fav.game_id); + const game = gamesMap.get(fav.game_id); if (game) { return { ...game, diff --git a/backend/src/services/GameDataService.ts b/backend/src/services/GameDataService.ts index 10ecaec..c0e1559 100644 --- a/backend/src/services/GameDataService.ts +++ b/backend/src/services/GameDataService.ts @@ -1,9 +1,22 @@ import fs from 'fs/promises'; import path from 'path'; -import axios from 'axios'; import { logger } from '../utils/logger'; import { config } from '../config'; import { DatabaseService } from './DatabaseService'; +import { gameZipServer } from '../game/gamezipserver'; +import { GameDataDownloader } from '../game/services/GameDataDownloader'; + +/** + * Game data entry from the Flashpoint database + */ +interface GameDataEntry { + id: number; + gameId: string; + dateAdded: string; + sha256: string; + path: string | null; + presentOnDisk: number; +} /** * Service for managing game data ZIP files @@ -11,66 +24,117 @@ import { DatabaseService } from './DatabaseService'; */ export class GameDataService { /** - * Mount the ZIP file for a game if it has game data - * @param gameId The game's UUID - * @returns true if ZIP was mounted or already mounted, false if no ZIP found + * Get game data entry from the Flashpoint database */ - async mountGameZip(gameId: string): Promise { + private getGameDataEntry(gameId: string): GameDataEntry | null { + try { + const sql = ` + SELECT id, gameId, dateAdded, sha256, path, presentOnDisk + FROM game_data + WHERE gameId = ? + ORDER BY id DESC + LIMIT 1 + `; + + const result = DatabaseService.get(sql, [gameId]) as GameDataEntry | null; + return result; + } catch (error) { + logger.error(`[GameDataService] Error getting game data entry for ${gameId}:`, error); + return null; + } + } + + /** + * Mount the ZIP file for a game if it has game data. + * Calls gameZipServer directly (in-process, no HTTP). + * If the ZIP doesn't exist locally, triggers background download + * from the configured gameDataSources in preferences.json. + * + * @param gameId The game's UUID + * @returns { mounted: true } if ZIP is mounted and ready, + * { downloading: true } if a background download was started, + * { mounted: false, downloading: false } if no ZIP could be found + */ + async mountGameZip(gameId: string): Promise<{ mounted: boolean; downloading: boolean }> { try { const mountId = gameId; - - // Find ZIP file in Data/Games/ directory - // ZIP files are named: {gameId}-{timestamp}.zip const gamesPath = config.flashpointGamesPath; - const files = await fs.readdir(gamesPath); - // Look for ZIP file starting with gameId - const zipPattern = `${gameId}-`; - const zipFile = files.find((f) => f.startsWith(zipPattern) && f.endsWith('.zip')); + // Get game data entry from database to get dateAdded and sha256 + const gameDataEntry = this.getGameDataEntry(gameId); - if (!zipFile) { - logger.warn(`[GameDataService] No ZIP file found for game ${gameId}`); - return false; + // Build ZIP path - try direct path first when we have dateAdded + let zipPath: string | null = null; + + // Try direct path first when we have enough info (avoids expensive readdir) + if (gameDataEntry?.dateAdded) { + try { + const expectedFilename = GameDataDownloader.getFilename(gameId, gameDataEntry.dateAdded); + const expectedPath = path.join(gamesPath, expectedFilename); + await fs.access(expectedPath); + // File exists at expected path - use it directly + zipPath = expectedPath; + logger.debug(`[GameDataService] Found ZIP at expected path: ${expectedFilename}`); + } catch { + // Expected file not found, fall through to readdir + logger.debug(`[GameDataService] Expected ZIP not found, trying readdir fallback`); + } } - const zipPath = path.join(gamesPath, zipFile); - - logger.info( - `[GameDataService] Requesting game-service to mount ZIP for game ${gameId}: ${zipFile}` - ); - - // Call game-service API to mount the ZIP - const gameZipServerUrl = config.gameServiceGameZipUrl || 'http://localhost:22501'; - try { - const response = await axios.post( - `${gameZipServerUrl}/mount/${mountId}`, - { - zipPath: zipPath, - }, - { - timeout: 5000, - validateStatus: (status) => status === 200 || status === 409, // 409 = already mounted - } - ); - - if (response.status === 409) { - logger.debug(`[GameDataService] ZIP already mounted for game ${gameId}`); + // Only readdir as fallback when direct path doesn't exist + if (!zipPath) { + let files: string[]; + try { + files = await fs.readdir(gamesPath); + } catch (err) { + logger.warn(`[GameDataService] Cannot read games directory: ${gamesPath}`, err); + // Fall through to attempt download via gameZipServer + files = []; } + const zipPattern = `${gameId}-`; + const zipFile = files.find((f) => f.startsWith(zipPattern) && f.endsWith('.zip')); + if (zipFile) { + zipPath = path.join(gamesPath, zipFile); + } else if (gameDataEntry?.dateAdded) { + // Use GameDataDownloader.getFilename() to construct expected filename + const expectedFilename = GameDataDownloader.getFilename(gameId, gameDataEntry.dateAdded); + zipPath = path.join(gamesPath, expectedFilename); + logger.info( + `[GameDataService] ZIP not found locally, will request download: ${expectedFilename}` + ); + } else { + logger.warn(`[GameDataService] No ZIP file found and no game data entry for ${gameId}`); + return { mounted: false, downloading: false }; + } + } + + logger.info(`[GameDataService] Mounting ZIP for game ${gameId}`); + + // Direct call to gameZipServer (in-process, no HTTP) + const result = await gameZipServer.mountZip({ + id: mountId, + zipPath, + gameId: gameDataEntry?.gameId || gameId, + dateAdded: gameDataEntry?.dateAdded, + sha256: gameDataEntry?.sha256, + }); + + if (result.downloading) { + logger.info(`[GameDataService] Download started in background for game ${gameId}`); + return { mounted: false, downloading: true }; + } else if (result.success) { logger.info(`[GameDataService] ✓ ZIP mounted successfully for game ${gameId}`); - return true; - } catch (error) { - if (axios.isAxiosError(error) && error.response?.status === 409) { - // Already mounted - logger.debug(`[GameDataService] ZIP already mounted for game ${gameId}`); - return true; - } - logger.error(`[GameDataService] Failed to mount ZIP via game-service:`, error); - return false; + return { mounted: true, downloading: false }; + } else { + logger.warn( + `[GameDataService] Mount failed for game ${gameId} (status: ${result.statusCode})` + ); + return { mounted: false, downloading: false }; } } catch (error) { logger.error(`[GameDataService] Error mounting ZIP for game ${gameId}:`, error); - return false; + return { mounted: false, downloading: false }; } } @@ -81,26 +145,24 @@ export class GameDataService { async mountMultipleGameZips(activeDataIds: number[]): Promise { logger.info(`[GameDataService] Mounting ${activeDataIds.length} game ZIPs...`); - // Fetch game data for each activeDataId to get gameId - const gameDataPromises = activeDataIds.map(async (activeDataId) => { - const sql = 'SELECT gameId FROM game_data WHERE id = ?'; - const result = DatabaseService.get(sql, [activeDataId]); - return result ? { gameId: result.gameId, activeDataId } : null; - }); + if (activeDataIds.length === 0) return; - const gameDataResults = await Promise.allSettled(gameDataPromises); - const validGameData = gameDataResults - .filter( - (r): r is PromiseFulfilledResult<{ gameId: string; activeDataId: number } | null> => - r.status === 'fulfilled' && r.value !== null - ) - .map((r) => r.value!); + // Batch query: single IN clause instead of N individual queries + const placeholders = activeDataIds.map(() => '?').join(', '); + const rows = DatabaseService.all( + `SELECT id, gameId FROM game_data WHERE id IN (${placeholders})`, + activeDataIds + ) as Array<{ id: number; gameId: string }>; + + const validGameData = rows.map((r) => ({ gameId: r.gameId, activeDataId: r.id })); const results = await Promise.allSettled( validGameData.map(({ gameId }) => this.mountGameZip(gameId)) ); - const succeeded = results.filter((r) => r.status === 'fulfilled' && r.value).length; + const succeeded = results.filter( + (r) => r.status === 'fulfilled' && (r.value.mounted || r.value.downloading) + ).length; const failed = results.length - succeeded; logger.info( @@ -122,15 +184,20 @@ export class GameDataService { for (const file of files) { if (file.endsWith('.zip')) { - // Extract activeDataId from filename (format: {id}-{timestamp}.zip) - const match = file.match(/^(\d+)-\d+\.zip$/); + // Extract gameId from filename (format: {gameId}-{timestamp}.zip) + // gameId is a UUID, not a numeric ID + const match = file.match( + /^([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})-(\d+)\.zip$/i + ); if (match) { - const activeDataId = parseInt(match[1], 10); + const gameId = match[1]; const filePath = path.join(gamesPath, file); const stats = await fs.stat(filePath); + // Note: This returns gameId instead of activeDataId + // The return type would need to be updated if this is actually used zipFiles.push({ - activeDataId, + activeDataId: 0, // Placeholder - would need to query database filename: file, size: stats.size, }); diff --git a/backend/src/services/GameDatabaseUpdater.ts b/backend/src/services/GameDatabaseUpdater.ts index 3a9c2e0..febe2cb 100644 --- a/backend/src/services/GameDatabaseUpdater.ts +++ b/backend/src/services/GameDatabaseUpdater.ts @@ -16,9 +16,33 @@ export interface GameData { parameters: string | null; } +/** + * Internal interface for database rows returned by SQL queries. + */ +interface GameDataRow { + id: number; + gameId: string; + title: string; + dateAdded: string; + sha256: string; + crc32: number; + presentOnDisk: number; + path: string | null; + size: number; + parameters: string | null; +} + /** * Service for updating the Flashpoint database after game data downloads. - * Handles transaction safety and proper updates to both game_data and game tables. + * + * NOTE: This service intentionally writes to flashpoint.sqlite to update + * presentOnDisk and path fields in game_data/game tables. This matches + * the Flashpoint Launcher's behavior when downloading game data. + * better-sqlite3 handles file locking, and the backend's DatabaseService + * file watcher handles reloads when the Launcher makes changes. + * + * All write operations are wrapped in transactions for atomicity to prevent + * database inconsistencies if any step fails. */ export class GameDatabaseUpdater { /** @@ -53,55 +77,64 @@ export class GameDatabaseUpdater { relativePath, }); - // Update game_data table - const updateGameDataSql = ` - UPDATE game_data - SET presentOnDisk = 1, path = ? - WHERE id = ? - `; - - logger.debug('Executing UPDATE on game_data', { - sql: updateGameDataSql, - params: [relativePath, gameDataId], - }); - - DatabaseService.run(updateGameDataSql, [relativePath, gameDataId]); - - logger.debug('game_data UPDATE successful'); - - // Check if this game_data is the active data for the game - const gameSql = ` - SELECT id, activeDataId - FROM game - WHERE id = ? - `; - const game = DatabaseService.get(gameSql, [gameData.gameId]); - - if (game && game.activeDataId === gameDataId) { - // Update game table to mark active data as on disk - const updateGameSql = ` - UPDATE game - SET activeDataOnDisk = 1 - WHERE id = ? AND activeDataId = ? + // Wrap both UPDATE statements in a transaction for atomicity + const db = DatabaseService.getDatabase(); + const updateTransaction = db.transaction(() => { + // Update game_data table + const updateGameDataSql = ` + UPDATE game_data + SET presentOnDisk = 1, path = ? + WHERE id = ? `; - logger.debug('Executing UPDATE on game table', { - sql: updateGameSql, - params: [gameData.gameId, gameDataId], + logger.debug('Executing UPDATE on game_data', { + sql: updateGameDataSql, + params: [relativePath, gameDataId], }); - DatabaseService.run(updateGameSql, [gameData.gameId, gameDataId]); + db.prepare(updateGameDataSql).run(relativePath, gameDataId); - logger.debug('game table UPDATE successful - activeDataOnDisk set to 1', { - gameId: gameData.gameId, - }); - } else { - logger.debug('Skipping game table update - game data is not active', { - gameId: gameData.gameId, - activeDataId: game?.activeDataId, - requestedDataId: gameDataId, - }); - } + logger.debug('game_data UPDATE successful'); + + // Check if this game_data is the active data for the game + const gameSql = ` + SELECT id, activeDataId + FROM game + WHERE id = ? + `; + const game = db.prepare(gameSql).get(gameData.gameId) as + | { id: string; activeDataId: number } + | undefined; + + if (game && game.activeDataId === gameDataId) { + // Update game table to mark active data as on disk + const updateGameSql = ` + UPDATE game + SET activeDataOnDisk = 1 + WHERE id = ? AND activeDataId = ? + `; + + logger.debug('Executing UPDATE on game table', { + sql: updateGameSql, + params: [gameData.gameId, gameDataId], + }); + + db.prepare(updateGameSql).run(gameData.gameId, gameDataId); + + logger.debug('game table UPDATE successful - activeDataOnDisk set to 1', { + gameId: gameData.gameId, + }); + } else { + logger.debug('Skipping game table update - game data is not active', { + gameId: gameData.gameId, + activeDataId: game?.activeDataId, + requestedDataId: gameDataId, + }); + } + }); + + // Execute the transaction + updateTransaction(); // Save database to disk await this.saveDatabase(); @@ -191,7 +224,7 @@ export class GameDatabaseUpdater { /** * Map database row to GameData object. */ - private static mapRowToGameData(row: any): GameData { + private static mapRowToGameData(row: GameDataRow): GameData { return { id: row.id, gameId: row.gameId, @@ -230,18 +263,28 @@ export class GameDatabaseUpdater { /** * Mark game data as not downloaded (for cleanup/error scenarios). + * Wrapped in transaction for consistency with markAsDownloaded. * * @param gameDataId - The game_data.id */ static async markAsNotDownloaded(gameDataId: number): Promise { try { - const sql = ` - UPDATE game_data - SET presentOnDisk = 0, path = NULL - WHERE id = ? - `; + const db = DatabaseService.getDatabase(); + + // Wrap UPDATE in transaction for atomicity + const updateTransaction = db.transaction(() => { + const sql = ` + UPDATE game_data + SET presentOnDisk = 0, path = NULL + WHERE id = ? + `; + + db.prepare(sql).run(gameDataId); + }); + + // Execute the transaction + updateTransaction(); - DatabaseService.run(sql, [gameDataId]); await this.saveDatabase(); logger.info('Marked game data as not downloaded', { gameDataId }); diff --git a/backend/src/services/GameFileService.ts b/backend/src/services/GameFileService.ts deleted file mode 100644 index 07ed139..0000000 --- a/backend/src/services/GameFileService.ts +++ /dev/null @@ -1,239 +0,0 @@ -import axios from 'axios'; -import fs from 'fs/promises'; -import path from 'path'; -import http from 'http'; -import { logger } from '../utils/logger'; -import { config } from '../config'; - -interface ProxySettings { - infinityServerURL?: string; - externalFilePaths?: string[]; -} - -/** - * Service to handle game file requests with automatic fallback to external archives - * This makes the web app independent of the Game Server's useInfinityServer setting - */ -export class GameFileService { - private infinityServerURL: string | null = null; - private externalFilePaths: string[] = []; - private gameServerUrl = 'http://localhost:22500'; - - constructor() { - this.loadProxySettings(); - } - - /** - * Load external fallback URLs from proxySettings.json - */ - private async loadProxySettings() { - try { - const proxySettingsPath = path.join(config.flashpointPath, 'Server', 'proxySettings.json'); - const data = await fs.readFile(proxySettingsPath, 'utf-8'); - const settings: ProxySettings = JSON.parse(data); - - // Use externalFilePaths directly - these have the correct HTTP URLs - this.externalFilePaths = settings.externalFilePaths || []; - - // Fix infinityServerURL if it uses HTTPS (should be HTTP) - if (settings.infinityServerURL) { - this.infinityServerURL = settings.infinityServerURL.replace(/^https:\/\//, 'http://'); - } - - logger.info(`[GameFileService] Loaded proxy settings`); - logger.info(`[GameFileService] Infinity Server: ${this.infinityServerURL}`); - logger.info(`[GameFileService] External paths: ${this.externalFilePaths.length} sources`); - - // Log all sources for debugging - this.externalFilePaths.forEach((path, i) => { - logger.info(`[GameFileService] [${i + 1}] ${path}`); - }); - } catch (error) { - logger.warn('[GameFileService] Could not load proxySettings.json, using defaults'); - // Use HTTP (not HTTPS) as per Flashpoint documentation - this.infinityServerURL = 'http://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs/'; - this.externalFilePaths = [ - 'http://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs', - 'http://infinity.unstable.life/Flashpoint/Legacy/htdocs/', - ]; - } - } - - /** - * Get a game file, with automatic fallback to external sources - * @param urlPath The path like "www.example.com/path/file.swf" - * @returns File buffer and content type - */ - async getGameFile( - urlPath: string - ): Promise<{ data: Buffer; contentType: string; source: string }> { - logger.info(`[GameFileService] ========================================`); - logger.info(`[GameFileService] Requested: ${urlPath}`); - - // First, try the local Game Server - logger.info(`[GameFileService] [1/4] Trying local Game Server (localhost:22500)...`); - try { - const result = await this.tryGameServer(urlPath); - if (result) { - logger.info(`[GameFileService] ✓ Found in Game Server!`); - return { ...result, source: 'game-server' }; - } - } catch (error) { - logger.warn(`[GameFileService] ✗ Game Server error: ${error}`); - } - - // If Game Server returns 404, try external sources - logger.info(`[GameFileService] File not found locally, trying external archives...`); - - // Try Infinity Server first - if (this.infinityServerURL) { - logger.info(`[GameFileService] [2/4] Trying Infinity Server...`); - const result = await this.tryExternalSource(this.infinityServerURL, urlPath); - if (result) { - logger.info(`[GameFileService] ✓ Found in Infinity Server!`); - return { ...result, source: 'infinity-server' }; - } - } - - // Try other external paths - let attemptNum = 3; - for (const externalPath of this.externalFilePaths) { - logger.info( - `[GameFileService] [${attemptNum}/${3 + this.externalFilePaths.length}] Trying: ${externalPath}` - ); - const result = await this.tryExternalSource(externalPath, urlPath); - if (result) { - logger.info(`[GameFileService] ✓ Found in external source!`); - return { ...result, source: externalPath }; - } - attemptNum++; - } - - logger.error( - `[GameFileService] ✗ File not found in any source (tried ${2 + this.externalFilePaths.length} sources)` - ); - throw new Error('File not found in Game Server or any external source'); - } - - /** - * Try to get file from the local Game Server - */ - private async tryGameServer( - urlPath: string - ): Promise<{ data: Buffer; contentType: string } | null> { - return new Promise((resolve, reject) => { - // Make proxy-style HTTP request - const fullUrl = `http://${urlPath}`; - const parsedUrl = new URL(fullUrl); - - const options: http.RequestOptions = { - hostname: 'localhost', - port: 22500, - path: fullUrl, - method: 'GET', - headers: { - Host: parsedUrl.hostname, - 'User-Agent': 'Flashpoint-WebApp/1.0', - }, - timeout: 10000, - }; - - const req = http.request(options, (res) => { - if (res.statusCode === 404) { - resolve(null); // Not found, will try external sources - return; - } - - if (res.statusCode !== 200) { - reject(new Error(`HTTP ${res.statusCode}`)); - return; - } - - const chunks: Buffer[] = []; - res.on('data', (chunk) => chunks.push(chunk)); - res.on('end', () => { - const data = Buffer.concat(chunks); - const contentType = res.headers['content-type'] || 'application/octet-stream'; - resolve({ data, contentType }); - }); - }); - - req.on('error', reject); - req.on('timeout', () => { - req.destroy(); - reject(new Error('Timeout')); - }); - - req.end(); - }); - } - - /** - * Try to download file from an external source - */ - private async tryExternalSource( - baseUrl: string, - urlPath: string - ): Promise<{ data: Buffer; contentType: string } | null> { - // Construct full URL - // baseUrl might be: http://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs/ - // urlPath is: www.example.com/path/file.swf - // Result: http://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs/www.example.com/path/file.swf - // - // Special case for archive.org ZIP files: - // baseUrl: http://archive.org/download/FP90Data/FP90Data.zip/htdocs - // Result: http://archive.org/download/FP90Data/FP90Data.zip/htdocs/www.example.com/path/file.swf - - const normalizedBase = baseUrl.endsWith('/') ? baseUrl : `${baseUrl}/`; - const fullUrl = `${normalizedBase}${urlPath}`; - - logger.info(`[GameFileService] Downloading: ${fullUrl}`); - - try { - const response = await axios.get(fullUrl, { - responseType: 'arraybuffer', - timeout: 45000, // Increased timeout for slow external servers - maxRedirects: 5, - validateStatus: (status) => status === 200, - headers: { - 'User-Agent': 'Flashpoint-WebApp/1.0', - }, - }); - - const data = Buffer.from(response.data); - const contentType = response.headers['content-type'] || 'application/octet-stream'; - - logger.info(`[GameFileService] ✓ Success: ${data.length} bytes (${contentType})`); - - // TODO: Optionally cache the file locally for future requests - - return { data, contentType }; - } catch (error) { - // Log detailed error but return null to try next source - if (axios.isAxiosError(error)) { - if (error.response?.status === 404) { - logger.warn(`[GameFileService] ✗ Not found (404)`); - } else if (error.response?.status === 403) { - logger.warn(`[GameFileService] ✗ Forbidden (403) - access denied`); - } else if (error.code === 'ECONNABORTED' || error.code === 'ETIMEDOUT') { - logger.warn(`[GameFileService] ✗ Timeout after 45s`); - } else if (error.code === 'ENOTFOUND') { - logger.warn(`[GameFileService] ✗ DNS resolution failed`); - } else if (error.response) { - logger.warn( - `[GameFileService] ✗ HTTP ${error.response.status}: ${error.response.statusText}` - ); - } else if (error.request) { - logger.warn(`[GameFileService] ✗ No response received: ${error.message}`); - } else { - logger.warn(`[GameFileService] ✗ Network error: ${error.message}`); - } - } else { - logger.warn(`[GameFileService] ✗ Unexpected error: ${error}`); - } - - // Always return null instead of throwing, so we can try the next source - return null; - } - } -} diff --git a/backend/src/services/GameSearchCache.ts b/backend/src/services/GameSearchCache.ts index 6d4618b..4e56876 100644 --- a/backend/src/services/GameSearchCache.ts +++ b/backend/src/services/GameSearchCache.ts @@ -53,14 +53,14 @@ export class GameSearchCache { private static generateCacheKey(query: GameSearchQuery): string { const normalizedQuery = { search: query.search || '', - platforms: query.platforms?.sort().join(',') || '', - series: query.series?.sort().join(',') || '', - developers: query.developers?.sort().join(',') || '', - publishers: query.publishers?.sort().join(',') || '', - playModes: query.playModes?.sort().join(',') || '', - languages: query.languages?.sort().join(',') || '', + platforms: query.platforms ? [...query.platforms].sort().join(',') : '', + series: query.series ? [...query.series].sort().join(',') : '', + developers: query.developers ? [...query.developers].sort().join(',') : '', + publishers: query.publishers ? [...query.publishers].sort().join(',') : '', + playModes: query.playModes ? [...query.playModes].sort().join(',') : '', + languages: query.languages ? [...query.languages].sort().join(',') : '', library: query.library || '', - tags: query.tags?.sort().join(',') || '', + tags: query.tags ? [...query.tags].sort().join(',') : '', yearFrom: query.yearFrom || '', yearTo: query.yearTo || '', dateAddedSince: query.dateAddedSince || '', diff --git a/backend/src/services/GameService.ts b/backend/src/services/GameService.ts index adfe309..b965a87 100644 --- a/backend/src/services/GameService.ts +++ b/backend/src/services/GameService.ts @@ -58,6 +58,8 @@ export interface Game { playtime?: number; playCounter?: number; archiveState?: number; + activeDataId?: number; // ID of the active game_data entry (null/0 = no game data) + activeDataOnDisk?: number; // Whether the active data is downloaded (0 = no, 1 = yes) logoPath?: string; // Infinity edition only screenshotPath?: string; // Infinity edition only } @@ -71,6 +73,77 @@ export interface PaginatedResult { } export class GameService { + /** + * Cached set of Flash game IDs that have .swf launch commands in game_data. + * Pre-computed once and cached with TTL to avoid per-row EXISTS subqueries + * which would be O(N*M) on every search (N=games, M=game_data rows). + * + * Since flashpoint.sqlite is read-only, this cache only needs to refresh + * when the database file is reloaded (detected via DatabaseService file watcher). + */ + private static flashSwfGameIds: Set | null = null; + private static flashSwfCacheExpiry = 0; + private static readonly FLASH_SWF_CACHE_TTL = 3600000; // 1 hour + + /** + * Get the set of Flash game IDs whose game_data.launchCommand points to a .swf file. + * Results are cached in memory for 1 hour (read-only DB changes rarely). + */ + private getFlashSwfGameIds(): Set { + const now = Date.now(); + if (GameService.flashSwfGameIds && now < GameService.flashSwfCacheExpiry) { + return GameService.flashSwfGameIds; + } + + const startTime = performance.now(); + const sql = ` + SELECT DISTINCT gd.gameId + FROM game_data gd + JOIN game g ON g.id = gd.gameId + WHERE g.platformName = 'Flash' + AND (LOWER(gd.launchCommand) LIKE '%.swf' OR LOWER(gd.launchCommand) LIKE '%.swf?%') + `; + + const rows = DatabaseService.all(sql, []) as Array<{ gameId: string }>; + GameService.flashSwfGameIds = new Set(rows.map((r) => r.gameId)); + GameService.flashSwfCacheExpiry = now + GameService.FLASH_SWF_CACHE_TTL; + + const duration = Math.round(performance.now() - startTime); + logger.info( + `[GameService] Cached ${GameService.flashSwfGameIds.size} Flash SWF game IDs in ${duration}ms` + ); + return GameService.flashSwfGameIds; + } + + /** + * Clear the Flash SWF game IDs cache. + * Call when the flashpoint.sqlite database is reloaded. + */ + static clearFlashSwfCache(): void { + GameService.flashSwfGameIds = null; + GameService.flashSwfCacheExpiry = 0; + logger.debug('[GameService] Flash SWF game IDs cache cleared'); + } + + /** + * SQL condition to exclude Flash games without SWF launch commands. + * Uses EXISTS subquery against game_data table (leverages UNIQUE(gameId, dateAdded) index). + * + * Performance: ~150ms overhead per query. Acceptable for searchGames() since results + * are cached by GameSearchCache (5-min TTL). For aggregate queries like + * getPlatformOptions(), use getFlashSwfGameIds() cache instead. + */ + private getFlashSwfCondition(tableAlias: string = 'g'): string { + return `( + ${tableAlias}.platformName != 'Flash' + OR EXISTS ( + SELECT 1 FROM game_data gd + WHERE gd.gameId = ${tableAlias}.id + AND (LOWER(gd.launchCommand) LIKE '%.swf' OR LOWER(gd.launchCommand) LIKE '%.swf?%') + ) + )`; + } + /** * Check if running Infinity edition (has logoPath/screenshotPath columns) */ @@ -86,11 +159,13 @@ export class GameService { const isInfinity = this.isInfinityEdition(); if (fields === 'list') { - // Minimal columns for list views (12 essential fields) + // Minimal columns for list views (7 essential fields) + // GameCard uses: id, title, developer, platformName + // GameListItem uses: id, title, developer, platformName, tagsStr + // library and orderTitle needed for filtering/sorting return ` - g.id, g.title, g.developer, g.publisher, g.platformName, - g.library, g.orderTitle, g.releaseDate, g.tagsStr, - g.broken, g.extreme, g.series + g.id, g.title, g.developer, g.platformName, + g.tagsStr, g.library, g.orderTitle `; } @@ -102,7 +177,8 @@ export class GameService { g.source, g.applicationPath, g.launchCommand, g.releaseDate, g.version, g.originalDescription, g.language, g.library, g.orderTitle, g.dateAdded, g.dateModified, - g.lastPlayed, g.playtime, g.playCounter, g.archiveState`; + g.lastPlayed, g.playtime, g.playCounter, g.archiveState, + g.activeDataId, g.activeDataOnDisk`; // Infinity edition has logoPath and screenshotPath columns if (isInfinity) { @@ -133,7 +209,7 @@ export class GameService { WHERE 1=1 `; - const params: any[] = []; + const params: unknown[] = []; // Apply filters if (query.platforms && query.platforms.length > 0) { @@ -223,6 +299,9 @@ export class GameService { sql += ` AND (g.extreme = 0 OR g.extreme IS NULL)`; } + // Exclude Flash games without SWF launch commands (not web-playable) + sql += ` AND ${this.getFlashSwfCondition('g')}`; + if (query.search) { sql += ` AND ( g.title LIKE ? OR @@ -314,14 +393,11 @@ export class GameService { async getGameById(id: string): Promise { try { const columns = this.getColumnSelection('detail'); + // No need for LEFT JOIN - activeDataId and activeDataOnDisk are on game table const sql = ` - SELECT - ${columns}, - MAX(gd.presentOnDisk) as presentOnDisk + SELECT ${columns} FROM game g - LEFT JOIN game_data gd ON gd.gameId = g.id WHERE g.id = ? - GROUP BY g.id `; const game = DatabaseService.get(sql, [id]) as Game | null; @@ -344,18 +420,35 @@ export class GameService { const columns = this.getColumnSelection('detail'); const placeholders = ids.map(() => '?').join(', '); + + // Query WITHOUT LEFT JOIN to avoid row explosion from multiple game_data entries const sql = ` - SELECT - ${columns}, - MAX(gd.presentOnDisk) as presentOnDisk + SELECT ${columns} FROM game g - LEFT JOIN game_data gd ON gd.gameId = g.id WHERE g.id IN (${placeholders}) - GROUP BY g.id `; const games = DatabaseService.all(sql, ids) as Game[]; + // Fetch presentOnDisk separately (same post-processing pattern as searchGames) + if (games.length > 0) { + const gameIds = games.map((g) => g.id); + const pdPlaceholders = gameIds.map(() => '?').join(', '); + const presentOnDiskResults = DatabaseService.all( + `SELECT gameId, MAX(presentOnDisk) as presentOnDisk + FROM game_data WHERE gameId IN (${pdPlaceholders}) GROUP BY gameId`, + gameIds + ) as Array<{ gameId: string; presentOnDisk: number }>; + + const presentOnDiskMap = new Map( + presentOnDiskResults.map((r) => [r.gameId, r.presentOnDisk]) + ); + + games.forEach((game) => { + game.presentOnDisk = presentOnDiskMap.get(game.id) ?? 0; + }); + } + // Sort games to match the order of input IDs const gamesMap = new Map(games.map((g) => [g.id, g])); return ids.map((id) => gamesMap.get(id)).filter((g): g is Game => g !== undefined); @@ -375,7 +468,7 @@ export class GameService { SELECT id FROM game_data WHERE gameId = ? - ORDER BY presentOnDisk DESC, id DESC + ORDER BY id DESC LIMIT 1 `; @@ -393,7 +486,7 @@ export class GameService { SELECT launchCommand, path, parameters FROM game_data WHERE gameId = ? - ORDER BY presentOnDisk DESC, id DESC + ORDER BY id DESC LIMIT 1 `; @@ -484,7 +577,7 @@ export class GameService { WHERE (g.broken = 0 OR g.broken IS NULL) `; - const params: any[] = []; + const params: unknown[] = []; if (library) { sql += ` AND g.library = ?`; @@ -515,6 +608,7 @@ export class GameService { /** * Get all filter options for dropdowns in a single call * Returns: series, developers, publishers, playModes, languages, tags, platforms, yearRange + * Note: Methods are synchronous but wrapped in async for consistent API contract */ async getFilterOptions(): Promise<{ series: Array<{ name: string; count: number }>; @@ -527,18 +621,15 @@ export class GameService { yearRange: { min: number; max: number }; }> { try { - // Fetch all filter options in parallel for performance - const [series, developers, publishers, playModes, languages, tags, platforms, yearRange] = - await Promise.all([ - this.getSeriesOptions(), - this.getDeveloperOptions(), - this.getPublisherOptions(), - this.getPlayModeOptions(), - this.getLanguageOptions(), - this.getTagOptions(), - this.getPlatformOptions(), - this.getYearRange(), - ]); + // Call synchronous methods directly (no Promise.all needed) + const series = this.getSeriesOptions(); + const developers = this.getDeveloperOptions(); + const publishers = this.getPublisherOptions(); + const playModes = this.getPlayModeOptions(); + const languages = this.getLanguageOptions(); + const tags = this.getTagOptions(); + const platforms = this.getPlatformOptions(); + const yearRange = this.getYearRange(); return { series, @@ -728,8 +819,9 @@ export class GameService { */ getPlatformOptions(): Array<{ name: string; count: number }> { try { + // Query all platforms without Flash SWF filter (fast, no EXISTS) const sql = ` - SELECT DISTINCT platformName as name, COUNT(*) as count + SELECT platformName as name, COUNT(*) as count FROM game WHERE platformName IS NOT NULL AND platformName != '' AND (broken = 0 OR broken IS NULL) @@ -739,6 +831,14 @@ export class GameService { `; const results = DatabaseService.all(sql, []) as Array<{ name: string; count: number }>; + + // Adjust Flash count using cached SWF game IDs (avoids slow EXISTS subquery) + const flashSwfIds = this.getFlashSwfGameIds(); + const flashIndex = results.findIndex((r) => r.name === 'Flash'); + if (flashIndex !== -1) { + results[flashIndex].count = flashSwfIds.size; + } + return results; } catch (error) { logger.error('Error getting platform options:', error); diff --git a/backend/src/services/ImageUrlService.ts b/backend/src/services/ImageUrlService.ts index ed8e33d..183b7d1 100644 --- a/backend/src/services/ImageUrlService.ts +++ b/backend/src/services/ImageUrlService.ts @@ -23,7 +23,7 @@ export class ImageUrlService { 'https://infinity.unstable.life/Flashpoint/Data/Images', // Secondary fallback ]; - logger.info('Using external image URLs from Flashpoint preferences', { + logger.debug('Using external image URLs from Flashpoint preferences', { baseUrl: preferences.onDemandBaseUrl, constructedUrls: urls, }); @@ -42,7 +42,7 @@ export class ImageUrlService { 'https://infinity.unstable.life/Flashpoint/Data/Images', ]; - logger.info('Using default image URLs', { urls: defaultUrls }); + logger.debug('Using default image URLs', { urls: defaultUrls }); return defaultUrls; } @@ -72,5 +72,3 @@ export class ImageUrlService { } } } - -export default ImageUrlService; diff --git a/backend/src/services/JobExecutionService.ts b/backend/src/services/JobExecutionService.ts index d2e604f..07d0dc6 100644 --- a/backend/src/services/JobExecutionService.ts +++ b/backend/src/services/JobExecutionService.ts @@ -1,6 +1,19 @@ import { UserDatabaseService } from './UserDatabaseService'; import { logger } from '../utils/logger'; +interface JobExecutionRow { + id: number; + job_id: string; + job_name: string; + status: string; + started_at: string; + completed_at: string | null; + duration_seconds: number | null; + message: string | null; + error_details: string | null; + triggered_by: string; +} + export interface JobExecutionLog { id: number; jobId: string; @@ -79,7 +92,7 @@ export class JobExecutionService { ORDER BY started_at DESC LIMIT ? OFFSET ?`, [jobId, limit, offset] - ) as any[]; + ) as JobExecutionRow[]; return { data: logs.map(this.mapToJobExecutionLog), @@ -101,7 +114,7 @@ export class JobExecutionService { ORDER BY started_at DESC LIMIT ? OFFSET ?`, [limit, offset] - ) as any[]; + ) as JobExecutionRow[]; return { data: logs.map(this.mapToJobExecutionLog), @@ -119,7 +132,7 @@ export class JobExecutionService { ORDER BY started_at DESC LIMIT 1`, [jobId] - ) as any; + ) as JobExecutionRow | undefined; return log ? this.mapToJobExecutionLog(log) : null; } @@ -140,17 +153,17 @@ export class JobExecutionService { return result.changes; } - private mapToJobExecutionLog(row: any): JobExecutionLog { + private mapToJobExecutionLog(row: JobExecutionRow): JobExecutionLog { return { id: row.id, jobId: row.job_id, jobName: row.job_name, - status: row.status, + status: row.status as 'running' | 'success' | 'failed', startedAt: row.started_at, - completedAt: row.completed_at, - durationSeconds: row.duration_seconds, - message: row.message, - errorDetails: row.error_details, + completedAt: row.completed_at || undefined, + durationSeconds: row.duration_seconds || undefined, + message: row.message || undefined, + errorDetails: row.error_details || undefined, triggeredBy: row.triggered_by, }; } diff --git a/backend/src/services/JobScheduler.ts b/backend/src/services/JobScheduler.ts index 0f7142d..8e2873a 100644 --- a/backend/src/services/JobScheduler.ts +++ b/backend/src/services/JobScheduler.ts @@ -124,7 +124,9 @@ class JobSchedulerService { // Run asynchronously (don't await) // runJobWithLogging will handle execution logging - this.runJobWithLogging(jobId, triggeredBy); + this.runJobWithLogging(jobId, triggeredBy).catch((error) => { + logger.error(`[JobScheduler] Unexpected error in triggered job ${jobId}:`, error); + }); } /** diff --git a/backend/src/services/MetadataSyncService.ts b/backend/src/services/MetadataSyncService.ts index 5bf3935..0f08314 100644 --- a/backend/src/services/MetadataSyncService.ts +++ b/backend/src/services/MetadataSyncService.ts @@ -479,7 +479,7 @@ export class MetadataSyncService { ) .join(', '); - const values: any[] = []; + const values: (string | number | null)[] = []; sortedGames.forEach((game) => { values.push( game.id, @@ -553,7 +553,13 @@ export class MetadataSyncService { ruffleSupport = excluded.ruffleSupport `; - await DatabaseService.run(sql, values); + // Wrap in a transaction for atomicity - a partial upsert would leave the + // database in an inconsistent state (some games updated, others not). + const db = DatabaseService.getDatabase(); + db.transaction(() => { + const stmt = db.prepare(sql); + stmt.run(values); + })(); logger.debug(`[MetadataSync] Upserted ${sortedGames.length} games`); } diff --git a/backend/src/services/PerformanceMetrics.ts b/backend/src/services/PerformanceMetrics.ts index e5bc5d5..0ad9499 100644 --- a/backend/src/services/PerformanceMetrics.ts +++ b/backend/src/services/PerformanceMetrics.ts @@ -41,17 +41,39 @@ export class PerformanceMetrics { // Configuration private static readonly MAX_DATA_POINTS = 10000; // Keep last 10k measurements private static readonly RETENTION_MS = 24 * 60 * 60 * 1000; // 24 hours + private static readonly MAX_UNIQUE_ENDPOINTS = 1000; // Prevent unbounded memory growth + + /** + * Normalize path by replacing dynamic segments with :id + * Prevents memory growth from parametric routes like /api/games/:id + */ + private static normalizePath(path: string): string { + // Replace UUID segments: /api/games/550e8400-e29b-41d4-a716-446655440000 -> /api/games/:id + return ( + path + .replace(/\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi, '/:id') + // Replace numeric IDs: /api/users/123 -> /api/users/:id + .replace(/\/\d+(?=\/|$)/g, '/:id') + ); + } /** * Record API endpoint response time */ static recordEndpoint(path: string, method: string, durationMs: number): void { - const key = `${method} ${path}`; + const normalizedPath = this.normalizePath(path); + const key = `${method} ${normalizedPath}`; if (!this.endpointMetrics.has(key)) { this.endpointMetrics.set(key, []); } + // Evict oldest entry if we exceed max unique endpoints + if (this.endpointMetrics.size > this.MAX_UNIQUE_ENDPOINTS) { + const firstKey = this.endpointMetrics.keys().next().value; + if (firstKey) this.endpointMetrics.delete(firstKey); + } + const metrics = this.endpointMetrics.get(key)!; metrics.push({ timestamp: Date.now(), diff --git a/backend/src/services/PermissionCache.ts b/backend/src/services/PermissionCache.ts index 29638e7..a9b6a94 100644 --- a/backend/src/services/PermissionCache.ts +++ b/backend/src/services/PermissionCache.ts @@ -135,16 +135,12 @@ export class PermissionCache { /** * Invalidate role permissions (when role permissions change) - * Also invalidates all users with this role + * Also invalidates all user permissions to force re-fetch */ static invalidateRole(roleId: number): void { this.rolePermissionsCache.delete(roleId); - logger.info(`[PermissionCache] Invalidated cache for role ${roleId}`); - - // Note: We don't invalidate all users with this role because we'd need - // to query the database to find them, defeating the purpose of caching. - // Instead, we rely on the TTL to eventually update all users. - // For immediate updates, consider invalidating specific users or clearing all. + this.userPermissionsCache.clear(); + logger.info(`[PermissionCache] Invalidated cache for role ${roleId} and all user permissions`); } /** diff --git a/backend/src/services/PlayTrackingService.ts b/backend/src/services/PlayTrackingService.ts index 1da4c00..539b2c1 100644 --- a/backend/src/services/PlayTrackingService.ts +++ b/backend/src/services/PlayTrackingService.ts @@ -1,7 +1,8 @@ import { UserDatabaseService } from './UserDatabaseService'; import { DatabaseService } from './DatabaseService'; import { logger } from '../utils/logger'; -import { randomBytes } from 'crypto'; +import { randomUUID } from 'crypto'; +import { AppError } from '../middleware/errorHandler'; import { PlaySessionRow, GameStatsRow, @@ -44,8 +45,8 @@ export class PlayTrackingService { */ async startPlaySession(userId: number, gameId: string, gameTitle: string): Promise { try { - // Generate unique session ID - const sessionId = randomBytes(16).toString('hex'); + // Generate unique session ID (UUID format) + const sessionId = randomUUID(); // Insert play session UserDatabaseService.run( @@ -68,7 +69,7 @@ export class PlayTrackingService { * End a play session * OPTIMIZED: Single query to get session data and calculate duration, then single UPDATE */ - async endPlaySession(sessionId: string): Promise { + async endPlaySession(sessionId: string, userId?: number): Promise { try { // OPTIMIZATION: Get session and calculate duration in single query const session = UserDatabaseService.get( @@ -87,6 +88,11 @@ export class PlayTrackingService { return; } + // Verify session ownership if userId provided + if (userId && session.user_id !== userId) { + throw new AppError(403, "Cannot end another user's play session"); + } + const durationSeconds = session.duration_seconds || 0; // Use the duration value from the SELECT query to avoid race condition @@ -149,19 +155,16 @@ export class PlayTrackingService { */ private async updateUserStats(userId: number, durationSeconds: number): Promise { try { - // Count unique games played - const gamesPlayed = - UserDatabaseService.get( - 'SELECT COUNT(DISTINCT game_id) as count FROM user_game_stats WHERE user_id = ?', - [userId] - )?.count || 0; + // Single query to get both aggregates (was 2 separate queries) + const counts = UserDatabaseService.get( + `SELECT + (SELECT COUNT(DISTINCT game_id) FROM user_game_stats WHERE user_id = ?) as games_played, + (SELECT COUNT(*) FROM user_game_plays WHERE user_id = ? AND ended_at IS NOT NULL) as total_sessions`, + [userId, userId] + ) as { games_played: number; total_sessions: number } | undefined; - // Count total sessions - const totalSessions = - UserDatabaseService.get( - 'SELECT COUNT(*) as count FROM user_game_plays WHERE user_id = ? AND ended_at IS NOT NULL', - [userId] - )?.count || 0; + const gamesPlayed = counts?.games_played ?? 0; + const totalSessions = counts?.total_sessions ?? 0; // UPSERT user stats UserDatabaseService.run( @@ -395,9 +398,11 @@ export class PlayTrackingService { ); for (const session of abandoned) { - // Calculate estimated duration (capped at 4 hours) + // Calculate actual duration from session start time, capped at 4 hours + const startTime = new Date(session.started_at).getTime(); + const actualSeconds = Math.floor((Date.now() - startTime) / 1000); const maxDuration = 4 * 60 * 60; // 4 hours in seconds - const duration = Math.min(maxDuration, 86400); // 24 hours max + const duration = Math.min(actualSeconds, maxDuration); // End the session UserDatabaseService.run( diff --git a/backend/src/services/PlaylistService.ts b/backend/src/services/PlaylistService.ts index a90c490..9ebf507 100644 --- a/backend/src/services/PlaylistService.ts +++ b/backend/src/services/PlaylistService.ts @@ -5,6 +5,8 @@ import { config } from '../config'; import { logger } from '../utils/logger'; import { GameService, Game } from './GameService'; +type PlaylistGameEntry = string | { gameId?: string; [key: string]: unknown }; + export interface Playlist { id: string; title: string; @@ -35,36 +37,42 @@ export class PlaylistService { const playlistsPath = config.flashpointPlaylistsPath; const files = await fs.readdir(playlistsPath); + const jsonFiles = files.filter((f) => f.endsWith('.json')); + + // Parallel file reads (was sequential for-of loop) + const results = await Promise.allSettled( + jsonFiles.map(async (file) => { + const filePath = path.join(playlistsPath, file); + const content = await fs.readFile(filePath, 'utf-8'); + const playlist = JSON.parse(content); + + const gameIds = Array.isArray(playlist.games) + ? playlist.games + .map((g: unknown) => + typeof g === 'string' ? g : (g as { gameId?: string }).gameId + ) + .filter(Boolean) + : []; + + return { + id: playlist.id || path.basename(file, '.json'), + title: playlist.title, + description: playlist.description, + author: playlist.author, + library: playlist.library, + icon: playlist.icon, + gameIds, + } as Playlist; + }) + ); + const playlists: Playlist[] = []; - - for (const file of files) { - if (file.endsWith('.json')) { - try { - const filePath = path.join(playlistsPath, file); - const content = await fs.readFile(filePath, 'utf-8'); - const playlist = JSON.parse(content); - - // Extract game IDs from playlist (they're objects with gameId property) - const gameIds = Array.isArray(playlist.games) - ? playlist.games - .map((g: unknown) => - typeof g === 'string' ? g : (g as { gameId?: string }).gameId - ) - .filter(Boolean) - : []; - - playlists.push({ - id: playlist.id || path.basename(file, '.json'), - title: playlist.title, - description: playlist.description, - author: playlist.author, - library: playlist.library, - icon: playlist.icon, - gameIds, - }); - } catch (error) { - logger.warn(`Failed to parse playlist file: ${file}`, error); - } + for (let i = 0; i < results.length; i++) { + const result = results[i]; + if (result.status === 'fulfilled') { + playlists.push(result.value); + } else { + logger.warn(`Failed to parse playlist file: ${jsonFiles[i]}`, result.reason); } } @@ -78,48 +86,19 @@ export class PlaylistService { async getPlaylistById(id: string): Promise { try { const playlistsPath = config.flashpointPlaylistsPath; + + // Try direct file access first (O(1) instead of scanning all files) + const directPath = path.join(playlistsPath, `${id}.json`); + const playlist = await this.tryReadPlaylist(directPath, id); + if (playlist) return playlist; + + // Fallback: scan all files (some playlists may have filename != id) const files = await fs.readdir(playlistsPath); - - // Try to find playlist file by ID for (const file of files) { - if (file.endsWith('.json')) { - const filePath = path.join(playlistsPath, file); - const content = await fs.readFile(filePath, 'utf-8'); - const playlist = JSON.parse(content); - - const playlistId = playlist.id || path.basename(file, '.json'); - - if (playlistId === id) { - // Extract game IDs from playlist - const gameIds: string[] = []; - - if (playlist.games && Array.isArray(playlist.games)) { - for (const playlistGame of playlist.games) { - // Playlist games are objects with gameId property, not just strings - const gameId = - typeof playlistGame === 'string' ? playlistGame : playlistGame.gameId; - - if (gameId) { - gameIds.push(gameId); - } - } - } - - // Batch fetch all games at once (avoid N+1 query pattern) - // getGamesByIds returns games in the same order as input IDs - const games = await this.gameService.getGamesByIds(gameIds); - - return { - id: playlistId, - title: playlist.title, - description: playlist.description, - author: playlist.author, - library: playlist.library, - icon: playlist.icon, - games, - }; - } - } + if (!file.endsWith('.json') || file === `${id}.json`) continue; // Already tried direct + const filePath = path.join(playlistsPath, file); + const result = await this.tryReadPlaylist(filePath, id); + if (result) return result; } return null; @@ -129,6 +108,44 @@ export class PlaylistService { } } + /** + * Try to read a playlist file and return it if its ID matches + */ + private async tryReadPlaylist(filePath: string, targetId: string): Promise { + try { + const content = await fs.readFile(filePath, 'utf-8'); + const playlist = JSON.parse(content); + + const playlistId = playlist.id || path.basename(filePath, '.json'); + if (playlistId !== targetId) return null; + + // Extract game IDs from playlist + const gameIds: string[] = []; + if (playlist.games && Array.isArray(playlist.games)) { + for (const playlistGame of playlist.games) { + const gameId = typeof playlistGame === 'string' ? playlistGame : playlistGame.gameId; + if (gameId) gameIds.push(gameId); + } + } + + // Batch fetch all games at once (avoid N+1 query pattern) + const games = await this.gameService.getGamesByIds(gameIds); + + return { + id: playlistId, + title: playlist.title, + description: playlist.description, + author: playlist.author, + library: playlist.library, + icon: playlist.icon, + games, + }; + } catch { + // File doesn't exist or invalid JSON - return null + return null; + } + } + async createPlaylist(data: CreatePlaylistDto): Promise { try { const playlistsPath = config.flashpointPlaylistsPath; @@ -163,44 +180,25 @@ export class PlaylistService { data: AddGamesToPlaylistDto ): Promise { try { - const playlistsPath = config.flashpointPlaylistsPath; - const files = await fs.readdir(playlistsPath); + const found = await this.findPlaylistFile(playlistId); + if (!found) return null; - // Find playlist file - for (const file of files) { - if (file.endsWith('.json')) { - const filePath = path.join(playlistsPath, file); - const content = await fs.readFile(filePath, 'utf-8'); - const playlist = JSON.parse(content); + const { filePath, data: playlist } = found; - const id = playlist.id || path.basename(file, '.json'); + // Get existing game IDs + const existingGameIds = Array.isArray(playlist.games) + ? playlist.games.map((g: PlaylistGameEntry) => (typeof g === 'string' ? g : g.gameId)) + : []; - if (id === playlistId) { - // Get existing game IDs - const existingGameIds = Array.isArray(playlist.games) - ? playlist.games.map((g: any) => (typeof g === 'string' ? g : g.gameId)) - : []; + // Add new game IDs (avoid duplicates) + const newGameIds = data.gameIds.filter((gameId) => !existingGameIds.includes(gameId)); + const newGames = newGameIds.map((gameId) => ({ gameId })); + playlist.games = [...(playlist.games || []), ...newGames]; - // Add new game IDs (avoid duplicates) - const newGameIds = data.gameIds.filter((gameId) => !existingGameIds.includes(gameId)); + await fs.writeFile(filePath, JSON.stringify(playlist, null, '\t'), 'utf-8'); + logger.info(`Added ${newGameIds.length} games to playlist: ${playlist.title}`); - // Convert to playlist game format - const newGames = newGameIds.map((gameId) => ({ gameId })); - - playlist.games = [...(playlist.games || []), ...newGames]; - - // Write updated playlist - await fs.writeFile(filePath, JSON.stringify(playlist, null, '\t'), 'utf-8'); - - logger.info(`Added ${newGameIds.length} games to playlist: ${playlist.title}`); - - // Return updated playlist with full game data - return await this.getPlaylistById(playlistId); - } - } - } - - return null; + return await this.getPlaylistById(playlistId); } catch (error) { logger.error('Error adding games to playlist:', error); throw error; @@ -212,36 +210,20 @@ export class PlaylistService { data: AddGamesToPlaylistDto ): Promise { try { - const playlistsPath = config.flashpointPlaylistsPath; - const files = await fs.readdir(playlistsPath); + const found = await this.findPlaylistFile(playlistId); + if (!found) return null; - // Find playlist file - for (const file of files) { - if (file.endsWith('.json')) { - const filePath = path.join(playlistsPath, file); - const content = await fs.readFile(filePath, 'utf-8'); - const playlist = JSON.parse(content); + const { filePath, data: playlist } = found; - const id = playlist.id || path.basename(file, '.json'); + playlist.games = (playlist.games || []).filter((g: PlaylistGameEntry) => { + const gameId = typeof g === 'string' ? g : g.gameId; + return gameId && !data.gameIds.includes(gameId); + }); - if (id === playlistId) { - // Remove games - playlist.games = (playlist.games || []).filter((g: any) => { - const gameId = typeof g === 'string' ? g : g.gameId; - return !data.gameIds.includes(gameId); - }); + await fs.writeFile(filePath, JSON.stringify(playlist, null, '\t'), 'utf-8'); + logger.info(`Removed ${data.gameIds.length} games from playlist: ${playlist.title}`); - // Write updated playlist - await fs.writeFile(filePath, JSON.stringify(playlist, null, '\t'), 'utf-8'); - - logger.info(`Removed ${data.gameIds.length} games from playlist: ${playlist.title}`); - - return await this.getPlaylistById(playlistId); - } - } - } - - return null; + return await this.getPlaylistById(playlistId); } catch (error) { logger.error('Error removing games from playlist:', error); throw error; @@ -250,32 +232,55 @@ export class PlaylistService { async deletePlaylist(playlistId: string): Promise { try { - const playlistsPath = config.flashpointPlaylistsPath; - const files = await fs.readdir(playlistsPath); + const found = await this.findPlaylistFile(playlistId); + if (!found) return false; - for (const file of files) { - if (file.endsWith('.json')) { - const filePath = path.join(playlistsPath, file); - const content = await fs.readFile(filePath, 'utf-8'); - const playlist = JSON.parse(content); - - const id = playlist.id || path.basename(file, '.json'); - - if (id === playlistId) { - await fs.unlink(filePath); - logger.info(`Deleted playlist: ${playlist.title} (${id})`); - return true; - } - } - } - - return false; + await fs.unlink(found.filePath); + logger.info(`Deleted playlist: ${found.data.title} (${playlistId})`); + return true; } catch (error) { logger.error('Error deleting playlist:', error); throw error; } } + /** + * Find a playlist file by ID. Tries direct {id}.json first, then scans all files. + * Returns file path and parsed data, or null if not found. + */ + // eslint-disable-next-line @typescript-eslint/no-explicit-any + private async findPlaylistFile(id: string): Promise<{ filePath: string; data: any } | null> { + const playlistsPath = config.flashpointPlaylistsPath; + + // Try direct path first (O(1)) + const directPath = path.join(playlistsPath, `${id}.json`); + try { + const content = await fs.readFile(directPath, 'utf-8'); + const data = JSON.parse(content); + const playlistId = data.id || path.basename(directPath, '.json'); + if (playlistId === id) return { filePath: directPath, data }; + } catch { + // Not found at direct path + } + + // Fallback: scan all files + const files = await fs.readdir(playlistsPath); + for (const file of files) { + if (!file.endsWith('.json') || file === `${id}.json`) continue; + const filePath = path.join(playlistsPath, file); + try { + const content = await fs.readFile(filePath, 'utf-8'); + const data = JSON.parse(content); + const playlistId = data.id || path.basename(file, '.json'); + if (playlistId === id) return { filePath, data }; + } catch { + // Skip unreadable files + } + } + + return null; + } + private generateUUID(): string { return crypto.randomUUID(); } diff --git a/backend/src/services/RoleService.ts b/backend/src/services/RoleService.ts index be206c3..73e88bb 100644 --- a/backend/src/services/RoleService.ts +++ b/backend/src/services/RoleService.ts @@ -173,7 +173,7 @@ export class RoleService { } const updates: string[] = []; - const params: any[] = []; + const params: unknown[] = []; if (name !== undefined) { const existing = UserDatabaseService.get('SELECT id FROM roles WHERE name = ? AND id != ?', [ @@ -224,16 +224,20 @@ export class RoleService { throw new AppError(403, 'Cannot modify system role permissions'); } - // Delete existing permissions - UserDatabaseService.run('DELETE FROM role_permissions WHERE role_id = ?', [roleId]); + // Wrap delete + insert in transaction to ensure atomicity + const db = UserDatabaseService.getDatabase(); + const updatePermissions = db.transaction(() => { + db.prepare('DELETE FROM role_permissions WHERE role_id = ?').run(roleId); - // Insert new permissions - for (const permissionId of permissionIds) { - UserDatabaseService.run( - 'INSERT INTO role_permissions (role_id, permission_id) VALUES (?, ?)', - [roleId, permissionId] + const insert = db.prepare( + 'INSERT INTO role_permissions (role_id, permission_id) VALUES (?, ?)' ); - } + for (const permissionId of permissionIds) { + insert.run(roleId, permissionId); + } + }); + + updatePermissions(); // Invalidate permission cache for this role PermissionCache.invalidateRole(roleId); diff --git a/backend/src/services/RuffleService.ts b/backend/src/services/RuffleService.ts index a115814..0a1b86f 100644 --- a/backend/src/services/RuffleService.ts +++ b/backend/src/services/RuffleService.ts @@ -5,6 +5,11 @@ import AdmZip from 'adm-zip'; import { AppError } from '../middleware/errorHandler'; import { logger } from '../utils/logger'; +interface GitHubAsset { + name: string; + browser_download_url: string; +} + /** * Service for managing Ruffle emulator versions */ @@ -84,7 +89,7 @@ export class RuffleService { } // Find web-selfhosted asset - const asset = release.assets.find((a: any) => a.name.includes('web-selfhosted.zip')); + const asset = release.assets.find((a: GitHubAsset) => a.name.includes('web-selfhosted.zip')); if (!asset) { throw new Error('web-selfhosted.zip not found in release'); diff --git a/backend/src/services/SystemSettingsService.ts b/backend/src/services/SystemSettingsService.ts index 8bd5d72..be26a6f 100644 --- a/backend/src/services/SystemSettingsService.ts +++ b/backend/src/services/SystemSettingsService.ts @@ -19,7 +19,7 @@ export class SystemSettingsService { return null; } - const valueToUse = row.value || row.default_value; + const valueToUse = row.value ?? row.default_value; return this.parseValue(valueToUse, row.data_type); } @@ -38,7 +38,7 @@ export class SystemSettingsService { // Remove category prefix from key (e.g., 'auth.guest_access_enabled' -> 'guestAccessEnabled') const shortKey = row.key.replace(`${category}.`, ''); const camelCaseKey = this.snakeToCamel(shortKey); - const valueToUse = row.value || row.default_value; + const valueToUse = row.value ?? row.default_value; const parsedValue = this.parseValue(valueToUse, row.data_type); if (parsedValue !== null) { settings[camelCaseKey] = parsedValue; @@ -66,7 +66,7 @@ export class SystemSettingsService { const shortKey = row.key.replace(`${row.category}.`, ''); const camelCaseKey = this.snakeToCamel(shortKey); - const valueToUse = row.value || row.default_value; + const valueToUse = row.value ?? row.default_value; const parsedValue = this.parseValue(valueToUse, row.data_type); if (parsedValue !== null) { allSettings[row.category][camelCaseKey] = parsedValue; @@ -97,12 +97,14 @@ export class SystemSettingsService { * Update multiple settings in a category */ updateCategory(category: string, settings: CategorySettings, updatedBy?: number): void { - // Use individual updates (transactions would require accessing private db property) - Object.entries(settings).forEach(([key, value]) => { - const snakeKey = this.camelToSnake(key); - const fullKey = `${category}.${snakeKey}`; - this.set(fullKey, value, updatedBy); - }); + const db = UserDatabaseService.getDatabase(); + db.transaction(() => { + Object.entries(settings).forEach(([key, value]) => { + const snakeKey = this.camelToSnake(key); + const fullKey = `${category}.${snakeKey}`; + this.set(fullKey, value, updatedBy); + }); + })(); } /** @@ -123,7 +125,7 @@ export class SystemSettingsService { const shortKey = row.key.replace(`${row.category}.`, ''); const camelCaseKey = this.snakeToCamel(shortKey); - const valueToUse = row.value || row.default_value; + const valueToUse = row.value ?? row.default_value; const parsedValue = this.parseValue(valueToUse, row.data_type); if (parsedValue !== null) { publicSettings[row.category][camelCaseKey] = parsedValue; diff --git a/backend/src/services/UpdateService.ts b/backend/src/services/UpdateService.ts index 767da78..1292608 100644 --- a/backend/src/services/UpdateService.ts +++ b/backend/src/services/UpdateService.ts @@ -169,6 +169,15 @@ export class UpdateService { let errorOutput = ''; + // Store timeout handle so we can clear it when process completes + const timeoutHandle = setTimeout( + () => { + fpm.kill(); + reject(new Error('Update process timed out')); + }, + 5 * 60 * 1000 + ); + fpm.stdout?.on('data', (data) => { logger.info('[UpdateService] FPM Output:', data.toString()); }); @@ -179,6 +188,7 @@ export class UpdateService { }); fpm.on('close', (code) => { + clearTimeout(timeoutHandle); if (code === 0) { logger.info('[UpdateService] Updates installed successfully'); resolve({ @@ -195,18 +205,10 @@ export class UpdateService { }); fpm.on('error', (error) => { + clearTimeout(timeoutHandle); logger.error('[UpdateService] Failed to start FPM:', error); reject(error); }); - - // Timeout after 5 minutes - setTimeout( - () => { - fpm.kill(); - reject(new Error('Update process timed out')); - }, - 5 * 60 * 1000 - ); }); } catch (error) { logger.error('[UpdateService] Error during update installation:', error); diff --git a/backend/src/services/UserDatabaseService.ts b/backend/src/services/UserDatabaseService.ts index 7d492f3..7113e69 100644 --- a/backend/src/services/UserDatabaseService.ts +++ b/backend/src/services/UserDatabaseService.ts @@ -162,15 +162,19 @@ export class UserDatabaseService { const checksum = this.computeChecksum(sql); const start = Date.now(); - // Execute migration - this.db!.exec(sql); + // Wrap migration execution and registry insert in a single transaction + const runMigration = this.db!.transaction(() => { + this.db!.exec(sql); + + const executionTime = Date.now() - start; + this.db!.prepare( + 'INSERT INTO migrations (name, applied_at, checksum, execution_time_ms) VALUES (?, ?, ?, ?)' + ).run(name, new Date().toISOString(), checksum, executionTime); + }); + + runMigration(); - // Record migration in registry const executionTime = Date.now() - start; - this.db!.prepare( - 'INSERT INTO migrations (name, applied_at, checksum, execution_time_ms) VALUES (?, ?, ?, ?)' - ).run(name, new Date().toISOString(), checksum, executionTime); - logger.info(`[UserDB] Migration completed: ${name} (${executionTime}ms)`); } else { logger.debug(`[UserDB] Migration already applied: ${name}`); diff --git a/backend/src/services/UserPlaylistService.ts b/backend/src/services/UserPlaylistService.ts index 6b86161..1d1ef94 100644 --- a/backend/src/services/UserPlaylistService.ts +++ b/backend/src/services/UserPlaylistService.ts @@ -199,7 +199,7 @@ export class UserPlaylistService { } const updates: string[] = []; - const values: any[] = []; + const values: (string | number | null)[] = []; if (data.title !== undefined) { updates.push('title = ?'); @@ -285,7 +285,7 @@ export class UserPlaylistService { WHERE playlist_id = ? `); - let orderIndex = (maxOrderStmt.get(playlistId) as any).maxOrder + 1; + let orderIndex = (maxOrderStmt.get(playlistId) as { maxOrder: number }).maxOrder + 1; // Insert games (ignore duplicates) const insertStmt = db.prepare(` @@ -699,41 +699,59 @@ export class UserPlaylistService { return null; } - // Create new playlist - const title = newTitle || `${sourcePlaylist.title} (Copy)`; - const newPlaylist = this.createPlaylist(userId, { - title, - description: sourcePlaylist.description || undefined, - icon: sourcePlaylist.icon || undefined, - }); + // Wrap create + game inserts in transaction to ensure atomicity + const newPlaylistId = db.transaction(() => { + // Create new playlist + const title = newTitle || `${sourcePlaylist.title} (Copy)`; + const now = new Date().toISOString(); + const result = db + .prepare( + `INSERT INTO user_playlists (user_id, title, description, icon, created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?)` + ) + .run( + userId, + title, + sourcePlaylist.description || null, + sourcePlaylist.icon || null, + now, + now + ); - // Copy games with same order - const gamesStmt = db.prepare(` - SELECT game_id, order_index, notes - FROM user_playlist_games - WHERE playlist_id = ? - ORDER BY order_index - `); + const playlistId = result.lastInsertRowid as number; - const games = gamesStmt.all(sourcePlaylist.id) as { - game_id: string; - order_index: number; - notes: string | null; - }[]; - - if (games.length > 0) { - const insertStmt = db.prepare(` - INSERT INTO user_playlist_games (playlist_id, game_id, order_index, notes) - VALUES (?, ?, ?, ?) + // Copy games with same order + const gamesStmt = db.prepare(` + SELECT game_id, order_index, notes + FROM user_playlist_games + WHERE playlist_id = ? + ORDER BY order_index `); - for (const game of games) { - insertStmt.run(newPlaylist.id, game.game_id, game.order_index, game.notes); + const games = gamesStmt.all(sourcePlaylist.id) as { + game_id: string; + order_index: number; + notes: string | null; + }[]; + + if (games.length > 0) { + const insertStmt = db.prepare(` + INSERT INTO user_playlist_games (playlist_id, game_id, order_index, notes) + VALUES (?, ?, ?, ?) + `); + + for (const game of games) { + insertStmt.run(playlistId, game.game_id, game.order_index, game.notes); + } } - } - logger.info(`Cloned shared playlist "${sourcePlaylist.title}" to user ${userId} as "${title}"`); + return playlistId; + })(); - return this.getPlaylistById(newPlaylist.id, userId); + logger.info( + `Cloned shared playlist "${sourcePlaylist.title}" to user ${userId} as "${newTitle || `${sourcePlaylist.title} (Copy)`}"` + ); + + return this.getPlaylistById(newPlaylistId, userId); } } diff --git a/backend/src/services/UserService.ts b/backend/src/services/UserService.ts index a398339..39c8de0 100644 --- a/backend/src/services/UserService.ts +++ b/backend/src/services/UserService.ts @@ -13,20 +13,27 @@ export class UserService { async getUsers(page: number = 1, limit: number = 50): Promise> { const offset = calculateOffset(page, limit); + // Single query with window function for total count (was 2 separate queries) const users = UserDatabaseService.all( `SELECT u.id, u.username, u.email, u.role_id as roleId, r.name as roleName, u.is_active as isActive, u.created_at as createdAt, - u.updated_at as updatedAt, u.last_login_at as lastLoginAt + u.updated_at as updatedAt, u.last_login_at as lastLoginAt, + COUNT(*) OVER() as total_count FROM users u JOIN roles r ON u.role_id = r.id ORDER BY u.created_at DESC LIMIT ? OFFSET ?`, [limit, offset] - ); + ) as (User & { total_count: number })[]; - const total = UserDatabaseService.get('SELECT COUNT(*) as count FROM users', [])?.count || 0; + const total = users.length > 0 ? users[0].total_count : 0; - return createPaginatedResponse(users, total, page, limit); + // Remove total_count from result objects + users.forEach((user) => { + delete (user as unknown as Record).total_count; + }); + + return createPaginatedResponse(users as User[], total, page, limit); } /** @@ -50,34 +57,35 @@ export class UserService { * Create new user */ async createUser(data: CreateUserData): Promise { - // Check if username exists - const existingUser = UserDatabaseService.get('SELECT id FROM users WHERE username = ?', [ - data.username, - ]); - if (existingUser) { - throw new AppError(409, 'Username already exists'); - } - - // Check if email exists - const existingEmail = UserDatabaseService.get('SELECT id FROM users WHERE email = ?', [ - data.email, - ]); - if (existingEmail) { - throw new AppError(409, 'Email already exists'); - } - - // Hash password + // Hash password before transaction const passwordHash = await hashPassword(data.password); - // Create user - const result = UserDatabaseService.run( - 'INSERT INTO users (username, email, password_hash, role_id) VALUES (?, ?, ?, ?)', - [data.username, data.email, passwordHash, data.roleId] - ); + // Wrap uniqueness checks and insert in transaction to prevent race condition + const db = UserDatabaseService.getDatabase(); + const userId = db.transaction(() => { + // Check username uniqueness inside transaction + const existingUser = db.prepare('SELECT id FROM users WHERE username = ?').get(data.username); + if (existingUser) { + throw new AppError(409, 'Username already exists'); + } - const userId = result.lastInsertRowid as number; + // Check email uniqueness inside transaction + const existingEmail = db.prepare('SELECT id FROM users WHERE email = ?').get(data.email); + if (existingEmail) { + throw new AppError(409, 'Email already exists'); + } - // Return created user + // Create user + const result = db + .prepare( + 'INSERT INTO users (username, email, password_hash, role_id, is_active) VALUES (?, ?, ?, ?, ?)' + ) + .run(data.username, data.email, passwordHash, data.roleId, data.isActive !== false ? 1 : 0); + + return result.lastInsertRowid as number; + })(); + + // Return created user (outside transaction is fine) return (await this.getUserById(userId))!; } @@ -92,7 +100,7 @@ export class UserService { // Build update query dynamically const updates: string[] = []; - const params: any[] = []; + const params: unknown[] = []; if (data.email !== undefined) { // Check if email is taken by another user @@ -164,17 +172,24 @@ export class UserService { /** * Change user password */ - async changePassword(id: number, currentPassword: string, newPassword: string): Promise { + async changePassword( + id: number, + currentPassword: string, + newPassword: string, + isAdminReset: boolean = false + ): Promise { const user = UserDatabaseService.get('SELECT password_hash FROM users WHERE id = ?', [id]); if (!user) { throw new AppError(404, 'User not found'); } - // Verify current password using centralized utility - const isValid = await verifyPassword(currentPassword, user.password_hash); - if (!isValid) { - throw new AppError(401, 'Current password is incorrect'); + // Verify current password using centralized utility (skip if admin reset) + if (!isAdminReset) { + const isValid = await verifyPassword(currentPassword, user.password_hash); + if (!isValid) { + throw new AppError(401, 'Current password is incorrect'); + } } // Hash new password @@ -239,7 +254,7 @@ export class UserService { } /** - * Update theme settings in user_settings table + * Update theme settings in user_settings table (atomic transaction) */ async updateThemeSettings(userId: number, mode: string, primaryColor: string): Promise { const user = await this.getUserById(userId); @@ -247,8 +262,11 @@ export class UserService { throw new AppError(404, 'User not found'); } - await this.setUserSetting(userId, 'theme_mode', mode); - await this.setUserSetting(userId, 'primary_color', primaryColor); + // Execute both updates in parallel for better performance + await Promise.all([ + this.setUserSetting(userId, 'theme_mode', mode), + this.setUserSetting(userId, 'primary_color', primaryColor), + ]); } /** diff --git a/backend/src/utils/cookies.ts b/backend/src/utils/cookies.ts new file mode 100644 index 0000000..824a008 --- /dev/null +++ b/backend/src/utils/cookies.ts @@ -0,0 +1,32 @@ +import { CookieOptions, Response } from 'express'; +import { config } from '../config'; + +const COOKIE_NAME = 'fp_refresh'; +const isProduction = config.nodeEnv === 'production'; + +function getRefreshCookieOptions(): CookieOptions { + return { + httpOnly: true, + secure: isProduction, + sameSite: 'lax', + path: '/api/auth', + maxAge: 30 * 24 * 60 * 60 * 1000, // 30 days in ms + }; +} + +export function setRefreshTokenCookie(res: Response, refreshToken: string): void { + res.cookie(COOKIE_NAME, refreshToken, getRefreshCookieOptions()); +} + +export function clearRefreshTokenCookie(res: Response): void { + res.clearCookie(COOKIE_NAME, { + httpOnly: true, + secure: isProduction, + sameSite: 'lax', + path: '/api/auth', + }); +} + +export function getRefreshTokenFromCookie(cookies: Record): string | undefined { + return cookies?.[COOKIE_NAME]; +} diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml index 681d07f..a67923c 100644 --- a/docker-compose.dev.yml +++ b/docker-compose.dev.yml @@ -27,6 +27,7 @@ services: restart: unless-stopped ports: - "${API_PORT:-3100}:3100" + # Backend now includes game content serving on /game-proxy and /game-zip volumes: - ${FLASHPOINT_HOST_PATH:?FLASHPOINT_HOST_PATH is required}:/data/flashpoint:ro - backend-db:/app/data @@ -36,7 +37,6 @@ services: - PGID=${PGID:-1000} - NODE_ENV=${NODE_ENV:-production} - TZ=${TZ:-UTC} - - GAME_SERVICE_HOST=game-service - DOMAIN=${DOMAIN:-http://localhost:${WEB_PORT:-80}} - LOG_LEVEL=${LOG_LEVEL:-info} - JWT_SECRET=${JWT_SECRET:?JWT_SECRET is required} @@ -48,11 +48,8 @@ services: - OTEL_EXPORTER_OTLP_ENDPOINT=${OTEL_EXPORTER_OTLP_ENDPOINT:-} - OTEL_API_KEY=${OTEL_API_KEY:-} - OTEL_SERVICE_NAME=${OTEL_SERVICE_NAME:-flashpoint-web-backend} - depends_on: - game-service: - condition: service_healthy healthcheck: - test: ["CMD", "curl", "--fail", "http://localhost:3100/api/health"] + test: ["CMD", "curl", "--fail", "http://localhost:3100/health"] interval: 30s timeout: 10s retries: 3 @@ -60,34 +57,6 @@ services: networks: - flashpoint-network - game-service: - build: - context: . - dockerfile: ./game-service/Dockerfile - image: flashpoint-game-service:dev - container_name: flashpoint-game-service - restart: unless-stopped - ports: - - "${PROXY_PORT:-22500}:22500" - - "${GAMEZIP_PORT:-22501}:22501" - volumes: - - ${FLASHPOINT_HOST_PATH:?FLASHPOINT_HOST_PATH is required}:/data/flashpoint:ro - - ${LOGS_PATH:-./logs}/game-service:/app/logs - environment: - - PUID=${PUID:-1000} - - PGID=${PGID:-1000} - - NODE_ENV=${NODE_ENV:-production} - - TZ=${TZ:-UTC} - - LOG_LEVEL=${LOG_LEVEL:-info} - healthcheck: - test: ["CMD", "curl", "--fail", "http://localhost:22500/health"] - interval: 30s - timeout: 10s - retries: 3 - start_period: 30s - networks: - - flashpoint-network - frontend: build: context: . @@ -103,12 +72,9 @@ services: - TZ=${TZ:-UTC} - BACKEND_HOST=${BACKEND_HOST:-backend} - BACKEND_PORT=${BACKEND_PORT:-3100} - - GAME_SERVICE_HOST=${GAME_SERVICE_HOST:-game-service} depends_on: backend: condition: service_healthy - game-service: - condition: service_healthy healthcheck: test: ["CMD", "curl", "--fail", "http://localhost:8080/"] interval: 30s diff --git a/docker-compose.yml b/docker-compose.yml index d78735d..ad4f7f4 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -22,6 +22,7 @@ services: restart: unless-stopped ports: - "${API_PORT:-3100}:3100" + # Backend now includes game content serving on /game-proxy and /game-zip volumes: - ${FLASHPOINT_HOST_PATH:?FLASHPOINT_HOST_PATH is required}:/data/flashpoint:ro - ${DATA_PATH:-./data}:/app/data @@ -31,7 +32,6 @@ services: - PGID=${PGID:-1000} - NODE_ENV=production - TZ=${TZ:-UTC} - - GAME_SERVICE_HOST=game-service - DOMAIN=${DOMAIN:-http://localhost:${WEB_PORT:-80}} - LOG_LEVEL=${LOG_LEVEL:-info} - JWT_SECRET=${JWT_SECRET:?JWT_SECRET is required} @@ -43,39 +43,13 @@ services: - OTEL_EXPORTER_OTLP_ENDPOINT=${OTEL_EXPORTER_OTLP_ENDPOINT:-} - OTEL_API_KEY=${OTEL_API_KEY:-} - OTEL_SERVICE_NAME=${OTEL_SERVICE_NAME:-flashpoint-web-backend} - depends_on: - game-service: - condition: service_healthy healthcheck: - test: ["CMD", "curl", "--fail", "http://localhost:3100/api/health"] + test: ["CMD", "curl", "--fail", "http://localhost:3100/health"] interval: 30s timeout: 10s retries: 3 start_period: 40s - game-service: - image: darkraise/flashpoint-game-service:${IMAGE_TAG:-latest} - container_name: flashpoint-game-service - restart: unless-stopped - ports: - - "${PROXY_PORT:-22500}:22500" - - "${GAMEZIP_PORT:-22501}:22501" - volumes: - - ${FLASHPOINT_HOST_PATH:?FLASHPOINT_HOST_PATH is required}:/data/flashpoint:ro - - ${LOGS_PATH:-./logs}:/app/logs - environment: - - PUID=${PUID:-1000} - - PGID=${PGID:-1000} - - NODE_ENV=production - - TZ=${TZ:-UTC} - - LOG_LEVEL=${LOG_LEVEL:-info} - healthcheck: - test: ["CMD", "curl", "--fail", "http://localhost:22500/health"] - interval: 30s - timeout: 10s - retries: 3 - start_period: 30s - frontend: image: darkraise/flashpoint-frontend:${IMAGE_TAG:-latest} container_name: flashpoint-frontend @@ -88,12 +62,9 @@ services: - TZ=${TZ:-UTC} - BACKEND_HOST=${BACKEND_HOST:-backend} - BACKEND_PORT=${BACKEND_PORT:-3100} - - GAME_SERVICE_HOST=${GAME_SERVICE_HOST:-game-service} depends_on: backend: condition: service_healthy - game-service: - condition: service_healthy healthcheck: test: ["CMD", "curl", "--fail", "http://localhost:8080/"] interval: 30s diff --git a/docs/01-overview/project-overview.md b/docs/01-overview/project-overview.md index 165ff7b..2441d49 100644 --- a/docs/01-overview/project-overview.md +++ b/docs/01-overview/project-overview.md @@ -65,27 +65,25 @@ offering: ## Architecture Highlights -Flashpoint Web consists of three independent services: +Flashpoint Web consists of two independent services: 1. **Backend (Port 3100)** - REST API server handling game metadata, user - management, and authentication + management, authentication, and game file serving 2. **Frontend (Port 5173)** - React web application providing the user interface -3. **Game Service (Ports 22500/22501)** - File serving proxy for game content - and ZIP archives This separation allows: -- Independent scaling of services -- Backend non-blocking on large file transfers -- Game service caching and CDN compatibility - Clear separation of concerns +- Backend handles both metadata and game content +- Frontend focuses on user interface and client-side state +- Simplified deployment with fewer containers ## Technology Stack Summary -- **Backend**: Node.js, Express, TypeScript, SQLite (BetterSqlite3), JWT, bcrypt +- **Backend**: Node.js, Express, TypeScript, SQLite (BetterSqlite3), JWT, bcrypt, + node-stream-zip - **Frontend**: React 18, TypeScript, Vite, TanStack Query, Zustand, Tailwind CSS, Shadcn UI -- **Game Service**: Node.js, TypeScript, node-stream-zip, Axios - **Game Emulation**: Ruffle (Flash), Native Browser (HTML5) ## Use Cases diff --git a/docs/01-overview/technology-stack.md b/docs/01-overview/technology-stack.md index 913d695..da121ee 100644 --- a/docs/01-overview/technology-stack.md +++ b/docs/01-overview/technology-stack.md @@ -31,17 +31,6 @@ experience, and ecosystem maturity. | Shadcn UI | Latest | Component library | | Ruffle | 0.2.0 | Flash emulator | -### Game Service Technologies - -| Technology | Version | Purpose | -| --------------- | ------- | --------------------- | -| Node.js | 20+ | JavaScript runtime | -| Express | 4.18 | HTTP server | -| TypeScript | 5.3 | Type safety | -| node-stream-zip | 1.15 | ZIP archive streaming | -| Axios | 1.6 | HTTP client | -| Winston | 3.11 | Logging | - --- ## Backend Stack @@ -63,6 +52,8 @@ JavaScript runtime built on Chrome's V8 engine. - Database operations - File watching (hot-reload) - Background jobs +- Game file serving and proxying +- ZIP archive mounting and streaming ### Express (v4.18) @@ -122,6 +113,26 @@ const stmt = fpDb.prepare( const games = stmt.all(`%${search}%`, platform); ``` +### node-stream-zip (v1.15) + +Zero-dependency ZIP file streaming library integrated into backend. + +**Why chosen:** + +- Streaming (no extraction required) +- Memory efficient random access +- Fast direct reads +- Simple API + +**Use:** + +```typescript +const zip = new StreamZip.async({ file: '/path/to/game.zip' }); +const entry = await zip.entry(filePath); +const stream = await zip.stream(entry.name); +stream.pipe(res); +``` + ### JWT & bcrypt **JWT**: Stateless authentication, works across instances, mobile-friendly. @@ -251,28 +262,6 @@ Flash Player emulator written in Rust, compiled to WebAssembly. --- -## Game Service Stack - -### node-stream-zip (v1.15) - -Zero-dependency ZIP file streaming library. - -**Why chosen:** - -- Streaming (no extraction required) -- Memory efficient random access -- Fast direct reads -- Simple API - -**Usage:** - -```typescript -const zip = new StreamZip.async({ file: '/path/to/game.zip' }); -const entry = await zip.entry(filePath); -const stream = await zip.stream(entry.name); -stream.pipe(res); -``` - --- ## Development Tools @@ -393,6 +382,9 @@ For each technology, evaluated: - Average response: <50ms - Game search: <100ms - Database queries: <10ms (indexed) +- File streaming: Wire speed +- ZIP file access: <50ms +- CDN fallback: 200-500ms (network dependent) **Frontend:** @@ -400,12 +392,6 @@ For each technology, evaluated: - Time to interactive: <2s - Game grid render: <100ms (virtualized) -**Game Service:** - -- File streaming: Wire speed -- ZIP file access: <50ms -- CDN fallback: 200-500ms (network dependent) - --- ## Browser Support diff --git a/docs/02-architecture/README.md b/docs/02-architecture/README.md index a4d2d44..4444bb6 100644 --- a/docs/02-architecture/README.md +++ b/docs/02-architecture/README.md @@ -12,10 +12,11 @@ flows. Topics covered: -- Three-service architecture overview (Frontend, Backend, Game Service) +- Two-service architecture overview (Frontend, Backend with integrated game serving) - Technology stack and rationale - Service responsibilities and boundaries - Database architecture (flashpoint.sqlite and user.db) +- Game content serving architecture (integrated in backend) - Security architecture (JWT, RBAC) - File system layout - Scalability considerations @@ -28,13 +29,13 @@ system. ### 2. [Service Communication](./service-communication.md) -**Inter-service communication patterns and protocols** +**Communication patterns and protocols between Frontend and Backend** Topics covered: - Frontend ↔ Backend communication (REST API) -- Backend ↔ Game Service communication (internal APIs) -- Frontend ↔ Game Service communication (direct file loading) +- Integrated game file serving in backend +- Backend game routes (`/game-proxy/*` and `/game-zip/*`) - API endpoints by domain - Authentication token flow - Request/response formats @@ -233,12 +234,12 @@ Express middleware chains handle cross-cutting concerns: - Single deployment - Consistent tooling -### Why Three Services? +### Why Two Services? - Clear separation of concerns -- Independent scaling -- Backend doesn't block on file I/O -- Game service can be replicated +- Frontend focuses on UI and client-side state +- Backend handles all business logic and content serving +- Simplified deployment (2 containers instead of 3) - Easier to reason about ## Common Workflows diff --git a/docs/02-architecture/authentication-flow.md b/docs/02-architecture/authentication-flow.md index e5faf27..8027f3a 100644 --- a/docs/02-architecture/authentication-flow.md +++ b/docs/02-architecture/authentication-flow.md @@ -11,9 +11,9 @@ Flashpoint Web uses JWT-based authentication with role-based access control graph TB subgraph "Frontend" LoginForm[Login Form] - AuthStore[Zustand Auth Store] + AuthStore[Zustand Auth Store
Access Token Only] APIClient[Axios API Client] - LocalStorage[localStorage] + Cookies[HTTP-only Cookies
fp_refresh] end subgraph "Backend" @@ -30,15 +30,16 @@ graph TB end LoginForm --> AuthStore + AuthRoute -.Set Cookie.-> Cookies + AuthRoute -.accessToken.-> APIClient AuthStore --> APIClient APIClient --> AuthRoute AuthRoute --> AuthService AuthService --> UserDB UserDB -.return user.-> AuthService AuthService -.JWT tokens.-> AuthRoute - AuthRoute -.tokens + user.-> APIClient + AuthRoute -.accessToken only.-> APIClient APIClient --> AuthStore - AuthStore --> LocalStorage APIClient -.JWT Bearer.-> GameRoutes GameRoutes --> AuthMiddleware @@ -48,6 +49,7 @@ graph TB style AuthStore fill:#61dafb style AuthService fill:#90ee90 style UserDB fill:#ff9999 + style Cookies fill:#ffd700 ``` ## 1. User Registration Flow @@ -87,9 +89,9 @@ sequenceDiagram AuthService->>AuthService: Generate JWT tokens AuthService-->>Backend: {user, accessToken, refreshToken} - Backend-->>API: 201 Created - API->>AuthStore: setAuth(user, tokens) - AuthStore->>LocalStorage: Persist tokens + Backend->>Backend: Set fp_refresh HTTP-only cookie + Backend-->>API: 201 Created (only accessToken in body) + API->>AuthStore: setAuth(user, accessToken) AuthStore-->>RegisterForm: Success RegisterForm->>User: Redirect to dashboard end @@ -141,12 +143,12 @@ sequenceDiagram else Valid credentials AuthService->>UserDB: SELECT user permissions AuthService->>AuthService: Generate JWT access token (15m expiry) - AuthService->>AuthService: Generate JWT refresh token (7d expiry) + AuthService->>AuthService: Generate JWT refresh token (30d expiry) AuthService-->>Backend: {user, accessToken, refreshToken} - Backend-->>API: 200 OK with tokens + Backend->>Backend: Set fp_refresh HTTP-only cookie + Backend-->>API: 200 OK (only accessToken in body) - API->>AuthStore: setAuth(user, tokens) - AuthStore->>LocalStorage: Persist auth state + API->>AuthStore: setAuth(user, accessToken) AuthStore->>ThemeStore: loadThemeFromServer() AuthStore-->>LoginForm: Success LoginForm->>User: Redirect to /browse @@ -167,12 +169,12 @@ sequenceDiagram exp: 1705580100 // 15 minutes } -// Refresh Token +// Refresh Token (stored in HTTP-only cookie) { userId: 1, type: "refresh", iat: 1705579200, - exp: 1706184000 // 7 days + exp: 1708257600 // 30 days } ``` @@ -183,6 +185,7 @@ sequenceDiagram participant Component participant API Client participant Interceptor + participant AuthStore participant Backend participant AuthMiddleware participant RBACMiddleware @@ -191,10 +194,13 @@ sequenceDiagram Component->>API Client: gamesApi.search(filters) API Client->>Interceptor: Request interceptor + Interceptor->>AuthStore: Get accessToken from store Interceptor->>Interceptor: Add Authorization header - Note over Interceptor: Authorization: Bearer {token} + Note over Interceptor: Authorization: Bearer {accessToken} Interceptor->>Backend: GET /api/games?filters + Note over Interceptor,Backend: Cookies sent automatically + Backend->>AuthMiddleware: authenticate middleware AuthMiddleware->>AuthMiddleware: Extract & verify JWT @@ -267,6 +273,7 @@ sequenceDiagram participant API Client participant Interceptor participant AuthStore + participant Cookies participant Backend participant AuthService @@ -275,22 +282,27 @@ sequenceDiagram Backend-->>API Client: 401 Unauthorized API Client->>Interceptor: Response interceptor (401 check) - Interceptor->>AuthStore: getState().refreshToken - alt No refresh token + Interceptor->>Backend: POST /api/auth/refresh (empty body) + Note over Interceptor,Backend: Cookies (fp_refresh) sent automatically + + alt No refresh token in cookie + Backend-->>Interceptor: 401 Unauthorized Interceptor->>AuthStore: clearAuth() Interceptor->>Component: Redirect to login else Has refresh token - Interceptor->>Backend: POST /api/auth/refresh {refreshToken} - Backend->>AuthService: refreshAccessToken(token) + Backend->>AuthService: refreshAccessToken(token from cookie) alt Invalid refresh token AuthService-->>Backend: 401 Unauthorized + Backend-->>Interceptor: 401 Unauthorized Interceptor->>AuthStore: clearAuth() Interceptor->>Component: Redirect to login else Valid - AuthService->>AuthService: Generate new tokens - Backend-->>Interceptor: {accessToken, refreshToken} + AuthService->>AuthService: Generate new accessToken + AuthService->>AuthService: Generate new refreshToken + Backend->>Cookies: Set new fp_refresh cookie + Backend-->>Interceptor: {accessToken, expiresIn} Interceptor->>AuthStore: updateAccessToken(newToken) Interceptor->>Interceptor: Retry original request Backend-->>API Client: 200 OK @@ -311,16 +323,10 @@ api.interceptors.response.use( originalRequest._retry = true; try { - const refreshToken = useAuthStore.getState().refreshToken; - if (!refreshToken) { - useAuthStore.getState().clearAuth(); - window.location.href = '/login'; - return Promise.reject(error); - } - - const tokens = await authApi.refreshToken(refreshToken); - useAuthStore.getState().updateAccessToken(tokens.accessToken); - originalRequest.headers.Authorization = `Bearer ${tokens.accessToken}`; + // Refresh token is sent in HTTP-only cookie automatically + const { data } = await authApi.refreshToken(); + useAuthStore.getState().updateAccessToken(data.accessToken); + originalRequest.headers.Authorization = `Bearer ${data.accessToken}`; return api(originalRequest); } catch (refreshError) { useAuthStore.getState().clearAuth(); @@ -334,6 +340,25 @@ api.interceptors.response.use( ); ``` +**Page Reload Recovery**: + +When the page reloads, the access token in memory is lost, but the refresh token +remains in the HTTP-only cookie. The frontend auth initialization sequence detects +this and automatically calls `/auth/refresh` to recover the session: + +```typescript +// During app initialization +useEffect(() => { + const { isAuthenticated, accessToken } = useAuthStore.getState(); + if (isAuthenticated && !accessToken) { + // Session exists (has refresh cookie) but no access token in memory + authApi.refreshToken().then(({ data }) => { + useAuthStore.getState().updateAccessToken(data.accessToken); + }); + } +}, []); +``` + ## 5. Logout Flow ```mermaid @@ -342,17 +367,19 @@ sequenceDiagram participant Header participant AuthStore participant API + participant Cookies participant Backend User->>Header: Click "Logout" - Header->>AuthStore: Get refreshToken - Header->>API: authApi.logout(refreshToken) - API->>Backend: POST /api/auth/logout + Header->>API: authApi.logout() + API->>Backend: POST /api/auth/logout (empty body) + Note over API,Backend: Cookies (fp_refresh) sent automatically + Backend->>Backend: Read refresh token from cookie Backend->>Backend: Invalidate refresh token + Backend->>Cookies: Clear fp_refresh cookie Backend-->>API: 200 OK API->>AuthStore: clearAuth() - AuthStore->>LocalStorage: Clear tokens AuthStore-->>Header: Success Header->>User: Redirect to /login ``` @@ -488,10 +515,12 @@ export const optionalAuth = async (req, res, next) => { **JWT Security**: -- Short-lived access tokens (15 minutes) -- Long-lived refresh tokens (7 days) +- Short-lived access tokens (15 minutes) - held in memory only +- Long-lived refresh tokens (30 days) - stored in HTTP-only cookies +- Refresh tokens are rotated on each use to prevent token reuse attacks - Signed with random 256-bit JWT_SECRET - Verify algorithm: HS256 +- HTTP-only cookie prevents XSS attacks from accessing the refresh token **Rate Limiting**: 5 login attempts per 15 minutes @@ -499,6 +528,13 @@ export const optionalAuth = async (req, res, next) => { **HTTPS**: Secure cookies in production only +**HTTP-only Cookies**: `fp_refresh` cookie is configured with: +- `httpOnly: true` - Not accessible to JavaScript (prevents XSS) +- `secure: true` - Only sent over HTTPS in production +- `sameSite: 'lax'` - CSRF protection +- `path: '/api/auth'` - Only sent to auth endpoints +- `maxAge: 2592000000` - 30 days (in milliseconds) + ## Troubleshooting | Issue | Cause | Solution | diff --git a/docs/02-architecture/game-launch-flow.md b/docs/02-architecture/game-launch-flow.md index b36a3ef..0aefaf5 100644 --- a/docs/02-architecture/game-launch-flow.md +++ b/docs/02-architecture/game-launch-flow.md @@ -2,9 +2,11 @@ ## Overview -Game launch orchestrates multiple services: frontend requests launch data, -backend mounts ZIPs, and frontend loads content directly from game-service into -Ruffle or iframe. +Game launch orchestrates backend and frontend with support for background ZIP +downloads: frontend requests launch data, backend mounts ZIPs (or starts background +downloads), and frontend polls for completion. When ready, frontend loads content +directly from backend game routes into Ruffle or iframe. Downloads show a UI while +in progress, with TanStack Query auto-polling every 2 seconds. ## Architecture @@ -22,11 +24,7 @@ graph TB GameService[GameService] GameDataSvc[GameDataService] FlashpointDB[(flashpoint.sqlite)] - end - - subgraph "Game Service :22500/:22501" - ProxyServer[HTTP Proxy :22500] - ZipServer[GameZip Server :22501] + GameRoutes[Game Routes
/game-proxy/*
/game-zip/*] ZipManager[ZIP Manager] FileSystem[File System] end @@ -37,22 +35,20 @@ graph TB GameService --> FlashpointDB LaunchRoute --> GameDataSvc - GameDataSvc -.mount ZIP.-> ZipServer - ZipServer --> ZipManager + GameDataSvc -.mount ZIP.-> ZipManager ZipManager --> FileSystem LaunchRoute -.return contentUrl.-> GamePlayer GamePlayer -->|Flash game| Ruffle GamePlayer -->|HTML5 game| IFrame - Ruffle -->|load SWF| ProxyServer - IFrame -->|load HTML| ProxyServer - ProxyServer --> FileSystem - ProxyServer -.fallback.-> ZipServer + Ruffle -->|load SWF| GameRoutes + IFrame -->|load HTML| GameRoutes + GameRoutes --> FileSystem style GamePlayer fill:#61dafb style LaunchRoute fill:#90ee90 - style ProxyServer fill:#ffd700 + style GameRoutes fill:#ffd700 ``` ## 1. Game Launch Request Flow @@ -66,7 +62,9 @@ sequenceDiagram participant Backend participant GameService participant GameDataService + participant ZipManager participant FlashpointDB + participant Downloader User->>GameCard: Click "Play Game" GameCard->>GamePlayerView: Navigate to /play/:id @@ -79,25 +77,42 @@ sequenceDiagram FlashpointDB-->>GameService: Game metadata alt Game has data (presentOnDisk) - Backend->>GameDataService: mountGameZip(gameId) - GameDataService->>FlashpointDB: SELECT path from game_data - GameDataService->>Backend: ZIP mounted + Backend->>GameDataService: mountGameZip(gameId) [AWAITED] + GameDataService->>ZipManager: Check if ZIP exists locally - Backend->>GameService: getGameDataPath(gameId) - GameService->>FlashpointDB: SELECT launchCommand + alt ZIP exists locally + ZipManager->>GameDataService: Mount and return {mounted: true} + else ZIP not found locally + GameDataService->>Downloader: Start background download + Downloader-->>GameDataService: {downloading: true} [returns quickly] + end + + Backend->>Backend: Extract launchCommand from game_data else No game data Backend->>Backend: Use game.launchCommand end Backend->>Backend: Construct contentUrl Backend-->>API: Launch data response - API-->>GamePlayerView: {gameId, title, contentUrl} + API-->>GamePlayerView: {gameId, title, contentUrl, downloading: boolean} - GamePlayerView->>GamePlayerView: Determine player type - alt Flash game - GamePlayerView->>GamePlayerView: Initialize Ruffle - else HTML5 game - GamePlayerView->>GamePlayerView: Create iframe + alt downloading: true (ZIP not ready yet) + GamePlayerView->>User: Show download progress UI + GamePlayerView->>GamePlayerView: Start polling every 2s via TanStack Query + loop Poll every 2 seconds + GamePlayerView->>API: gamesApi.getLaunchData(gameId) + API->>Backend: GET /api/games/{id}/launch + Backend->>Backend: Check mount status + Backend-->>API: {downloading: false} when ready + API-->>GamePlayerView: Update with downloading: false + end + else downloading: false (ZIP ready or not needed) + GamePlayerView->>GamePlayerView: Determine player type + alt Flash game + GamePlayerView->>GamePlayerView: Initialize Ruffle + else HTML5 game + GamePlayerView->>GamePlayerView: Create iframe + end end GamePlayerView->>User: Show game player @@ -113,16 +128,30 @@ interface GameLaunchData { launchCommand: string; contentUrl: string; canPlayInBrowser: boolean; + /** True when the game ZIP is being downloaded in the background */ + downloading?: boolean; } -// Example +// Example - ZIP already mounted (ready to play) { "gameId": "abc-123-def-456", "title": "Super Mario Flash", "platform": "Flash", "launchCommand": "http://example.com/games/mario.swf", - "contentUrl": "http://localhost:22500/http://example.com/games/mario.swf", - "canPlayInBrowser": true + "contentUrl": "http://localhost:3100/game-proxy/http://example.com/games/mario.swf", + "canPlayInBrowser": true, + "downloading": false +} + +// Example - ZIP downloading (show download UI, poll every 2s) +{ + "gameId": "xyz-789-abc-123", + "title": "Legacy Game", + "platform": "HTML5", + "launchCommand": "game/index.html", + "contentUrl": "http://localhost:3100/game-proxy/game/index.html", + "canPlayInBrowser": true, + "downloading": true } ``` @@ -132,92 +161,142 @@ interface GameLaunchData { sequenceDiagram participant Backend participant GameDataService - participant ZipServer participant ZipManager participant FileSystem + participant Downloader - Backend->>GameDataService: mountGameZip(gameId) + Backend->>GameDataService: mountGameZip(gameId) [AWAITED] - alt ZIP already mounted - GameDataService->>ZipServer: Check if mounted - ZipServer-->>GameDataService: Already mounted - else ZIP not mounted - GameDataService->>FileSystem: Resolve ZIP path - GameDataService->>ZipServer: POST /mount {gameId, zipPath} + GameDataService->>FileSystem: Look for ZIP file locally - ZipServer->>FileSystem: Check file exists - ZipServer->>ZipManager: Load ZIP (StreamZip) - ZipManager->>ZipManager: Create instance and verify - ZipServer->>ZipServer: Store in mounts map + alt ZIP exists locally + GameDataService->>ZipManager: mount(gameId, zipPath, downloadParams) - ZipServer-->>GameDataService: {success: true, mountPoint} + alt Already mounted + ZipManager-->>GameDataService: {success: true, mounted: true} + else Not yet mounted + ZipManager->>FileSystem: Load ZIP (StreamZip) + ZipManager->>ZipManager: Verify and cache + ZipManager-->>GameDataService: {success: true, mounted: true} + end + + GameDataService-->>Backend: {mounted: true, downloading: false} + else ZIP not found, start download + GameDataService->>Downloader: Start background download [NO AWAIT] + Downloader->>Downloader: Read gameDataSources from preferences.json + Downloader->>Downloader: Download from configured sources (retry/fallback) + Downloader->>Downloader: Verify SHA256 hash + Downloader->>FileSystem: Save to Data/Games/ + + Downloader->>ZipManager: Mount when download completes [async] + + GameDataService-->>Backend: {mounted: false, downloading: true} [returns immediately] end ``` -**ZIP Manager Implementation**: +**Key Behavior**: +- `mountGameZip()` is **awaited** by the backend launch endpoint +- Returns `{ mounted: true, downloading: false }` immediately if ZIP exists +- Returns `{ mounted: false, downloading: true }` immediately if starting download +- Frontend polls every 2s when `downloading: true` until mount completes +- Download happens in background without blocking the response + +**GameDataService Implementation**: ```typescript -class ZipManager { - private mounts: Map = new Map(); - private lastAccess: Map = new Map(); +interface MountResult { + mounted: boolean; + downloading: boolean; +} - async mount(gameId: string, zipPath: string): Promise { - if (this.mounts.has(gameId)) { - this.lastAccess.set(gameId, Date.now()); - return `/gamedata/${gameId}`; - } +async mountGameZip(gameId: string): Promise { + // Get game data entry for dateAdded and sha256 + const gameDataEntry = this.getGameDataEntry(gameId); - if (!fs.existsSync(zipPath)) { - throw new Error(`ZIP file not found: ${zipPath}`); - } + // Find ZIP file in Data/Games/ + const zipFile = files.find((f) => f.startsWith(gameId + '-') && f.endsWith('.zip')); + let zipPath: string; - const zip = new StreamZip.async({ file: zipPath }); - await zip.entries(); + if (zipFile) { + zipPath = path.join(gamesPath, zipFile); + } else if (gameDataEntry?.dateAdded) { + // Construct expected filename (needed for download request) + const timestamp = new Date(gameDataEntry.dateAdded).getTime(); + zipPath = path.join(gamesPath, `${gameId}-${timestamp}.zip`); - this.mounts.set(gameId, zip); - this.lastAccess.set(gameId, Date.now()); + // Start background download (no await) + gameZipServer.mountZip({ + id: gameId, + zipPath, + dateAdded: gameDataEntry.dateAdded, + sha256: gameDataEntry.sha256 + }); - logger.info(`Mounted ZIP: ${gameId}`); - return `/gamedata/${gameId}`; + // Return immediately with downloading flag + return { mounted: false, downloading: true }; } - async getFile(gameId: string, filePath: string): Promise { - const zip = this.mounts.get(gameId); - if (!zip) return null; + // ZIP exists locally - mount it synchronously + const result = await gameZipServer.mountZip({ + id: gameId, + zipPath, + // ... parameters + }); - this.lastAccess.set(gameId, Date.now()); - - try { - return await zip.entryData(filePath); - } catch (error) { - logger.warn(`File not found in ZIP: ${filePath}`); - return null; - } + if (result.success) { + return { mounted: true, downloading: false }; } - async unmount(gameId: string): Promise { - const zip = this.mounts.get(gameId); - if (zip) { - await zip.close(); - this.mounts.delete(gameId); - logger.info(`Unmounted ZIP: ${gameId}`); - } - } - - async cleanupStale(): Promise { - const now = Date.now(); - const maxAge = 60 * 60 * 1000; // 1 hour - - for (const [gameId, lastAccess] of this.lastAccess.entries()) { - if (now - lastAccess > maxAge) { - await this.unmount(gameId); - } - } - } + return { mounted: false, downloading: false }; } ``` -## 3. Game Content Loading +**Frontend Polling Configuration** (`useGameLaunchData` hook): + +```typescript +export function useGameLaunchData(id: string) { + return useQuery({ + queryKey: ['game', id, 'launch'], + queryFn: () => gamesApi.getLaunchData(id), + enabled: !!id, + // Auto-poll every 2s while the game ZIP is being downloaded + refetchInterval: (query) => (query.state.data?.downloading ? 2000 : false), + }); +} +``` + +When `downloading: true`, TanStack Query automatically polls every 2 seconds. When +the backend response changes to `downloading: false`, polling stops automatically. + +## 3. Download Progress UI + +While `downloading: true`, the frontend shows a download progress UI: + +```typescript +// GamePlayerView.tsx +{launchData?.downloading ? ( +
+
+ +

Downloading Game Data...

+

+ The game files are being downloaded. This page will update automatically when ready. +

+ +
+
+) : ( + +)} +``` + +Features: +- Bouncing Download icon and spinner animation +- User-friendly message +- Auto-updates when download completes +- No manual refresh needed (TanStack Query handles polling) + +## 4. Game Content Loading ### Flash Game (Ruffle) @@ -225,26 +304,26 @@ class ZipManager { sequenceDiagram participant GamePlayer participant Ruffle - participant ProxyServer + participant Backend participant FileSystem GamePlayer->>Ruffle: window.RufflePlayer.newest() Ruffle-->>GamePlayer: Player instance GamePlayer->>Ruffle: player.load(contentUrl) - Note over GamePlayer,Ruffle: contentUrl: http://localhost:22500/http://example.com/game.swf + Note over GamePlayer,Ruffle: contentUrl: http://localhost:3100/game-proxy/http://example.com/game.swf - Ruffle->>ProxyServer: GET /http://example.com/game.swf - ProxyServer->>FileSystem: Check htdocs → game data → ZIPs - FileSystem-->>ProxyServer: SWF file bytes + Ruffle->>Backend: GET /game-proxy/http://example.com/game.swf + Backend->>FileSystem: Check htdocs → game data → ZIPs + FileSystem-->>Backend: SWF file bytes - ProxyServer-->>Ruffle: SWF with MIME type + Backend-->>Ruffle: SWF with MIME type Ruffle->>Ruffle: Parse and execute SWF Ruffle->>GamePlayer: Render game loop Asset Loading - Ruffle->>ProxyServer: GET /http://example.com/assets/sprite.png - ProxyServer-->>Ruffle: Asset bytes + Ruffle->>Backend: GET /game-proxy/http://example.com/assets/sprite.png + Backend-->>Ruffle: Asset bytes end ``` @@ -254,35 +333,35 @@ sequenceDiagram sequenceDiagram participant GamePlayer participant IFrame - participant ProxyServer + participant Backend participant FileSystem GamePlayer->>IFrame: Create iframe GamePlayer->>IFrame: Set src to contentUrl - Note over GamePlayer,IFrame: src: http://localhost:22500/http://example.com/game/index.html + Note over GamePlayer,IFrame: src: http://localhost:3100/game-proxy/http://example.com/game/index.html - IFrame->>ProxyServer: GET /http://example.com/game/index.html - ProxyServer->>FileSystem: Resolve HTML file - FileSystem-->>ProxyServer: HTML content + IFrame->>Backend: GET /game-proxy/http://example.com/game/index.html + Backend->>FileSystem: Resolve HTML file + FileSystem-->>Backend: HTML content - ProxyServer-->>IFrame: HTML with MIME type + Backend-->>IFrame: HTML with MIME type IFrame->>IFrame: Parse HTML loop Asset Loading - IFrame->>ProxyServer: GET /http://example.com/game/style.css - ProxyServer-->>IFrame: CSS content - IFrame->>ProxyServer: GET /http://example.com/game/script.js - ProxyServer-->>IFrame: JS content + IFrame->>Backend: GET /game-proxy/http://example.com/game/style.css + Backend-->>IFrame: CSS content + IFrame->>Backend: GET /game-proxy/http://example.com/game/script.js + Backend-->>IFrame: JS content end IFrame->>GamePlayer: Game fully loaded ``` -## 4. HTTP Proxy Server Request Handling +## 5. Game Proxy Route Request Handling ```mermaid graph TB - Request[Incoming Request] + Request[Incoming Request
/game-proxy/*] ParseURL[Parse URL] CheckHTDocs{Check htdocs} CheckGameData{Check game data} @@ -306,6 +385,7 @@ graph TB CheckCDN -->|Found| ReturnFile CheckCDN -->|Not Found| Return404 + style Request fill:#ffd700 style ReturnFile fill:#90ee90 style Return404 fill:#ffcccb ``` @@ -381,7 +461,7 @@ app.get('*', async (req, res) => { }); ``` -## 5. MIME Type Detection +## 6. MIME Type Detection ```typescript function getMimeType(filePath: string): string { @@ -406,7 +486,7 @@ function getMimeType(filePath: string): string { } ``` -## 6. Ruffle Player Integration +## 7. Ruffle Player Integration ```typescript export const RufflePlayer: React.FC<{ gameUrl: string; gameTitle: string }> = ({ @@ -458,19 +538,43 @@ export const RufflePlayer: React.FC<{ gameUrl: string; gameTitle: string }> = ({ }; ``` -## 7. Full Example: Playing "Super Mario Flash" +## 8. Full Example: Playing "Super Mario Flash" (ZIP Already Available) + +**Scenario**: ZIP file is already in `Data/Games/` directory 1. User clicks "Play Game" on game card 2. Frontend: `GET /api/games/abc-123-def-456/launch` 3. Backend queries database, finds `presentOnDisk = 1` -4. Backend: `POST http://localhost:22501/mount` → Mounts ZIP -5. Backend returns: - `contentUrl: http://localhost:22500/http://example.com/games/mario.swf` -6. Frontend creates Ruffle player with this URL -7. Ruffle: `GET http://localhost:22500/http://example.com/games/mario.swf` -8. Proxy server checks htdocs → game data → mounted ZIPs → **Found in ZIP!** -9. Returns `mario.swf` from ZIP with proper MIME type -10. Ruffle executes game, user plays +4. Backend awaits `mountGameZip(abc-123-def-456)` (ZIP already exists locally) +5. Returns: `{downloading: false, contentUrl: http://localhost:3100/game-proxy/...}` +6. Frontend immediately renders Ruffle player +7. Ruffle loads SWF from backend proxy +8. User plays game + +## 9. Full Example: Playing "Legacy Game" (ZIP Needs Download) + +**Scenario**: ZIP file doesn't exist, needs to be downloaded + +1. User clicks "Play Game" on game card +2. Frontend: `GET /api/games/xyz-789-abc-123/launch` +3. Backend queries database, finds `presentOnDisk = 1` but ZIP not found locally +4. Backend awaits `mountGameZip(xyz-789-abc-123)` +5. GameDataService starts **background download** (no await) and returns immediately +6. Backend returns: `{downloading: true, contentUrl: http://localhost:3100/game-proxy/...}` +7. Frontend shows **download progress UI** (bouncing icon, spinner, message) +8. Frontend starts **auto-polling** every 2s via TanStack Query +9. Meanwhile, background downloader: + - Reads `gameDataSources` from `preferences.json` + - Downloads ZIP from configured sources (with retry/fallback) + - Verifies SHA256 hash + - Saves to `Data/Games/{gameId}-{timestamp}.zip` + - Mounts ZIP via ZipManager +10. On next poll (within 2s), backend returns: `{downloading: false, contentUrl: ...}` +11. Frontend automatically **stops polling** and renders Ruffle player +12. User plays game + +**Key Improvement**: User sees friendly download UI instead of a timeout or error, and +the download happens automatically without blocking the initial response. ## Error Handling @@ -486,6 +590,20 @@ export const RufflePlayer: React.FC<{ gameUrl: string; gameTitle: string }> = ({ **Lazy Loading**: Load Ruffle only when needed -**ZIP Caching**: Keep frequently accessed ZIPs mounted (1 hour TTL) +**ZIP Caching**: Keep frequently accessed ZIPs mounted (LRU cache, 30-min TTL) **File Streaming**: Stream large files instead of loading into memory + +**Background Downloads**: Start downloads immediately without waiting for completion, +allowing the frontend to show download progress instead of a timeout + +**Connection Management**: +- `keepAliveTimeout: 65s` - Keeps connections alive for repeated file requests +- `headersTimeout: 66s` - Slightly higher than keepAliveTimeout +- `timeout: 120s` - 2-minute max for any request (handles slow file downloads) +- `maxConnections: 500` - Prevents connection exhaustion during game file streaming + +**Frontend Resource Management**: +- TanStack Query polling stops automatically when download completes +- No unnecessary network traffic when not downloading +- Memory-efficient polling interval (2 seconds) diff --git a/docs/02-architecture/service-communication.md b/docs/02-architecture/service-communication.md index 95d6165..318b70a 100644 --- a/docs/02-architecture/service-communication.md +++ b/docs/02-architecture/service-communication.md @@ -2,8 +2,9 @@ ## Overview -Flashpoint Web uses HTTP-based communication between three services. Frontend -communicates with backend, which proxies game file requests to game-service. +Flashpoint Web uses HTTP-based communication between two services. Frontend +communicates with backend via REST API. The backend handles both business logic +and game file serving through integrated routes. ## Communication Architecture @@ -17,14 +18,12 @@ graph TB subgraph "Backend :3100" Routes[Express Routes] + GameRoutes[Game Routes
/game-proxy/*
/game-zip/*] Auth[Auth Middleware] RBAC[RBAC Middleware] Services[Service Layer] - end - - subgraph "Game Service :22500/:22501" - Proxy[HTTP Proxy] - ZipServer[GameZip Server] + GameProxy[Game Proxy Handler] + ZipManager[ZIP Manager] end UI --> API @@ -34,15 +33,17 @@ graph TB Auth --> RBAC RBAC --> Services - Services -.proxy request.-> Proxy - Services -.proxy request.-> ZipServer + UI --> Interceptor + Interceptor -->|Game Routes
No Auth| GameRoutes + GameRoutes --> GameProxy + GameRoutes --> ZipManager - Proxy -.game files.-> Interceptor - ZipServer -.game files.-> Interceptor + GameProxy -.game files.-> Interceptor + ZipManager -.game files.-> Interceptor style UI fill:#61dafb style Routes fill:#90ee90 - style Proxy fill:#ffd700 + style GameRoutes fill:#ffd700 ``` ## Frontend ↔ Backend Communication @@ -127,59 +128,57 @@ PATCH /api/users/me/settings {theme_mode, primary_color} } ``` -## Backend ↔ Game Service Communication +## Backend Game File Serving -Backend does NOT directly proxy game files. Instead: +Game file serving is integrated into the backend service. When a game launch is +requested: -1. Returns launch URLs pointing to game service -2. Frontend loads content directly from game service -3. Backend mounts ZIPs when needed via internal API +1. Backend mounts ZIP directly in-process via GameZipManager +2. Returns launch URLs pointing to backend game routes +3. Frontend loads content directly from backend game routes +4. Game routes bypass auth middleware but include permissive CORS headers -**Mount Game ZIP**: +**Example: Game Launch**: ```http -POST http://localhost:22501/mount -{ "gameId": "uuid", "zipPath": "D:/Flashpoint/Data/Games/A/uuid.zip" } -Response: { "success": true, "mountPoint": "/gamedata/uuid" } +GET /api/games/{gameId}/launch +Response: { "contentUrl": "http://localhost:3100/game-proxy/http://example.com/game.swf" } ``` -**Unmount Game ZIP**: +**Backend Game Routes**: -```http -POST http://localhost:22501/unmount -{ "gameId": "uuid" } -``` +- `GET /game-proxy/{url}` - HTTP proxy with fallback chain (htdocs → game data → ZIPs → CDN) +- `GET /game-zip/{gameId}/{filePath}` - Stream files from mounted ZIPs -## Frontend ↔ Game Service Communication +## Frontend ↔ Backend Game Routes Communication -Frontend loads game content **directly** from game service: +Frontend loads game content **directly** from backend game routes: ```mermaid sequenceDiagram participant Frontend participant Backend - participant GameService participant FileSystem Frontend->>Backend: GET /api/games/{id}/launch Backend->>Backend: Mount ZIP if needed - Backend-->>Frontend: {contentUrl: "http://localhost:22500/..."} + Backend-->>Frontend: {contentUrl: "http://localhost:3100/game-proxy/..."} - Frontend->>GameService: GET /http://domain.com/game.swf - GameService->>FileSystem: Read from htdocs/ZIP - FileSystem-->>GameService: File bytes - GameService-->>Frontend: game.swf (binary) + Frontend->>Backend: GET /game-proxy/http://domain.com/game.swf + Backend->>FileSystem: Read from htdocs/ZIP + FileSystem-->>Backend: File bytes + Backend-->>Frontend: game.swf (binary) ``` -### HTTP Proxy Server (:22500) +### Game Proxy Route (`/game-proxy/*`) -**Request**: `GET /http://example.com/path/game.swf` +**Request**: `GET /game-proxy/http://example.com/path/game.swf` **Fallback Chain**: 1. Local htdocs: `D:/Flashpoint/Legacy/htdocs/example.com/path/game.swf` 2. Game data directory -3. Mounted ZIPs (via zip-manager) +3. Mounted ZIPs (via ZIP manager) 4. External CDN: `http://infinity.flashpointarchive.org/...` 5. Cache downloaded content locally @@ -191,15 +190,19 @@ Access-Control-Allow-Origin: * Cache-Control: public, max-age=31536000 ``` -### GameZip Server (:22501) +**Important**: This route is registered **before** auth middleware and has +permissive CORS to allow cross-origin game content loading. -**Request**: `GET /gamedata/{gameId}/path/to/file.swf` +### Game ZIP Route (`/game-zip/*`) + +**Request**: `GET /game-zip/{gameId}/path/to/file.swf` - Streams files directly from ZIP (no extraction) - Appropriate MIME type header - CORS headers for cross-origin access - LRU cache for ZIP mounts (max 100, 30-min TTL) - Auto-cleanup on eviction +- Registered before auth middleware ## Development Environment: Vite Proxy @@ -214,8 +217,12 @@ export default defineConfig({ target: 'http://localhost:3100', changeOrigin: true, }, - '/proxy': { - target: 'http://localhost:22500', + '/game-proxy': { + target: 'http://localhost:3100', + changeOrigin: true, + }, + '/game-zip': { + target: 'http://localhost:3100', changeOrigin: true, }, }, @@ -343,7 +350,7 @@ SQLite). **CORS Configuration**: ```typescript -// Backend +// Backend - Restricted CORS for API routes app.use( cors({ origin: process.env.DOMAIN || 'http://localhost:5173', @@ -352,13 +359,9 @@ app.use( }) ); -// Game Service (wide open for cross-domain content) -app.use( - cors({ - origin: '*', - methods: ['GET', 'HEAD', 'OPTIONS'], - }) -); +// Backend - Permissive CORS for game routes (registered before general CORS) +app.use('/game-proxy', cors({ origin: '*', methods: ['GET', 'HEAD', 'OPTIONS'] })); +app.use('/game-zip', cors({ origin: '*', methods: ['GET', 'HEAD', 'OPTIONS'] })); ``` **Request Validation**: diff --git a/docs/02-architecture/system-architecture.md b/docs/02-architecture/system-architecture.md index 00952e8..342099b 100644 --- a/docs/02-architecture/system-architecture.md +++ b/docs/02-architecture/system-architecture.md @@ -2,9 +2,11 @@ ## Overview -Flashpoint Web is a monorepo containing three independent microservices: -frontend (React UI), backend (Express API), and game-service (game file proxy). -Services communicate via HTTP with clean separation of concerns. +Flashpoint Web is a monorepo containing **2 independent services**: frontend (React UI) +and backend (Express API with integrated game file serving). Services communicate via HTTP +with clean separation of concerns. The backend handles metadata, authentication, business +logic, and game file serving with hardened resource management (connection limits, stream +cleanup, download concurrency limits). ## Architecture Diagram @@ -30,13 +32,9 @@ graph TB AuthSvc[AuthService] PlaySvc[PlayTrackingService] Middleware[Auth/RBAC/Logging] - end - - subgraph "Game Service :22500/:22501" - ProxyServer[HTTP Proxy :22500] - ZipServer[GameZip Server :22501] + ProxyHandler[Game Proxy Handler] ZipManager[ZIP Manager] - MimeHandler[MIME Handler] + GameRoutes[Game Routes
/game-proxy/*
/game-zip/*] end subgraph "Data Layer" @@ -60,6 +58,7 @@ graph TB Middleware --> UserSvc Middleware --> AuthSvc Middleware --> PlaySvc + Express --> GameRoutes GameSvc --> FlashpointDB UserSvc --> UserDB @@ -67,23 +66,21 @@ graph TB PlaySvc --> UserDB PlaySvc --> FlashpointDB - Express -.proxy.-> ProxyServer - Express -.proxy.-> ZipServer - - ProxyServer --> FileSystem - ProxyServer -.fallback.-> CDN - ZipServer --> ZipManager + GameRoutes --> ProxyHandler + GameRoutes --> ZipManager + ProxyHandler --> FileSystem + ProxyHandler -.fallback.-> CDN ZipManager --> FileSystem React --> Ruffle - Ruffle -.loads.-> ProxyServer + Ruffle -.loads.-> GameRoutes Vite -.dev proxy.-> Express style Browser fill:#e1f5ff style React fill:#61dafb style Express fill:#90ee90 - style ProxyServer fill:#ffd700 + style GameRoutes fill:#ffd700 style FlashpointDB fill:#ff9999 style UserDB fill:#ff9999 ``` @@ -129,11 +126,11 @@ playlists **Architecture Pattern**: Routes → Middleware → Services → Databases -### Game Service (Ports 22500, 22501) +### Backend Game Serving (Integrated) -**Purpose**: Game file serving and ZIP mounting +**Purpose**: Game file serving and ZIP mounting (integrated into backend) -**HTTP Proxy Server (22500)**: +**Game Proxy Routes** (`/game-proxy/*`): - Serves legacy web content with fallback chain: 1. Local htdocs directory @@ -141,13 +138,26 @@ playlists 3. ZIP archives 4. External CDN fallback 5. Local cache +- Permissive CORS headers for cross-origin game content +- Registered before auth/CORS middleware +- URL sanitization to prevent directory traversal attacks -**GameZip Server (22501)**: +**Game ZIP Routes** (`/game-zip/*`): - Mounts and streams ZIP archives - Zero-extraction design - LRU cache (max 100, 30-min TTL) - Auto-cleanup on eviction +- Integrated with backend service layer +- Supports background downloads from `gameDataSources` in `preferences.json` + +**Resource Management** (server.ts): +- `keepAliveTimeout: 65s` - Prevents idle connections from blocking +- `headersTimeout: 66s` - Must exceed keepAliveTimeout +- `timeout: 120s` - 2-minute max for any request (handles file streaming) +- `maxConnections: 500` - Prevents connection exhaustion +- Stream cleanup on request completion +- Download concurrency limits (configured in preferences) ## Data Flow @@ -330,17 +340,22 @@ All services run on localhost with SQLite databases. - Frontend route-based lazy loading (38% bundle reduction) - React.memo for component optimization (98% fewer re-renders) -- Game Service LRU cache for ZIP mounts (prevents memory leaks) -- Game Service request size limits (1MB max, DoS protection) -- Backend query result caching +- Backend LRU cache for ZIP mounts (prevents memory leaks) +- Backend request size limits (1MB max, DoS protection) +- Backend query result caching (5-minute TTL) +- In-process game file serving (no inter-service HTTP overhead) +- TanStack Query auto-polling (only when downloading, 2s interval) +- Entry index optimization for large game databases +- Connection pooling with timeouts +- Stream cleanup on request completion or error +- Background downloads don't block initial response ### Scaling Strategies (Future) **Horizontal**: - Frontend: Static hosting on CDN (Vercel, Netlify) -- Backend: Multiple instances behind load balancer -- Game Service: Region-specific replicas +- Backend: Multiple instances behind load balancer with shared file storage **Vertical**: @@ -430,7 +445,8 @@ consistent tooling ## Conclusion -Flashpoint Web balances simplicity with scalability, maintaining clean service -separation while enabling efficient inter-service communication. The -three-service design allows independent scaling and deployment while keeping the -codebase maintainable and type-safe. +Flashpoint Web balances simplicity with scalability, maintaining clean separation +of concerns between frontend UI and backend business logic. Game file serving is +integrated into the backend, eliminating inter-service HTTP overhead while +keeping the codebase maintainable and type-safe. The two-service design is +straightforward to deploy and monitor while remaining extensible for future needs. diff --git a/docs/03-backend/README.md b/docs/03-backend/README.md index 082ab53..5496111 100644 --- a/docs/03-backend/README.md +++ b/docs/03-backend/README.md @@ -2,8 +2,8 @@ The backend is a REST API built with Express and TypeScript that serves as the central hub for the Flashpoint Web application. It manages game metadata, user -accounts, authentication, and coordinates with the separate game-service for -content delivery. +accounts, authentication, and serves game content directly via integrated +game-proxying (formerly provided by a separate game-service). ## Overview @@ -23,7 +23,7 @@ content delivery. - Play session tracking with automatic cleanup - Activity logging for auditing - Database hot-reload when Flashpoint Launcher updates -- Game file proxying to separate game-service +- Integrated game content serving (proxy and ZIP mounting) ## Quick Start @@ -44,7 +44,6 @@ npm start # Run production build - `FLASHPOINT_PATH` - Path to Flashpoint installation (all other paths derived automatically) -- `GAME_SERVICE_HOST` - Game service hostname/IP (default: localhost) - `DOMAIN` - Frontend URL (default: http://localhost:5173) See [configuration.md](./configuration.md) for complete environment variable @@ -102,6 +101,15 @@ backend/ │ ├── services/ # Business logic layer │ ├── types/ # TypeScript definitions │ ├── utils/ # Utility functions +│ ├── game/ # Integrated game content serving +│ │ ├── proxy-request-handler.ts # Legacy htdocs proxy +│ │ ├── gamezipserver.ts # ZIP mounting server +│ │ ├── legacy-server.ts # Legacy file serving +│ │ ├── zip-manager.ts # ZIP mounting logic +│ │ ├── cgi/ # CGI/PHP execution +│ │ ├── services/ # Game data services +│ │ ├── utils/ # Game utilities +│ │ └── config.ts # Game config │ └── migrations/ # Database migrations ├── user.db # User database (generated) ├── package.json @@ -120,6 +128,8 @@ Main route groups: - `/api/activities` - Activity logs - `/api/playlists` - User playlists - `/api/domains` - Domain management +- `/game-proxy/*` - Legacy htdocs and game file proxying (registered before auth) +- `/game-zip/*` - ZIP archive mounting and serving (registered before auth) See the [API Reference](../06-api-reference/README.md) for complete API documentation. @@ -156,6 +166,16 @@ documentation. - Response compression with gzip - File watching debouncing (500ms) +**Query Optimization Patterns:** + +- **Window functions for pagination**: `COUNT(*) OVER()` merges count into single query +- **Scalar subqueries**: Multiple aggregates in one SELECT (e.g., distinct counts) +- **Map-based data merging**: O(1) lookups instead of O(N²) .find() loops +- **Direct file access before directory scans**: O(1) fast path with O(N) fallback +- **Parallel file reads**: `Promise.allSettled()` for concurrent I/O operations + +See [architecture.md](./architecture.md#query-optimization) for implementation details. + ## Background Jobs **Abandoned Session Cleanup** @@ -229,8 +249,8 @@ npm test -- --coverage # Coverage report ### Game Files Not Loading -- Verify game-service is running on port 22500/22501 -- Check GAME_SERVICE_HOST in .env (default: localhost) +- Verify game content is being served via `/game-proxy/*` routes +- Check backend logs for game server initialization errors - Ensure FLASHPOINT_PATH is correct ## Development Workflow diff --git a/docs/03-backend/architecture.md b/docs/03-backend/architecture.md index 05869cb..e2835a4 100644 --- a/docs/03-backend/architecture.md +++ b/docs/03-backend/architecture.md @@ -149,7 +149,25 @@ Abstract data access from business logic; centralized query logic. - Non-blocking (setImmediate) - Automatic cleanup (90-day retention) -### 6. Play Session Tracking +### 6. Integrated Game Content Serving + +**Rationale:** + +- Simplifies deployment (single service instead of two) +- Removes inter-service HTTP calls (direct function calls instead) +- Unified error handling and logging +- Shared resource pools (database, file watchers) + +**Implementation:** + +- Game content routes (`/game-proxy/*`, `/game-zip/*`) registered before auth + middleware +- Direct function calls to game server internals (gamezipserver.mountZip(), + legacy proxy handlers) +- Serves legacy htdocs content, game ZIPs, and HTML5 games +- CGI/PHP execution support for interactive content + +### 7. Play Session Tracking **Rationale:** @@ -206,7 +224,17 @@ export class AppError extends Error { - Indexes on foreign keys and frequently queried columns - Pagination to prevent memory issues - Prepared statements for reusable queries -- Selective columns (avoid SELECT \*) +- Selective columns (avoid SELECT *) +- Flash SWF filtering: Flash games without `.swf` launchCommands excluded from browse/search +- Reduced column selection for list views (7 essential fields vs. full detail) + +**Optimization Patterns:** + +- **Scalar subqueries in SELECT**: Merge multiple COUNT queries into one SELECT with scalar subqueries. Example: `SELECT COUNT(*) as total, (SELECT COUNT(DISTINCT x) FROM ...) as count FROM ...` reduces roundtrips +- **Window functions for pagination**: Use `COUNT(*) OVER()` window function to get total count in the same query as paginated results, eliminating separate COUNT query +- **Map-based data merging**: When joining application data with external lookups, build a `Map` for O(1) lookups instead of O(N²) `.find()` loops through arrays +- **Direct file access first**: For file-based data (playlists), try direct path access (e.g., `{id}.json`) before falling back to directory scans. O(1) fast path with O(N) fallback +- **Parallel file I/O**: Use `Promise.allSettled()` to read multiple files concurrently instead of sequential loops ### Transaction Strategy diff --git a/docs/03-backend/configuration.md b/docs/03-backend/configuration.md index d184de7..d9ed87b 100644 --- a/docs/03-backend/configuration.md +++ b/docs/03-backend/configuration.md @@ -27,11 +27,6 @@ export const config = { // - logosPath: ${flashpointPath}/Data/Logos // - playlistsPath: ${flashpointPath}/Data/Playlists // - gamesPath: ${flashpointPath}/Data/Games - - // Game service host (proxy and gamezip URLs derived automatically) - gameServiceHost: process.env.GAME_SERVICE_HOST || 'localhost', - // - proxyUrl: http://${gameServiceHost}:22500 - // - gamezipUrl: http://${gameServiceHost}:22501 // ... more config }; ``` @@ -137,25 +132,6 @@ needed. The edition is displayed as read-only information in the Settings UI (not user-configurable). -### Game Service Configuration - -#### `GAME_SERVICE_HOST` - -- **Type**: string -- **Default**: `localhost` -- **Description**: Hostname/IP of game-service for constructing proxy and - GameZip URLs -- **Usage**: - ```bash - GAME_SERVICE_HOST=localhost # Development - GAME_SERVICE_HOST=game-service # Docker - GAME_SERVICE_HOST=192.168.1.100 # Remote - ``` - -The proxy and GameZip server URLs are automatically constructed as: - -- Proxy: `http://${GAME_SERVICE_HOST}:22500` -- GameZip: `http://${GAME_SERVICE_HOST}:22501` ### External Resources @@ -317,9 +293,6 @@ HOST=0.0.0.0 # All other paths are derived automatically from FLASHPOINT_PATH FLASHPOINT_PATH=D:/Flashpoint -# Game Service -GAME_SERVICE_HOST=localhost - # Database USER_DB_PATH=./user.db @@ -348,9 +321,6 @@ HOST=0.0.0.0 # All other paths are derived automatically from FLASHPOINT_PATH FLASHPOINT_PATH=/mnt/flashpoint -# Game Service -GAME_SERVICE_HOST=game-service - # Database USER_DB_PATH=/data/user.db @@ -383,9 +353,6 @@ HOST=0.0.0.0 # All other paths are derived automatically from FLASHPOINT_PATH FLASHPOINT_PATH=/flashpoint -# Game Service (Docker network) -GAME_SERVICE_HOST=game-service - # Database (Docker volume) USER_DB_PATH=/data/user.db diff --git a/docs/03-backend/services/database-service.md b/docs/03-backend/services/database-service.md index 24c28c2..679a85f 100644 --- a/docs/03-backend/services/database-service.md +++ b/docs/03-backend/services/database-service.md @@ -102,6 +102,14 @@ Close database connection and stop file watching. DatabaseService.close(); ``` +**Called by**: + +- Graceful shutdown handler in `server.ts` (SIGTERM/SIGINT) +- Ensures all database connections properly closed before process exit + +**Important**: Database connections are closed during graceful shutdown to +prevent resource leaks and ensure in-flight queries complete. + ## Usage Examples ### Game Queries @@ -216,6 +224,43 @@ this.watcher = fs.watch(config.flashpointDbPath, (eventType) => { **Debouncing**: Waits 500ms after last change to prevent excessive reloads during Launcher operations. +### Atomic Database Swap Pattern + +**Critical for Thread Safety**: The `syncAndReload()` and `forceSync()` methods +use an atomic swap pattern to prevent race conditions during hot-reload: + +```typescript +// CORRECT: Atomic swap pattern +const newDb = new BetterSqlite3(config.flashpointDbPath); +const oldDb = this.db; +this.db = newDb; // Atomic reference swap +oldDb.close(); // Close old connection +``` + +**Why this matters:** + +- Opens the new database connection FIRST +- Atomically swaps the reference (single operation) +- Then closes the old connection +- Prevents a window where `this.db = null` could cause 500 errors for + concurrent requests + +**What was wrong (before fix):** + +```typescript +// WRONG: Race condition window +this.db.close(); // Closes connection +this.db = null; // Window here where concurrent requests fail! +this.db = new BetterSqlite3(config.flashpointDbPath); // Then reconnects +``` + +**Impact:** + +- Concurrent requests during reload could get `Cannot read property 'prepare' + of null` errors +- Users could see 500 errors briefly during Flashpoint Launcher updates +- With atomic swap, there's no window where `this.db` is null + ### Edge Cases **Launcher Lock**: When Flashpoint Launcher is updating, reload waits 500ms then @@ -247,6 +292,26 @@ error state until file restored. - Debouncing prevents excessive reloads - Modification time check prevents unnecessary work +### Journal Mode and Checkpointing + +**Implementation**: `backend/src/services/DatabaseService.ts` + +The `save()` method no longer runs WAL checkpoint operations because the database +is configured with `journal_mode = DELETE` (not WAL): + +```typescript +// Journal mode: DELETE (not WAL) +// No need for checkpoint operations +this.db.pragma('journal_mode = delete'); +``` + +**Why**: + +- WAL (Write-Ahead Logging) not required for single-process SQLite +- DELETE mode simpler for read-mostly workloads +- Checkpoint operations (sync) would block readers +- No performance benefit in our single-process architecture + ## Troubleshooting ### Database Locked diff --git a/docs/03-backend/services/permission-cache-service.md b/docs/03-backend/services/permission-cache-service.md index 7b33c04..d7d3b8e 100644 --- a/docs/03-backend/services/permission-cache-service.md +++ b/docs/03-backend/services/permission-cache-service.md @@ -162,7 +162,8 @@ PermissionCache.invalidateUser(5); ### invalidateRole(roleId: number): void -Remove role's permission cache entry. +Remove role's permission cache entry AND propagate invalidation to all users +with that role. **Parameters:** @@ -181,6 +182,61 @@ await roleService.updateRolePermissions(2, [1, 2, 3]); PermissionCache.invalidateRole(2); ``` +**How it works:** + +When a role's permissions change, this method: + +1. Clears the specific role's cache entry: `rolePermissionsCache.delete(roleId)` +2. **Clears the entire user permissions cache**: `userPermissionsCache.clear()` + +**Why clear all user cache entries:** + +- Clearing just the role cache entry isn't enough +- Users with that role have cached permissions from the old role permissions +- Next request would still use the old cached user permissions +- By clearing the entire user cache, all users must refetch permissions on + their next request +- This ensures all users with the changed role immediately get fresh + permissions + +**Role Invalidation Propagation:** + +The propagation ensures permission changes immediately cascade to all affected +users: + +1. **Role permission changes** → `invalidateRole(roleId)` called +2. **Role cache entry cleared** → Next role permission lookup queries database +3. **All user cache cleared** → Every user must refetch permissions on next + request +4. **New permissions fetched** → Database queries include new role permissions +5. **Cache repopulated** → Users get fresh permissions immediately + +**Performance impact:** + +- Temporary increase in database queries as users refetch permissions +- Cache hits resume within seconds as users re-populate the cache +- Trade-off: Ensures permission changes are immediately visible + +**Alternative: More granular invalidation:** + +If performance becomes an issue with large user bases: + +```typescript +// Could track users by role and only invalidate those +const usersWithRole = this.db.all( + 'SELECT id FROM users WHERE role_id = ?', + [roleId] +); +for (const user of usersWithRole) { + PermissionCache.invalidateUser(user.id); +} +``` + +But for now, the simpler approach of clearing all user cache is more +reliable. + + + ### clearAll(): void Clear entire cache (both user and role caches). diff --git a/docs/03-backend/services/user-database-service.md b/docs/03-backend/services/user-database-service.md index 2605911..e4c923d 100644 --- a/docs/03-backend/services/user-database-service.md +++ b/docs/03-backend/services/user-database-service.md @@ -60,7 +60,7 @@ private static async createTables(): Promise { ### 3. Migration System -Runs database migrations on startup: +Runs database migrations on startup with transactional safety: ```typescript private static async runMigrations(): Promise { @@ -74,7 +74,14 @@ private static async runMigrations(): Promise { logger.info('[UserDB] Running migration: 002_create-user-settings'); const migrationPath = path.join(__dirname, '../migrations/002_create-user-settings.sql'); const migrationSQL = fs.readFileSync(migrationPath, 'utf-8'); - this.db!.exec(migrationSQL); + + // IMPORTANT: Wrap in transaction for atomicity + const transaction = this.db!.transaction(() => { + this.db!.exec(migrationSQL); + this.recordMigration('002_create-user-settings', migrationSQL); + }); + + transaction(); // Execute atomically } // Check for system_settings table @@ -87,11 +94,38 @@ private static async runMigrations(): Promise { // Run migration to create system_settings table const migrationPath = path.join(__dirname, '../migrations/003_create-system-settings.sql'); const migrationSQL = fs.readFileSync(migrationPath, 'utf-8'); - this.db!.exec(migrationSQL); + + // IMPORTANT: Wrap in transaction for atomicity + const transaction = this.db!.transaction(() => { + this.db!.exec(migrationSQL); + this.recordMigration('003_create-system-settings', migrationSQL); + }); + + transaction(); // Execute atomically } } ``` +**Transaction Safety for Migrations:** + +Each migration execution is wrapped in a better-sqlite3 `db.transaction()` that +includes: + +1. The SQL execution (`db.exec(sql)`) +2. The migration registry insert (`recordMigration()`) + +If either part fails, the entire transaction rolls back, preventing partial +migration states where: + +- SQL executed but not recorded in registry (orphaned changes) +- Migration recorded but SQL never executed (wrong schema state) +- Database left in inconsistent half-migrated state + +This ensures migrations are all-or-nothing: either the migration completely +succeeds or completely fails with no intermediate states. + + + ### 4. Default Admin User Creates admin account if database is empty: diff --git a/docs/04-frontend/README.md b/docs/04-frontend/README.md index fa4aa94..cd719a4 100644 --- a/docs/04-frontend/README.md +++ b/docs/04-frontend/README.md @@ -170,18 +170,19 @@ frontend/ ## Vite Configuration -Dev server proxies `/api/*` to backend: +Dev server proxies both API and game content to backend: ```typescript server: { proxy: { '/api': 'http://localhost:3100', - '/proxy': 'http://localhost:22500' + '/game-proxy': 'http://localhost:3100', + '/game-zip': 'http://localhost:3100' } } ``` -In production, Nginx proxies API requests to the backend container. +In production, Nginx proxies all requests to the backend container. ## Common Commands diff --git a/docs/04-frontend/api-client.md b/docs/04-frontend/api-client.md index fc92dc9..f90ff8a 100644 --- a/docs/04-frontend/api-client.md +++ b/docs/04-frontend/api-client.md @@ -18,9 +18,14 @@ const api = axios.create({ headers: { 'Content-Type': 'application/json', }, + withCredentials: true, // Enable cookie-based auth (refresh token) }); ``` +**Important:** `withCredentials: true` enables automatic cookie handling. The +backend sets the refresh token in an HTTP-only cookie during login, and the +browser automatically includes it in requests. + ## Interceptors ### Request Interceptor @@ -51,8 +56,9 @@ api.interceptors.response.use( originalRequest._retry = true; try { - const refreshToken = useAuthStore.getState().refreshToken; - const tokens = await authApi.refreshToken(refreshToken); + // Refresh token is sent automatically via withCredentials (HTTP-only cookie) + // No need to manually send refresh token + const tokens = await authApi.refreshToken(); useAuthStore.getState().updateAccessToken(tokens.accessToken); originalRequest.headers.Authorization = `Bearer ${tokens.accessToken}`; @@ -69,6 +75,10 @@ api.interceptors.response.use( ); ``` +**Note:** The refresh token is stored in an HTTP-only cookie (sent automatically +by the browser when `withCredentials: true`). The `refreshToken()` method takes +no parameters and relies on the cookie being present. + ## API Modules ### gamesApi @@ -89,11 +99,15 @@ gamesApi.cancelDownload(id: string) ```typescript authApi.login(credentials: LoginCredentials): Promise authApi.register(userData: RegisterData): Promise -authApi.logout(refreshToken: string): Promise -authApi.refreshToken(refreshToken: string): Promise +authApi.logout(): Promise +authApi.refreshToken(): Promise authApi.getMe(): Promise ``` +**Note:** `logout()` and `refreshToken()` no longer take parameters. The refresh +token is automatically sent via HTTP-only cookie when `withCredentials: true` +is set on the axios instance. + ### playlistsApi ```typescript diff --git a/docs/04-frontend/architecture.md b/docs/04-frontend/architecture.md index 60aaab3..ccd3e4d 100644 --- a/docs/04-frontend/architecture.md +++ b/docs/04-frontend/architecture.md @@ -589,6 +589,36 @@ Outputs: - Source maps - Asset hashing for cache busting +## Code Quality Improvements + +Recent improvements to frontend code quality focus on type safety, performance, correctness, and accessibility: + +### Type Safety + +- **Non-null assertions replaced**: `GamePlayerView` and `GameDetailView` now use safe defaults (`id ?? ''`) instead of non-null assertions (`id!`) +- **RufflePlayer types**: Proper TypeScript type declarations for `window.RufflePlayer` via global type augmentation (no `any` casts) + +### Performance Optimizations + +- **useCallback for formatters**: `useDateTimeFormat` formatters wrapped in `useCallback` to prevent unnecessary re-renders in date/time formatting +- **useCallback for relative time**: `formatRelativeTime` utility wrapped in `useCallback` in `GameDetailView` +- **Modal conditional rendering**: `GameCard` now conditionally renders `AddToPlaylistModal` only when open (prevents 50+ hidden modal instances in grid) + +### Correctness & Robustness + +- **Request cancellation**: `gamesApi.search()` now accepts and properly passes `AbortSignal` parameter for request cancellation on component unmount +- **Optimistic update fix**: `useFavorites` optimistic update uses snapshot of current cache instead of re-reading mutated cache +- **useState for state**: `usePlayTracking` returns `useState`-backed values instead of refs to ensure proper React re-renders + +### Accessibility + +- **Keyboard navigation**: `GameCard` clickable areas now have proper keyboard support with `role="link"`, `tabIndex={0}`, and `onKeyDown` handlers for Enter/Space +- **ARIA labels**: Back buttons now include descriptive `aria-label` attributes + +### Framework Compatibility + +- **Vite configuration**: `ErrorBoundary` now uses `import.meta.env.DEV` instead of `process.env.NODE_ENV` for Vite compatibility + ## Testing Strategy ### Unit Tests (Future) diff --git a/docs/04-frontend/components/component-overview.md b/docs/04-frontend/components/component-overview.md index a7dbf33..3db8fba 100644 --- a/docs/04-frontend/components/component-overview.md +++ b/docs/04-frontend/components/component-overview.md @@ -42,6 +42,13 @@ Located in `components/auth/` Located in `components/common/` +- **Breadcrumbs** - Navigation bar with integrated back button and contextual breadcrumb trail + - Now acts as a navigation bar instead of just breadcrumbs + - Integrated back button with vertical divider separator + - Props: `items` (breadcrumb trail), `showBackButton` (default: true) + - Visual: rounded bar with `bg-muted/50`, pill-style breadcrumb items + - Exports types: `BreadcrumbContext`, `PlayerBreadcrumbContext`, `BreadcrumbItem` + - `BreadcrumbContext` moved from `GameCard.tsx` to `Breadcrumbs.tsx` for centralization - **FormattedDate** - Reusable date/time formatting component with user preferences - **CardSizeControl** - Control for adjusting card size in grid views @@ -88,6 +95,8 @@ Located in `components/layout/` Located in `components/library/` and `components/game/` - **GameCard** - Card view for individual games + - Keyboard accessible: `role="link"`, `tabIndex={0}`, and `onKeyDown` handler for Enter/Space keys + - `AddToPlaylistModal` now conditionally renders only when open (prevents 50+ hidden modals in grid) - **GameGrid** - Responsive grid layout for game cards - **GameList** - List view for games - **GameListItem** - Individual list item @@ -99,7 +108,11 @@ Located in `components/library/` and `components/game/` Located in `components/player/` - **GamePlayer** - Universal game player wrapper + - Debug logging moved to `useEffect` to prevent re-renders on every log + - Game launch only proceeds once player is mounted - **RufflePlayer** - Flash game emulator component + - Proper TypeScript type declarations for `window.RufflePlayer` (no `any` casts) + - Supports scale modes: exactfit, noborder, showall (default), noscale ### UI Components diff --git a/docs/04-frontend/components/player-components.md b/docs/04-frontend/components/player-components.md index ec98096..9c56239 100644 --- a/docs/04-frontend/components/player-components.md +++ b/docs/04-frontend/components/player-components.md @@ -79,20 +79,35 @@ function GamePlayerView() { const { data: launchData } = useGameLaunchData(gameId); return ( - + <> + {launchData?.downloading ? ( +
+
+ +

Downloading Game Data...

+

Files are being downloaded automatically...

+
+
+ ) : ( + + )} + ); } ``` +**Note:** Download status is handled by parent view (GamePlayerView), not GamePlayer itself. +See [Download Flow](#download-flow) below. + ### Implementation Details #### Platform Detection @@ -114,12 +129,31 @@ if (!canPlayInBrowser) { } ``` -#### Flash Game Rendering +#### Flash Game Rendering with SWF Extraction + +Flash games with HTML wrapper pages are automatically resolved before passing +to Ruffle: ```typescript -{platform === 'Flash' && contentUrl ? ( +const useResolvedContentUrl = (platform: string, contentUrl?: string) => { + const [resolvedUrl, setResolvedUrl] = useState(contentUrl); + + useEffect(() => { + if (platform === 'Flash' && contentUrl?.endsWith('.html')) { + // Fetch HTML, extract SWF using regex patterns, resolve relative paths + resolveSwfFromHtml(contentUrl).then(setResolvedUrl); + } + }, [platform, contentUrl]); + + return resolvedUrl; +}; + +// In component: +const resolvedUrl = useResolvedContentUrl(platform, contentUrl); + +{platform === 'Flash' && resolvedUrl ? ( ` +2. Object tags: `` +3. Param movie: `` +4. Param src: `` +5. SWFObject v1: `swf = new SWFObject("...\.swf")` +6. SWFObject v2: `.addVariable("...")` +7. Adobe AC_FL_RunContent +8. Generic quotes: `"([^"]*\.swf)"` +9. Relative/absolute path resolution via URL constructor + #### HTML5 Game Rendering ```typescript @@ -262,6 +308,19 @@ Flash emulator component using Ruffle WebAssembly. `D:\Repositories\Personal\flashpoint-web\frontend\src\components\player\RufflePlayer.tsx` +### SWF URL Resolution + +RufflePlayer accepts the `swfUrl` prop after it has been resolved by the parent +GamePlayer component. For Flash games with HTML wrapper pages: + +- Parent component detects HTML content URLs +- Fetches the HTML wrapper +- Extracts the actual SWF URL using regex patterns +- Passes resolved `.swf` URL to RufflePlayer + +This allows RufflePlayer to focus purely on SWF playback without worrying about +wrapper page handling. + ### Props ```typescript @@ -579,8 +638,135 @@ console.log('[Ruffle] Waiting for cleanup to complete...'); - Use Performance tab for memory leaks - Inspect iframe content for HTML5 games +## Content URL Format + +Content URLs are now relative paths served by the integrated backend game +server: + +``` +/game-proxy/[relativePath] # Legacy htdocs or game files +/game-zip/[zipId]/[path] # Files within mounted ZIP archives +``` + +Example URLs: + +``` +/game-proxy/Games/HTML5/example-game/index.html +/game-proxy/Flash/sample.swf +/game-zip/12345/games/example/index.html +``` + +The backend serves game content directly with appropriate CORS headers and MIME +type detection. + +## Navigation in GamePlayerView + +GamePlayerView now includes breadcrumbs navigation (previously had none): + +### Breadcrumb Trail + +The breadcrumb trail shows contextual navigation: + +- **Home → [Context] → Game Title → Play** +- Reads `PlayerBreadcrumbContext` from React Router navigation state +- Passed from `GameDetailView` when user clicks "Play" +- Context examples: "Browse", "Favorites", "Playlist: [Name]", "Search Results" + +### Fallback Navigation + +When accessed via direct URL (no navigation state): + +- **Browse → Game → Play** +- Fallback context is "Browse" + +### Shared Access + +For shared playlists accessed via share link: + +- **Shared → Playlist → Game → Play** +- Special "Shared" context indicates external access + +### Fullscreen Behavior + +- Breadcrumbs hidden in fullscreen mode (same behavior as old back button) +- Controls remain visible until fullscreen is activated +- ESC key exits fullscreen and restores breadcrumbs + +### Implementation + +```typescript +// In GameDetailView - pass context to player +const handlePlayClick = () => { + navigate(`/games/${game.id}/play`, { + state: { + playerContext: { + type: 'browse' | 'favorites' | 'playlist' | 'search', + label: contextLabel, // e.g., "Favorites" or "Playlist: Best Games" + gameTitle: game.title, + }, + }, + }); +}; + +// In GamePlayerView - read context and build breadcrumbs +const location = useLocation(); +const context = location.state?.playerContext as PlayerBreadcrumbContext | undefined; + +const breadcrumbItems: BreadcrumbItem[] = [ + { label: 'Home', href: '/' }, + { label: context?.label ?? 'Browse', href: context?.href ?? '/games' }, + { label: context?.gameTitle ?? game?.title ?? 'Game', href: `/games/${id}` }, + { label: 'Play' }, +]; +``` + +### Visual Design + +- Rounded bar with `bg-muted/50` background +- Integrated back button with vertical divider +- Pill-style breadcrumb items with hover states +- Responsive to fullscreen mode changes + +## Download Flow + +Game downloads are coordinated at the view level (GamePlayerView), not in GamePlayer itself: + +1. **Fetch Launch Data:** `useGameLaunchData` hook calls `GET /api/games/:id/launch` +2. **Download Status:** Backend returns `{ downloading: true }` if game ZIP is being downloaded +3. **Poll for Completion:** Frontend polls every 2 seconds using TanStack Query's conditional `refetchInterval` +4. **Show UI:** GamePlayerView shows download progress while `downloading` is true +5. **Auto-Play:** When download completes, `downloading` becomes false and GamePlayer renders + +```typescript +// In GamePlayerView +const { data: launchData } = useGameLaunchData(id!); + +// This hook uses conditional polling: +// - Polls every 2 seconds if downloading +// - No polling when download is complete +useQuery({ + queryKey: ['game', id, 'launch'], + queryFn: () => gamesApi.getGameLaunchData(id!), + refetchInterval: (query) => query.state.data?.downloading ? 2000 : false, +}); + +// Show download UI or player based on status +{launchData?.downloading ? ( + +) : ( + +)} +``` + +**Benefits:** +- Parent view controls download flow, not player +- Clean separation of concerns +- Player focuses on rendering content +- Polling only happens when needed + ## Further Reading - [GamePlayerView Documentation](../views-routing.md#gameplayerview) -- [Game Service Documentation](../../05-game-service/README.md) +- [State Management - TanStack Query](../state-management/react-query.md) +- [Backend Game Service Documentation](../../03-backend/README.md) - [Ruffle Documentation](https://ruffle.rs/demo/) diff --git a/docs/04-frontend/player-implementation/html5-player.md b/docs/04-frontend/player-implementation/html5-player.md index 505c92b..b4ac4b1 100644 --- a/docs/04-frontend/player-implementation/html5-player.md +++ b/docs/04-frontend/player-implementation/html5-player.md @@ -90,19 +90,19 @@ With current sandbox settings: ## Content URL Structure -HTML5 game content is served through the game-service proxy: +HTML5 game content is served through the integrated backend game server: ``` -http://localhost:22500/[relativePath] +/game-proxy/[relativePath] ``` Example: ``` -http://localhost:22500/Games/HTML5/example-game/index.html +/game-proxy/Games/HTML5/example-game/index.html ``` -The game-service handles: +The backend game server handles: - Serving index.html - Serving game assets (JS, CSS, images) @@ -152,7 +152,7 @@ const [iframeError, setIframeError] = useState(null); - Check browser console for CORS errors - Verify content URL is correct -- Ensure game-service is running +- Ensure backend is running **Issue: Game loads but doesn't work** @@ -164,7 +164,7 @@ const [iframeError, setIframeError] = useState(null); - CORS may be blocking cross-origin requests - Game may need same-origin access -- Check game-service proxy configuration +- Check backend game server configuration ## Performance @@ -256,10 +256,26 @@ console.log('[HTML5 Game] Loaded successfully'); console.error('[HTML5 Game] Load error:', error); ``` +## Related Features + +### Flash Game SWF Extraction + +The GamePlayer component handles Flash games with HTML wrapper pages by +automatically extracting the actual SWF URL. See +[Game Playing - Flash Content Resolution](../../10-features/03-game-playing.md#flash-game-content-url-resolution) +for details on the 9-pattern SWF extraction system. + +### Flash SWF Filter + +Note that Flash games without `.swf` launchCommands are filtered from browse/search +results. See [Common Pitfalls](../../08-development/common-pitfalls.md#flash-games-must-have-swf-launch-commands-to-appear-in-browse) +for more information. + ## Further Reading - [GamePlayer Component](../components/player-components.md) - [Ruffle Player](./ruffle-player.md) -- [Game Service Documentation](../../05-game-service/README.md) +- [Backend Game Server Documentation](../../03-backend/README.md) +- [Game Playing Feature Guide](../../10-features/03-game-playing.md) - [MDN: iframe](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe) - [MDN: iframe sandbox](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#attr-sandbox) diff --git a/docs/04-frontend/state-management/react-query.md b/docs/04-frontend/state-management/react-query.md index 2364b4a..77ac28a 100644 --- a/docs/04-frontend/state-management/react-query.md +++ b/docs/04-frontend/state-management/react-query.md @@ -164,6 +164,34 @@ useQuery({ }); ``` +## Conditional Refetch Intervals + +Poll with dynamic intervals based on data state: + +```typescript +// Example: Poll for game download completion +useQuery({ + queryKey: ['game', id, 'launch'], + queryFn: () => gamesApi.getGameLaunchData(id), + // Poll every 2 seconds if downloading, stop when done + refetchInterval: (query) => query.state.data?.downloading ? 2000 : false, +}); + +// This replaces manual polling logic with declarative TanStack Query patterns +``` + +**Use Cases:** +- Download progress polling (poll while `downloading: true`) +- Live status updates (poll while in progress, stop when complete) +- Real-time data that becomes stale at specific times +- User activity tracking + +**Benefits:** +- Automatic cleanup when condition becomes false +- Respects query cache and stale time +- Deduplicates requests via query client +- Integrates with DevTools + ## Infinite Queries For pagination: diff --git a/docs/04-frontend/state-management/zustand-stores.md b/docs/04-frontend/state-management/zustand-stores.md index 0b3e027..97f5854 100644 --- a/docs/04-frontend/state-management/zustand-stores.md +++ b/docs/04-frontend/state-management/zustand-stores.md @@ -28,10 +28,10 @@ Manages authentication state, user information, and JWT tokens. ```typescript interface AuthState { user: User | null; - accessToken: string | null; - refreshToken: string | null; + accessToken: string | null; // Memory-only, not persisted isAuthenticated: boolean; isGuest: boolean; + isMaintenanceMode: boolean; // Actions setAuth: (user: User, tokens: AuthTokens) => void; @@ -48,6 +48,14 @@ interface AuthState { } ``` +**Important:** + +- `accessToken` is stored in memory only (not persisted to localStorage) +- `refreshToken` is stored in an HTTP-only cookie (handled automatically by the + browser) +- Only `user`, `isAuthenticated`, `isGuest`, and `isMaintenanceMode` are + persisted to localStorage (via `partialize`) + ### Usage ```typescript @@ -72,7 +80,7 @@ function MyComponent() { #### setAuth -Sets authenticated user and tokens after successful login: +Sets authenticated user and access token after successful login: ```typescript const { setAuth } = useAuthStore(); @@ -81,11 +89,16 @@ const handleLogin = async (credentials) => { const response = await authApi.login(credentials); setAuth(response.user, { accessToken: response.accessToken, - refreshToken: response.refreshToken, + expiresIn: response.expiresIn, }); + // Refresh token is automatically stored in HTTP-only cookie by backend }; ``` +**Note:** Only the access token is passed to `setAuth()`. The backend +automatically sets the refresh token in an HTTP-only cookie during login, which +the browser will include in all subsequent requests. + #### setGuestMode Enables guest mode with limited permissions: @@ -94,7 +107,8 @@ Enables guest mode with limited permissions: const { setGuestMode } = useAuthStore(); setGuestMode(); -// Creates a guest user with permissions: ['games.read', 'games.play'] +// Creates a guest user with permissions: ['games.read', 'playlists.read'] +// Note: guests cannot play games (games.play requires authentication) ``` #### clearAuth @@ -112,15 +126,19 @@ const handleLogout = () => { #### updateAccessToken -Updates the access token (used by refresh token flow): +Updates the access token when refreshed via the axios interceptor: ```typescript const { updateAccessToken } = useAuthStore(); -// Called by axios interceptor +// Called by axios interceptor on 401 after successful token refresh updateAccessToken(newAccessToken); ``` +This is called by the response interceptor when a token refresh succeeds. The +refresh token is automatically sent via HTTP-only cookie, and the new access +token is stored in memory only. + ### Permission Helpers #### hasPermission @@ -173,10 +191,17 @@ if (hasAllPermissions(['users.read', 'users.write', 'users.delete'])) { ### Persistence Strategy -The auth store uses a custom storage strategy that separates guest and -authenticated sessions: +The auth store uses selective persistence with the `partialize` middleware: ```typescript +partialize: (state) => ({ + user: state.user, + isAuthenticated: state.isAuthenticated, + isGuest: state.isGuest, + isMaintenanceMode: state.isMaintenanceMode, + // accessToken is intentionally NOT persisted (memory-only) +}); + storage: createJSONStorage(() => ({ getItem: (name) => { // Check sessionStorage first for guest sessions @@ -208,13 +233,46 @@ storage: createJSONStorage(() => ({ })); ``` +**Token Storage Strategy:** + +- **Access Token**: Memory-only in Zustand store, never persisted +- **Refresh Token**: HTTP-only cookie, sent automatically by browser +- **User Info**: Persisted to localStorage (restored on page reload) +- **Guest Sessions**: Stored in sessionStorage only, cleared on browser close + **Benefits:** +- XSS attacks cannot steal access token (memory-only) +- CSRF protection via HTTP-only cookies (auto-sent by browser) - Guest sessions cleared on browser close (sessionStorage) - Authenticated sessions persist across browser restarts (localStorage) -- Prevents guest data from polluting localStorage +- On page reload: if `isAuthenticated` but `accessToken` is null, the app calls + `/auth/refresh` to recover the session using the persisted refresh token + cookie -### Theme Loading on Login +### Session Recovery on Page Reload + +The app implements session recovery using the persisted refresh token cookie: + +```typescript +// In AuthContext or root App component +useEffect(() => { + const { isAuthenticated, accessToken } = useAuthStore.getState(); + + // If authenticated but no access token (page reload), refresh from cookie + if (isAuthenticated && !accessToken) { + authApi + .refreshToken() + .then((tokens) => { + useAuthStore.getState().updateAccessToken(tokens.accessToken); + }) + .catch(() => { + // Refresh failed, clear auth + useAuthStore.getState().clearAuth(); + }); + } +}, []); +``` After successful login, the auth store automatically loads theme settings from the server: @@ -224,7 +282,6 @@ setAuth: (user: User, tokens: AuthTokens) => { set({ user, accessToken: tokens.accessToken, - refreshToken: tokens.refreshToken, isAuthenticated: true, isGuest: false, }); @@ -237,6 +294,16 @@ setAuth: (user: User, tokens: AuthTokens) => { }; ``` +**How it works:** + +1. On login: backend sets refresh token in HTTP-only cookie, user info and + `isAuthenticated` are persisted to localStorage +2. On page reload: Zustand rehydrates from localStorage, checks if + `isAuthenticated && !accessToken` +3. If true: calls `/auth/refresh` (cookie sent automatically), recovers access + token +4. If false or refresh fails: clears auth state and shows login screen + ## useThemeStore Manages theme mode (light/dark/system) and primary color selection. @@ -439,6 +506,24 @@ window }); ``` +### Server Data Validation + +The theme store validates data received from the server before applying it: + +```typescript +// Validates mode against allowed values +if (!['light', 'dark', 'system'].includes(serverMode)) { + // Reject invalid mode, keep current +} + +// Validates primaryColor against known palette +if (!Object.keys(colorPalette).includes(serverColor)) { + // Reject invalid color, keep current +} +``` + +This prevents crashes from malformed server responses or corrupted settings. + ### Persistence Theme state is persisted to localStorage: diff --git a/docs/05-game-service/README.md b/docs/05-game-service/README.md index cde2443..17b7851 100644 --- a/docs/05-game-service/README.md +++ b/docs/05-game-service/README.md @@ -1,25 +1,28 @@ -# Game Service +# Game Content Module -The Game Service is an independent Node.js service that provides game content -delivery with intelligent fallback mechanisms for legacy web content and ZIP -archive serving. +The Game Content Module is integrated into the backend and provides game +content delivery with intelligent fallback mechanisms for legacy web content +and ZIP archive serving. ## Overview -The game-service runs two HTTP servers: +The game content module is now part of the backend Express application. Instead +of running separate HTTP servers on ports 22500/22501, it provides: -1. **HTTP Proxy Server** (Port 22500) - Legacy web content serving with fallback +1. **Express Route** `/game-proxy/*` - Legacy web content serving with fallback chain -2. **GameZip Server** (Port 22501) - ZIP archive streaming +2. **GameZip Server** - ZIP mounting and serving via direct API calls (no HTTP) + +Both integrate into the backend's main port (3100). ## Architecture ``` -┌─────────────────────────────────────────────────────────┐ -│ Game Service │ +┌──────────────────────────────────────────────────────────┐ +│ Backend (Port 3100) │ ├──────────────────────┬──────────────────────────────────┤ -│ HTTP Proxy Server │ GameZip Server │ -│ Port 22500 │ Port 22501 │ +│ Game Proxy Router │ GameZip Server │ +│ /game-proxy/* │ (Singleton, Direct-Call API) │ ├──────────────────────┼──────────────────────────────────┤ │ - Legacy htdocs │ - ZIP mounting (no extraction) │ │ - Game data files │ - Streaming file access │ @@ -40,91 +43,84 @@ The game-service runs two HTTP servers: ## Quick Start -### Installation & Configuration +The game content module is automatically part of the backend. No separate +setup needed: ```bash -cd game-service +# Backend starts all services including game content module +cd backend npm install -cp .env.example .env -# Edit .env with your Flashpoint path -FLASHPOINT_PATH=D:/Flashpoint -PROXY_PORT=22500 -GAMEZIPSERVER_PORT=22501 +npm run dev # Starts on port 3100 ``` -### Development +Game content is served at: + +- Proxy routes: `http://localhost:3100/game-proxy/*` +- Zip mounting API: Handled internally by backend + +## Configuration + +Game content module configuration is in `backend/.env`: ```bash -npm run dev # Start with hot reload -npm run build # Build TypeScript -npm start # Run production build +FLASHPOINT_PATH=D:/Flashpoint +LOG_LEVEL=info ``` +See [configuration.md](./configuration.md) for complete options. + ## Testing ```bash -# Test HTTP proxy -curl http://localhost:22500/http://www.example.com/test.html +# Test proxy via backend +curl http://localhost:3100/game-proxy/http://www.example.com/test.html -# Test GameZip server -curl http://localhost:22501/mounts - -# Mount a ZIP -curl -X POST http://localhost:22501/mount/game-123 \ - -H "Content-Type: application/json" \ - -d '{"zipPath": "D:/Flashpoint/Data/Games/game.zip"}' +# GameZip server is not exposed on HTTP - it's used internally by the proxy ``` -## Service Ports +## Request Endpoints -| Port | Service | Purpose | -| ----- | -------------- | -------------------------------- | -| 22500 | HTTP Proxy | Legacy web content and fallbacks | -| 22501 | GameZip Server | ZIP archive file serving | +The game content module is accessed through the backend: -## Environment Variables +| Route | Purpose | +| ------------------- | -------------------------------- | +| `/game-proxy/*` | Legacy web content and fallbacks | +| `/game-zip/*` | ZIP archive file serving | -| Variable | Default | Description | -| ---------------------- | -------------------------- | -------------------------- | -| PROXY_PORT | 22500 | HTTP proxy server port | -| GAMEZIPSERVER_PORT | 22501 | GameZip server port | -| FLASHPOINT_PATH | D:/Flashpoint | Flashpoint installation | -| LOG_LEVEL | info | Logging verbosity | - -> **Note:** External fallback URLs are configured via `proxySettings.json` in -> the Flashpoint installation directory. +Both run on the backend's main port (3100). ## Request Flow -### HTTP Proxy (Port 22500) +### Game Proxy Router (/game-proxy/*) 1. Try GameZip server (if ZIP mounted) 2. Try local htdocs 3. Try external CDN fallback 4. Return 404 if not found -### GameZip Server (Port 22501) +### GameZip Server (Direct API) -1. Mount ZIP via POST /mount/game-id -2. Request file: GET /http://domain.com/path/file.swf -3. Search mounted ZIPs for content +1. Backend calls GameZip methods: `mountZip()`, `unmountZip()`, etc. +2. Request via proxy router: `/game-proxy/http://domain.com/path/file.swf` +3. GameZip searches mounted ZIPs for content 4. Stream file directly (no extraction) ## Documentation - [architecture.md](./architecture.md) - Architecture and design patterns -- [proxy-server.md](./proxy-server.md) - HTTP Proxy Server (port 22500) -- [gamezip-server.md](./gamezip-server.md) - GameZip Server (port 22501) +- [proxy-server.md](./proxy-server.md) - Proxy Router (`/game-proxy/*`) +- [gamezip-server.md](./gamezip-server.md) - GameZip Server (Direct API) - [legacy-server.md](./legacy-server.md) - Legacy content serving - [configuration.md](./configuration.md) - Configuration reference ## Integration with Backend -The backend delegates file serving to game-service: +The game content module is fully integrated into the backend: -- Backend returns launch URLs pointing to proxy server -- Frontend loads game content from game-service URLs -- Game-service operates independently without backend communication +- Backend loads game content configuration on startup +- Backend mounts ZIPs via direct API calls +- Frontend requests game URLs from backend +- Proxy router serves content on `/game-proxy/*` endpoints ## Security @@ -135,50 +131,33 @@ The backend delegates file serving to game-service: ## Troubleshooting -### Port Already in Use - -```bash -# Find process using port (Windows) -netstat -ano | findstr :22500 - -# Kill process or change port in .env -PROXY_PORT=22510 -``` - ### File Not Found -1. Verify Flashpoint path is correct +1. Verify Flashpoint path is correct in backend `.env` 2. Check if file exists in htdocs directory 3. Verify external fallback URLs are accessible 4. Ensure ZIP is mounted via GameZip API +5. Check backend logs for game content module errors ### CORS Errors -- Verify `ALLOW_CROSS_DOMAIN=true` in `.env` +- Verify `ALLOW_CROSS_DOMAIN=true` in backend `.env` - Check response headers: `curl -I ` - Ensure error responses include CORS headers -## Docker +## Deployment + +The game content module deploys as part of the backend: ```bash -# Build -docker-compose build game-service +# Development +cd backend && npm run dev -# Run -docker-compose up -d game-service - -# Logs -docker-compose logs -f game-service +# Production +cd backend && npm run build && NODE_ENV=production npm start ``` -Set `FLASHPOINT_HOST_PATH` environment variable to point to your Flashpoint -installation. +Game content is served from the backend on port 3100. -## Production Deployment - -```bash -npm run build -NODE_ENV=production npm start -``` - -Monitor logs and set up process manager (PM2, systemd). +For Docker deployment, see the main backend documentation in +`docs/09-deployment/`. diff --git a/docs/05-game-service/architecture.md b/docs/05-game-service/architecture.md index 89fce42..da406dc 100644 --- a/docs/05-game-service/architecture.md +++ b/docs/05-game-service/architecture.md @@ -1,13 +1,13 @@ -# Game Service Architecture +# Game Content Module Architecture This document describes the architectural design and implementation patterns of -the Flashpoint Game Service. +the integrated Game Content Module. ## System Overview -The game-service is designed as a standalone microservice that handles all game -content delivery, replacing the original Go-based FlashpointGameServer with a -TypeScript/Node.js implementation. +The game content module is integrated into the backend Express application, +replacing the original standalone game-service with TypeScript/Node.js +implementations of game content delivery components. ### Design Principles @@ -19,69 +19,80 @@ TypeScript/Node.js implementation. 4. **Performance**: Streaming, connection pooling, and caching for optimal speed 5. **Compatibility**: 199+ MIME types and HTML polyfills for legacy content -## Dual-Server Architecture +## Integrated Architecture ``` -┌────────────────────────────────────────────────────────────────┐ -│ Game Service Process │ -│ │ -│ ┌──────────────────────────┐ ┌──────────────────────────┐ │ -│ │ HTTP Proxy Server │ │ GameZip Server │ │ -│ │ Port 22500 │ │ Port 22501 │ │ -│ ├──────────────────────────┤ ├──────────────────────────┤ │ -│ │ ProxyRequestHandler │ │ GameZipServer │ │ -│ │ ↓ │ │ ↓ │ │ -│ │ LegacyServer │ │ ZipManager │ │ -│ │ ↓ │ │ ↓ │ │ -│ │ - Local htdocs │ │ - node-stream-zip │ │ -│ │ - Override paths │ │ - In-memory ZIP index │ │ -│ │ - External CDN │ │ - Streaming access │ │ -│ │ - Local cache │ │ │ │ -│ └──────────────────────────┘ └──────────────────────────┘ │ -│ │ -│ ┌──────────────────────────────────────────────────────────┐ │ -│ │ Shared Components │ │ -│ ├──────────────────────────────────────────────────────────┤ │ -│ │ ConfigManager | MimeTypes | Logger | HTMLInjector │ │ -│ └──────────────────────────────────────────────────────────┘ │ -└────────────────────────────────────────────────────────────────┘ +┌──────────────────────────────────────────────────────────────────┐ +│ Backend Application (Port 3100) │ +│ │ +│ ┌────────────────────────┐ ┌────────────────────────────┐ │ +│ │ Game Proxy Router │ │ GameZip Server │ │ +│ │ /game-proxy/* │ │ (Singleton, Direct API) │ │ +│ ├────────────────────────┤ ├────────────────────────────┤ │ +│ │ ProxyRequestHandler │ │ GameZipServer │ │ +│ │ ↓ │ │ ↓ │ │ +│ │ LegacyServer │ │ ZipManager │ │ +│ │ ↓ │ │ ↓ │ │ +│ │ - Local htdocs │ │ - node-stream-zip │ │ +│ │ - Override paths │ │ - In-memory ZIP index │ │ +│ │ - External CDN │ │ - Streaming access │ │ +│ │ - Local cache │ │ │ │ +│ └────────────────────────┘ └────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────┐ │ +│ │ Shared Components │ │ +│ ├────────────────────────────────────────────────────────┤ │ +│ │ ConfigManager | MimeTypes | Logger | HTMLInjector │ │ +│ └────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────┐ │ +│ │ Services │ │ +│ ├────────────────────────────────────────────────────────┤ │ +│ │ PreferencesService | GameDataDownloader │ │ +│ └────────────────────────────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────────┘ ``` -### Why Two Servers? +### Architecture Changes from Standalone Service -1. **Different Request Patterns**: - - Proxy server: High-throughput, varied content types, fallback logic - - GameZip server: API-driven, mount/unmount operations, direct ZIP access +1. **No Separate HTTP Servers**: + - Proxy runs as Express router, not standalone HTTP server + - GameZip uses direct method calls instead of HTTP endpoints + - Both use backend's port 3100 -2. **Independent Scaling**: - - Each server can be configured with different timeouts - - GameZip can fail gracefully without affecting proxy server - - Future: Can run on different hosts if needed +2. **Single-Process Design**: + - Simplifies deployment (one Docker container) + - Shared memory for configuration and caching + - Unified logging and error handling -3. **Clear Separation**: - - Proxy: Content delivery with external fallbacks - - GameZip: ZIP management and streaming +3. **Direct API Integration**: + - ProxyRequestHandler calls ZipManager directly + - Backend calls GameZip methods: `mountZip()`, `unmountZip()`, etc. + - No network latency between components ## Component Architecture -### 1. HTTP Proxy Server (Port 22500) +### 1. Game Proxy Router (/game-proxy/*) -**Purpose**: Serve legacy web content with intelligent fallback chain +**Purpose**: Serve legacy web content with intelligent fallback chain (Express +router) + +**Location**: `backend/src/game/` **Components**: -- `http-proxy-server.ts`: Express server setup and configuration +- `proxy-router.ts`: Express route handler - `proxy-request-handler.ts`: Request routing and response handling - `legacy-server.ts`: Fallback chain implementation **Request Flow**: ``` -Request +Request: /game-proxy/http://domain.com/file.swf ↓ Parse URL (proxy-style or standard) ↓ -Try GameZip Server (if available) +Try GameZip Server (direct call) ↓ Try LegacyServer fallback chain ↓ @@ -96,29 +107,33 @@ Return response with CORS headers - Automatic subdomain variation matching (www, core, cdn, etc.) - External CDN fallback with local caching - HTML polyfill injection for game compatibility +- Integrated with backend's middleware and logging -### 2. GameZip Server (Port 22501) +### 2. GameZip Server (Singleton, Direct API) **Purpose**: Mount and serve files from ZIP archives without extraction +**Location**: `backend/src/game/` + **Components**: -- `gamezipserver.ts`: GameZip server implementation +- `gamezipserver.ts`: GameZip server implementation (singleton) - `zip-manager.ts`: ZIP mounting, indexing, and file access -**API Endpoints**: +**Direct API Methods**: -``` -POST /mount/:id Mount a ZIP file -DELETE /mount/:id Unmount a ZIP file -GET /mounts List all mounted ZIPs -GET /* Serve file from mounted ZIP +```typescript +// Called directly by backend, not via HTTP +mountZip(id: string, params: MountParams): Promise +unmountZip(id: string): Promise +listMounts(): Array +findFile(relPath: string): Promise ``` -**Request Flow**: +**Request Flow** (Backend calling GameZip): ``` -Mount Request +Backend calls: gameZipServer.mountZip('game-123', {...}) ↓ Verify ZIP exists ↓ @@ -128,9 +143,9 @@ Store in ZipManager map ↓ Return success -File Request +Proxy requesting file: ↓ -Parse URL for hostname + path +Call: gameZipServer.findFile(relPath) ↓ Search all mounted ZIPs ↓ @@ -147,11 +162,14 @@ Return with MIME type + CORS - Multi-ZIP search with path variation handling - Streaming for large files - Automatic unmount on shutdown +- Single-process memory sharing (no network overhead) ### 3. Configuration Manager **Purpose**: Centralized configuration from environment and proxySettings.json +**Location**: `backend/src/game/config.ts` + **Sources** (in priority order): 1. Environment variables (.env) @@ -169,12 +187,32 @@ class ConfigManager { **Settings Managed**: -- Server ports - File paths (htdocs, games, CGI) - External fallback URLs - Performance tuning (chunk size, timeouts) - Feature flags (CORS, Brotli, CGI) +**Shared Constants**: + +The configuration module exports shared constants used across the game service: + +```typescript +// config.ts +export const MAX_BUFFERED_FILE_SIZE = 50 * 1024 * 1024; // 50MB + +// Used in gamezipserver.ts, zip-manager.ts, legacy-server.ts +``` + +This eliminates magic number duplication and ensures consistent size limits across +all components. + +**Error Differentiation**: Config loading now distinguishes between missing files +(ENOENT, logged as warning) and corrupted JSON (SyntaxError, logged as error). +Both fall back to defaults but with appropriate severity levels for debugging +configuration issues. + +**Initialization**: Called by backend on startup via `backend/src/server.ts` + ### 4. Legacy Server (Fallback Chain) **Purpose**: Implement multi-level fallback for legacy content serving @@ -238,6 +276,38 @@ For a request to `mochibot.com/file.swf`, try: This handles cases where Flashpoint stores files with subdomain prefixes but games request without them. +**Hostname Variation Optimization** (2026-02-07): + +Subdomain variations now skip prefixes that already exist on the hostname to prevent +redundant candidates. For example, `www.example.com` won't generate +`www.www.example.com` as a variation candidate. This reduces the number of path +candidates checked and improves lookup performance. + +**Negative Lookup Cache Management** (2026-02-07): + +The `LegacyServer` includes a static `clearCache()` method for clearing the +negative lookup cache during shutdown or database reloads. The cache uses FIFO +eviction at 5000 entries to bound memory usage while still providing performance +benefits for repeated requests to missing files. + +**External Source URL Safety** + +**Status**: ✅ **Implemented** (2026-02-07) + +**Implementation**: `backend/src/game/legacy-server.ts` + +External source URL resolution now uses the `URL` constructor to prevent path traversal: + +```typescript +// SAFE: URL constructor handles path resolution correctly +const resolved = new URL(relPath, normalizedBase); +if (!resolved.href.startsWith(normalizedBase)) { + throw new Error('URL escapes base'); +} +``` + +**Negative Cache Fix**: Only ENOENT errors are cached in the not-found cache. Validation failures and transient errors are not cached, preventing permanent 404s for valid files. + ### 5. ZIP Manager **Purpose**: Manage ZIP mounting and provide file access without extraction @@ -290,17 +360,43 @@ constructor() { 1. Check if already mounted (prevent duplicates) 2. Verify ZIP file exists on disk 3. Create StreamZip.async instance -4. Store in Map with metadata (id, path, mount time) -5. Log file count for debugging +4. Build entry index: Set of all entry names +5. Store in Map with metadata (id, path, mount time, entryIndex) +6. Log file count for debugging ``` +**Entry Index Optimization**: + +At mount time, ZipManager builds a `Set` of all entry names in the ZIP +for O(1) lookup performance: + +```typescript +// Build index at mount time +const entryIndex = new Set(); +const entries = await zip.entries(); +for (const [name] of entries) { + entryIndex.add(name.toLowerCase()); +} + +// Store with the mounted ZIP +mountedZips.set(id, { zip, entryIndex, ... }); +``` + +**Benefits**: + +- O(1) entry existence checks (vs sequential `entryData()` calls) +- Eliminates unnecessary ZIP I/O for non-existent files +- Faster failure paths (404 responses) +- Minimal memory overhead (one string set per ZIP) + **File Access**: ``` -1. Normalize file path (remove leading slash) +1. Normalize file path (remove leading slash, lowercase) 2. Look up mounted ZIP by ID -3. Call zip.entryData(path) - streams from ZIP -4. Return Buffer directly (no temp files) +3. Check entryIndex.has(normalizedPath) - O(1) lookup +4. If found: Call zip.entryData(path) - streams from ZIP +5. Return Buffer directly (no temp files) ``` **Multi-ZIP Search**: @@ -313,11 +409,133 @@ Try path variations in order: - Legacy/htdocs/{relPath} (full path) For each variation: - Search all mounted ZIPs - Return first match + Search all mounted ZIPs: + Check entryIndex.has() - O(1) lookup + If found: Load and return immediately +Return null if not found in any ZIP ``` -### 6. MIME Type System +### 6. PreferencesService + +**Purpose**: Read Flashpoint preferences.json and expose gameDataSources for +auto-downloading game ZIPs. + +**Location**: `backend/src/game/services/PreferencesService.ts` + +**Architecture**: + +```typescript +class PreferencesService { + private preferences: FlashpointPreferences | null = null; + private lastLoadTime: number = 0; + private readonly CACHE_TTL = 60 * 1000; // 1 minute + + static initialize(flashpointPath: string): PreferencesService; + static getInstance(): PreferencesService; + async loadPreferences(): Promise; + async getGameDataSources(): Promise; + async getDataPacksFolderPath(): Promise; + invalidateCache(): void; +} +``` + +**Configuration Format** (`preferences.json`): + +```json +{ + "gameDataSources": [ + { + "type": "raw", + "name": "Flashpoint Project", + "arguments": ["https://download.flashpointarchive.org/gib-roms/Games/"] + } + ], + "dataPacksFolderPath": "Data/Games" +} +``` + +**Key Features**: + +- 60-second cache with automatic reload +- Singleton pattern for consistent state +- Matches Flashpoint Launcher's preferences format +- Falls back to empty sources if file missing + +### 7. GameDataDownloader + +**Purpose**: Download game data ZIPs from configured sources when not present +locally. + +**Location**: `backend/src/game/services/GameDataDownloader.ts` + +**Architecture**: + +```typescript +class GameDataDownloader { + static getInstance(): GameDataDownloader; + getFilename(gameId: string, dateAdded: string): string; + async download(params: GameDataDownloadParams, onProgress?: DownloadProgressCallback): Promise; + async exists(gameId: string, dateAdded: string, targetDir?: string): Promise; + async getFilePath(gameId: string, dateAdded: string, targetDir?: string): Promise; +} +``` + +**Download Process**: + +``` +1. Generate filename: {gameId}-{timestamp}.zip +2. Get sources from PreferencesService +3. For each source: + ├─ Build URL: baseUrl + filename + ├─ Download to temp file ({filename}.temp) + ├─ Retry up to 3 times with exponential backoff + ├─ Verify SHA256 hash (if provided) + ├─ Move temp to final location + └─ Return success +4. If all sources fail, return error summary +``` + +**Key Features**: + +- Sequential source fallback (like Flashpoint Launcher) +- SHA256 verification for integrity +- Retry logic with exponential backoff (1s, 2s, 4s) +- Temp file handling (atomic writes) +- Progress callbacks for UI updates +- 5-minute timeout, 500MB max file size + +**DNS Rebinding SSRF Prevention** + +**Status**: ✅ **Implemented** (2026-02-07) + +**Implementation**: `backend/src/game/services/GameDataDownloader.ts` + +Before connecting to download sources, DNS resolution is checked to block private addresses: + +```typescript +const addr = await dns.lookup(hostname); +if (isPrivateAddress(addr.address)) { + throw new Error('Blocked: resolves to private address'); +} +``` + +Blocks 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, and ::1. + +**Filename Generation**: + +Matches Flashpoint Launcher's `getGameDataFilename()`: + +```typescript +getFilename(gameId: string, dateAdded: string): string { + const cleanDate = dateAdded.includes('T') + ? dateAdded + : `${dateAdded} +0000 UTC`; + const timestamp = new Date(cleanDate).getTime(); + return `${gameId}-${timestamp}.zip`; +} +``` + +### 8. MIME Type System **Purpose**: Provide accurate content-type headers for 199+ file types @@ -348,11 +566,11 @@ getMimeType(ext: string): string { - VRML (.wrl) → model/vrml - Chemical formats (.pdb, .mol) → chemical/x-\* -### 7. CORS Utilities +### 9. CORS Utilities **Purpose**: Centralized CORS header management -**Location**: `game-service/src/utils/cors.ts` +**Location**: `backend/src/game/utils/cors.ts` **Benefits**: @@ -413,7 +631,7 @@ justified because: See `docs/12-reference/cors-security-decision.md` for detailed security rationale. -### 8. HTML Polyfill Injector +### 10. HTML Polyfill Injector **Purpose**: Inject compatibility polyfills into HTML files for game engines @@ -457,22 +675,22 @@ needsUnityPolyfills(html: string): boolean { ## Data Flow -### Complete Request Flow (Proxy Server) +### Complete Request Flow (Proxy Router) ``` 1. Frontend requests game content - GET http://localhost:22500/http://www.example.com/game.swf + GET http://localhost:3100/game-proxy/http://www.example.com/game.swf -2. Proxy server receives request +2. Backend proxy router receives request - Method: GET - - URL: http://www.example.com/game.swf + - URL path: /game-proxy/http://www.example.com/game.swf 3. ProxyRequestHandler.handleRequest() - Parse URL: hostname=www.example.com, path=/game.swf -4. Try GameZip server (port 22501) - - Make internal request: http://localhost:22501/http://www.example.com/game.swf - - If 200: Return file immediately +4. Try GameZip server (direct method call) + - Call: gameZipServer.findFile('www.example.com/game.swf') + - If found: Return file immediately - If not found: Continue to legacy server 5. LegacyServer.serveLegacy() @@ -506,49 +724,51 @@ needsUnityPolyfills(html: string): boolean { ``` 1. Backend mounts ZIP for game - POST http://localhost:22501/mount/game-123 - Body: {"zipPath": "D:/Flashpoint/Data/Games/Flash/G/game.zip"} + Call: gameZipServer.mountZip('game-123', { + zipPath: 'D:/Flashpoint/Data/Games/Flash/G/game.zip', + gameId: 'game-123', + dateAdded: '2024-01-15T10:30:00.000Z', + sha256: 'ABC123...' + }) -2. GameZipServer.handleMount() +2. GameZipServer.mountZip() - Extract mount ID: game-123 - - Parse body: zipPath - - Call zipManager.mount(id, zipPath) + - Call zipManager.mount(id, zipPath, params) 3. ZipManager.mount() + - Check if already mounted + - If not: trigger auto-download if ZIP not found locally - Verify ZIP exists - Create StreamZip instance - Store in Map: mountedZips.set('game-123', {...}) - Return success -4. Frontend requests file from mounted ZIP - GET http://localhost:22501/http://www.example.com/game.swf +4. Proxy requests file from mounted ZIP + Call: gameZipServer.findFile('www.example.com/game.swf') -5. GameZipServer.handleFileRequest() - - Parse URL: hostname=www.example.com, path=/game.swf - - Build relPath: www.example.com/game.swf - - Call zipManager.findFile(relPath) - -6. ZipManager.findFile() +5. ZipManager.findFile() - Try path variations: a. content/www.example.com/game.swf b. htdocs/www.example.com/game.swf c. www.example.com/game.swf - - For each variation, search all mounted ZIPs + - For each variation, search all mounted ZIPs: + - Check entryIndex.has() - O(1) lookup + - If found: Load file - Found in 'game-123': content/www.example.com/game.swf -7. ZipManager.getFile() +6. ZipManager.getFile() - Access mounted ZIP - Call zip.entryData('content/www.example.com/game.swf') - Stream file data into Buffer - Return {data: Buffer, mountId: 'game-123'} -8. Process response +7. Process response - Detect MIME type from extension: .swf - If HTML: Inject polyfills - Add CORS headers - Add X-Source: gamezipserver:game-123 -9. Send to client +8. Send to client - Status: 200 - Content-Type: application/x-shockwave-flash - X-Source: gamezipserver:game-123 @@ -574,29 +794,215 @@ Future improvements: **Memory**: -- ZIP indexes are loaded into memory -- Large ZIPs (>1GB) take ~500MB RAM for index -- Solution: Lazy loading, LRU cache for ZIP handles +- ZIP entry indexes: ~100KB per 10,000 files +- Large ZIPs (>1GB) take ~5-50MB RAM for index +- Solution: Lazy loading, LRU cache for ZIP handles, bounded at 100 ZIPs **CPU**: - MIME type lookup: O(1) hash map - Path candidates: O(n) where n = variations -- ZIP search: O(m × p) where m = mounted ZIPs, p = path variations -- Solution: Cache results, reduce variations +- ZIP search: O(m) × O(1) = O(m) where m = mounted ZIPs (via entry index) +- Solution: Entry index optimizations eliminate sequential I/O **Network**: -- External downloads: 45s timeout -- Concurrent limit: 10 requests -- Keep-alive: 65s -- Solution: Increase timeouts for slow connections +- External downloads: 15s timeout (reduced from 45s) +- Download concurrency: Limited to 3 simultaneous downloads +- Keep-alive: 65s TCP timeout, 66s header timeout, 120s total timeout +- Connection pool: 500 max connections, 10 per host with axios +- Solution: Adaptive timeouts, connection pooling **Disk I/O**: - ZIP reads: Random access via node-stream-zip +- Stream cleanup: Destroyed on client disconnect (`res.on('close')`) - Local cache: Sequential writes -- Solution: SSD recommended, RAM disk for cache +- Solution: SSD recommended, RAM disk for cache, bounded resource cleanup + +## Code Quality & Security Improvements (Second Pass Review) + +The following medium and low severity fixes were implemented to improve code +quality and maintainability: + +### HTML Injection & Logging +- `htmlInjector.ts` now uses project logger instead of `console.log` +- Dead code branch removed from polyfill injection logic +- Improves consistency with logging across the module + +### PreferencesService Initialization +- Logs warning on re-initialization with different path +- Prevents silent configuration changes that could affect game downloads +- Helps identify configuration issues during development + +### Schema Validation +- Hostname schema minimum length corrected to 2 (matches regex requirement) +- Enforces consistency between validation rules and runtime expectations + +### Regex Pattern Fixes +- `pathSecurity.ts` regex patterns no longer use `/g` flag with `test()` +- Prevents stateful matching bugs that could skip validation on consecutive calls +- Single-flag patterns are checked per call, not maintained across calls + +### Code Cleanup +- Unused exports removed from: + - `mimeTypes.ts` - Cleaning up unused MIME type exports + - `validation/schemas.ts` - Removing unused Zod schema exports + - `utils/cors.ts` - Removing unused utility functions + - `utils/pathSecurity.ts` - Removing unused security utilities +- Reduces module surface area and maintenance burden + +### Download Validation +- `GameDataDownloader.getFilename()` now validates timestamps +- Rejects NaN timestamps to prevent corrupted filenames +- Ensures valid download filename format + +### Resource Leak Prevention +- `GameDataDownloader` destroys response stream on size limit +- Prevents resource leaks when downloads exceed maximum size +- Proper cleanup when download constraints are violated + +### DRY Principle +- `GameDataService` uses `GameDataDownloader.getFilename()` instead of duplicating logic +- Centralizes filename generation, preventing consistency issues + +### Additional Low-Priority Improvements (2026-02-07) + +**Error Handling & Streaming Safety**: +- `proxy-request-handler` `sendError()` now guards against `headersSent` to prevent write-after-headers errors in streaming scenarios +- `collectRequestBody` now uses `settled` flag to prevent double resolve/reject when both `end` and `error` events fire + +**Visibility & Encapsulation**: +- `findActiveDownload` in `gamezipserver.ts` is now private (was publicly accessible but only used internally) +- `PRIVATE_IP_RANGES` in `GameDataDownloader.ts` is now `static readonly` + +**Module Boundaries**: +- Removed unused imports and convenience re-exports from service index files for cleaner module boundaries +- Internal-only constants like `CUSTOM_MIME_TYPES` no longer exported, reducing public API surface + +## Security Hardening + +The game content module implements multiple defense-in-depth security measures: + +### Authentication & Authorization + +- Game-zip management endpoints require JWT authentication +- `settings.update` permission required for mount/unmount operations +- Mount IDs validated via Zod schema (no path traversal characters) + +### XSS Prevention + +- HTML polyfills injected only in final `sendResponse()` (prevents double + injection) +- Download loading page values escaped via `escapeHtml()` function: + - Progress percentages + - Source names from gameDataSources + - Elapsed time values + - File size displays + +### File Size & Memory Protection + +- ZIP entry size limits: Files exceeding 50MB rejected to prevent memory + exhaustion +- Request body size limits: Maximum 1MB for ZIP mount requests +- Response size limits: Maximum 100MB for external CDN downloads + +### Race Condition Prevention + +- `mountingInProgress` Set prevents concurrent mount race conditions +- Duplicate mount requests return immediately instead of creating redundant + file handles + +### ZIP Handle Leak Prevention + +**Status**: ✅ **Implemented** (2026-02-07) + +**Implementation**: `backend/src/game/zip-manager.ts` + +ZIP file handles are now properly closed when mount operations fail after opening: + +```typescript +const zip = new StreamZip.async({ file: zipPath }); +try { + const entries = await zip.entries(); + // Build entry index... + mountedZips.set(id, { zip, entryIndex, ... }); +} catch (error) { + await zip.close(); // Prevent file handle leak + throw error; +} +``` + +### Graceful Shutdown Safety + +**Status**: ✅ **Implemented** (2026-02-07) + +**Implementation**: `backend/src/game/zip-manager.ts` + +A `shuttingDown` flag prevents new mounts during shutdown and `unmountAll()` waits for in-progress mounts: + +```typescript +private shuttingDown = false; + +async mount(id, zipPath) { + if (this.shuttingDown) throw new Error('Server shutting down'); + // ... mount logic +} + +async unmountAll() { + this.shuttingDown = true; + // Wait up to 10s for in-progress mounts to complete + await waitForInProgressMounts(10000); + // Close all mounted ZIPs +} +``` + +### Download & Redirect Security + +- Maximum 5 redirects allowed for external downloads (prevents SSRF chains) +- Response bodies properly drained before following redirects +- Timeout enforcement: 15 seconds for external downloads +- User-Agent header added to external requests for tracking + +### Downloaded File Path Re-validation + +**Status**: ✅ **Implemented** (2026-02-07) + +**Implementation**: `backend/src/game/gamezipserver.ts` + +After downloading a game ZIP, the file path is re-validated to ensure it stays within the target directory: + +```typescript +const resolvedPath = path.resolve(result.filePath); +if (!resolvedPath.startsWith(path.resolve(targetPath))) { + throw new Error('Downloaded file outside target directory'); +} +``` + +### CGI Symlink Bypass Prevention + +**Status**: ✅ **Implemented** (2026-02-07) + +**Implementation**: `backend/src/game/cgi/cgi-executor.ts` + +CGI script paths are resolved via `fs.realpath()` before security validation, preventing symlink-based directory escapes: + +```typescript +const realPath = await fs.realpath(scriptPath); +const validPath = sanitizeAndValidatePath(cgiBaseDir, realPath); +``` + +### Path Validation + +- ZIP mount paths validated to stay within `Data/Games/` directory +- No absolute path escapes or directory traversal sequences allowed +- Path normalization prevents Unicode/encoding attacks + +### Error Message Sanitization + +- Internal error details removed from responses (prevents path/version leakage) +- Sanitized via `sanitizeErrorMessage()` in logs and responses +- External CDN responses validated: 200-only accepted ## Error Handling @@ -748,17 +1154,61 @@ Prevents attacks like: - Preflight requests handled (OPTIONS) - Max-Age: 86400 (24 hours) +### collectRequestBody Double-Settlement Fix + +**Status**: ✅ **Implemented** (2026-02-07) + +**Implementation**: `backend/src/game/proxy-request-handler.ts` + +The `collectRequestBody` function now uses a `settled` flag to prevent double resolve/reject of its Promise, which could occur when both `end` and `error` events fire: + +```typescript +private async collectRequestBody(req: IncomingMessage): Promise { + return new Promise((resolve, reject) => { + let settled = false; + let body = ''; + + req.on('data', (chunk) => { + body += chunk.toString(); + }); + + req.on('end', () => { + if (!settled) { + settled = true; + resolve(body); + } + }); + + req.on('error', (error) => { + if (!settled) { + settled = true; + reject(error); + } + }); + }); +} +``` + +This prevents unhandled promise rejections when both events fire in rapid succession. + ## Monitoring and Observability ### Logging Strategy **Levels**: -- DEBUG: Path candidates, ZIP searches, file access +- DEBUG: Path candidates, ZIP searches, file access, **per-request file lookups** + (downgraded from INFO in 2026-02-07) - INFO: Requests, mounts, file serves, sources - WARN: Fallback attempts, missing files - ERROR: Server errors, mount failures, fatal errors +**Per-Request Logging** (2026-02-07): + +Granular per-file file lookups are now logged at DEBUG level instead of INFO +to reduce log noise on high-traffic servers while still providing detailed +information when debugging specific issues. + **Log Format**: ``` diff --git a/docs/05-game-service/configuration.md b/docs/05-game-service/configuration.md index 34ad9e9..3c96959 100644 --- a/docs/05-game-service/configuration.md +++ b/docs/05-game-service/configuration.md @@ -1,62 +1,33 @@ # Configuration -The game-service configuration system uses environment variables and -`proxySettings.json` to provide flexible deployment options. +The game content module configuration is part of the backend configuration. The +game-service is no longer a separate service, so these settings are now in +`backend/.env`. ## Configuration Sources Configuration is loaded from three sources in priority order: -1. **Environment variables** (.env file) - Highest priority +1. **Environment variables** (backend .env file) - Highest priority 2. **proxySettings.json** (Flashpoint standard) - Medium priority 3. **Hardcoded defaults** - Lowest priority ```typescript // Priority: ENV > proxySettings.json > defaults -const proxyPort = parseInt(process.env.PROXY_PORT || '22500'); +const chunkSize = parseInt(process.env.PROXY_CHUNK_SIZE || '8192'); ``` -## Environment Variables +## Environment Variables (backend/.env) -### Server Ports +Game content module shares the backend's configuration. All game-service-related +variables are now in `backend/.env`. -#### PROXY_PORT - -HTTP Proxy Server port. - -```bash -PROXY_PORT=22500 -``` - -**Default**: 22500 **Type**: Integer **Range**: 1024-65535 **Required**: No - -**Use case**: Change if port conflict occurs - -**Examples**: - -```bash -PROXY_PORT=22500 # Default -PROXY_PORT=8080 # Alternative HTTP port -PROXY_PORT=3000 # Development port -``` - -#### GAMEZIPSERVER_PORT - -GameZip Server port. - -```bash -GAMEZIPSERVER_PORT=22501 -``` - -**Default**: 22501 **Type**: Integer **Range**: 1024-65535 **Required**: No - -**Use case**: Change if port conflict occurs - -### Flashpoint Paths +### Flashpoint Paths (Shared with Backend) #### FLASHPOINT_PATH -Flashpoint installation root directory. +Flashpoint installation root directory. Used by both backend and game content +module. ```bash FLASHPOINT_PATH=D:/Flashpoint @@ -89,6 +60,7 @@ All other Flashpoint paths are automatically derived from `FLASHPOINT_PATH`: - HTDOCS: `${FLASHPOINT_PATH}/Legacy/htdocs` - Games: `${FLASHPOINT_PATH}/Data/Games` +- CGI: `${FLASHPOINT_PATH}/Legacy/cgi-bin` You do not need to set environment variables for these paths - they are calculated automatically from `FLASHPOINT_PATH`. @@ -119,6 +91,33 @@ PROXY_CHUNK_SIZE=65536 # High memory, low CPU - Smaller chunks: Lower memory, higher CPU, more I/O calls - Larger chunks: Higher memory, lower CPU, fewer I/O calls +### Server Connection Limits + +Server-wide connection and timeout management: + +**HTTP Server Configuration:** + +- **keepAliveTimeout**: 65 seconds - TCP keep-alive socket timeout +- **headersTimeout**: 66 seconds - HTTP header timeout +- **timeout**: 120 seconds - Total request timeout +- **maxConnections**: 500 - Maximum concurrent connections + +These settings are hardcoded to prevent resource exhaustion and ensure graceful +handling of slow clients and network issues. + +### Download Concurrency + +Game data downloads are limited to prevent resource exhaustion: + +- **MAX_CONCURRENT_DOWNLOADS**: 3 - Maximum simultaneous downloads +- Returns HTTP 503 if limit is exceeded during auto-download +- Downloads are queued and retried automatically + +This limit prevents: +- Excessive bandwidth usage from multiple simultaneous downloads +- Memory exhaustion from buffering large files +- Network saturation on the download source + #### ALLOW_CROSS_DOMAIN Enable CORS headers. @@ -155,11 +154,19 @@ If the file is missing, defaults to: - `https://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs/` - `https://infinity.unstable.life/Flashpoint/Legacy/htdocs/` +**External Request Configuration:** + +- **Timeout**: 15 seconds (EXTERNAL_REQUEST_TIMEOUT_MS) +- **Max Retries**: 2 attempts per URL (reduced from 3) +- **Connection Pool**: 10 sockets per host with Keep-Alive +- **User-Agent**: Custom header identifying the proxy + **Notes**: - HTTP URLs are auto-upgraded to HTTPS - URLs are tried in order - First successful response is used +- Failed sources are skipped with exponential backoff ### Logging @@ -235,6 +242,25 @@ requests to `.php`, `.php5`, `.phtml`, and `.pl` files in the `htdocs` or #### CGI Configuration (via proxySettings.json) +The `PreferencesService` is a singleton that reads Flashpoint's `preferences.json` +file. Re-initialization with a different path now logs a warning to help identify +configuration issues. This is primarily useful during development when testing +with different Flashpoint installations. + +**Re-initialization Warning**: + +If you call `PreferencesService.initialize()` with a different path after +initialization, a warning is logged: + +``` +[PreferencesService] Warning: Re-initializing with different path + Previous: /path/to/flashpoint1 + New: /path/to/flashpoint2 +``` + +This helps prevent silent configuration changes that could affect game data +downloads. + Additional CGI settings can be configured in `proxySettings.json`: ```json @@ -264,6 +290,82 @@ The CGI executor implements several security measures: 4. **Timeout enforcement**: Scripts are killed after the configured timeout 5. **Size limits**: Both request body and response size are limited +## preferences.json (Game Data Sources) + +### File Location + +``` +${FLASHPOINT_PATH}/preferences.json +``` + +**Example**: `D:/Flashpoint/preferences.json` + +### Purpose + +The `preferences.json` file is the Flashpoint Launcher's main configuration file. +The game-service reads `gameDataSources` from this file to enable auto-downloading +of game ZIPs when they're not present locally. + +### Game Data Sources Configuration + +```json +{ + "gameDataSources": [ + { + "type": "raw", + "name": "Flashpoint Project", + "arguments": ["https://download.flashpointarchive.org/gib-roms/Games/"] + }, + { + "type": "raw", + "name": "Backup Mirror", + "arguments": ["https://backup.example.com/games/"] + } + ], + "dataPacksFolderPath": "Data/Games" +} +``` + +### Configuration Fields + +#### gameDataSources + +Array of download sources for game ZIP files. + +| Field | Type | Description | +| ----------- | -------- | ---------------------------------------------- | +| `type` | String | Source type (currently only "raw" is used) | +| `name` | String | Display name for logging and error messages | +| `arguments` | String[] | Array with base URL as first element | + +**Download URL Construction:** + +``` +Full URL = arguments[0] + filename +Example: https://download.flashpointarchive.org/gib-roms/Games/ + abc123-1234567890.zip +``` + +**Multi-Source Fallback:** + +Sources are tried sequentially in array order. If one fails (network error, +404, hash mismatch), the next source is attempted. + +#### dataPacksFolderPath + +Relative path from `FLASHPOINT_PATH` to the game data folder. + +**Default**: `Data/Games` + +**Full path**: `${FLASHPOINT_PATH}/${dataPacksFolderPath}` + +### Compatibility + +This configuration format matches the Flashpoint Launcher exactly, so existing +Flashpoint installations should work without modification. The game-service +reads the same `preferences.json` that the Flashpoint Launcher uses. + +--- + ## proxySettings.json ### File Location @@ -369,69 +471,49 @@ MAD4FP content URLs. ## Configuration Examples -### Minimal Configuration +### Minimal Configuration (backend/.env) ```bash -# .env FLASHPOINT_PATH=D:/Flashpoint ``` All other settings use defaults. -### Development Configuration +### Development Configuration (backend/.env) ```bash -# .env FLASHPOINT_PATH=D:/Flashpoint -PROXY_PORT=22500 -GAMEZIPSERVER_PORT=22501 LOG_LEVEL=debug NODE_ENV=development ``` -Verbose logging for debugging. +Verbose logging for debugging. Game content module runs on backend port 3100. -### Production Configuration +### Production Configuration (backend/.env) ```bash -# .env FLASHPOINT_PATH=/opt/flashpoint -PROXY_PORT=22500 -GAMEZIPSERVER_PORT=22501 LOG_LEVEL=warn NODE_ENV=production ALLOW_CROSS_DOMAIN=true ``` -Optimized for production. External fallback URLs are configured via `proxySettings.json`. +Optimized for production. External fallback URLs are configured via +`proxySettings.json`. -### Docker Configuration +### Docker Configuration (backend/.env) ```bash -# .env FLASHPOINT_PATH=/flashpoint -PROXY_PORT=22500 -GAMEZIPSERVER_PORT=22501 LOG_LEVEL=info NODE_ENV=production ``` -Mounted volume path. +Mounted volume path. Game content module runs on same container as backend. -### Custom Ports Configuration +### High-Performance Configuration (backend/.env) ```bash -# .env -PROXY_PORT=8080 -GAMEZIPSERVER_PORT=8081 -``` - -Alternative ports to avoid conflicts. - -### High-Performance Configuration - -```bash -# .env PROXY_CHUNK_SIZE=65536 # 64KB chunks LOG_LEVEL=error # Minimal logging NODE_ENV=production # Production optimizations @@ -439,10 +521,9 @@ NODE_ENV=production # Production optimizations Optimized for throughput. -### Secure Configuration +### Secure Configuration (backend/.env) ```bash -# .env ALLOW_CROSS_DOMAIN=false # No CORS ENABLE_CGI=false # No script execution LOG_LEVEL=warn # Log security events @@ -452,7 +533,7 @@ Maximum security (breaks most games). ## Loading Configuration -### ConfigManager Class +### ConfigManager Class (in backend/src/game/) ```typescript export class ConfigManager { @@ -476,25 +557,27 @@ const proxySettings = JSON.parse(await fs.readFile(proxySettingsPath, 'utf-8')); // 2. Derive paths automatically from FLASHPOINT_PATH this.settings = { - proxyPort: parseInt(process.env.PROXY_PORT || '22500'), // All paths are derived automatically from flashpointPath: legacyHTDOCSPath: path.join(flashpointPath, 'Legacy', 'htdocs'), gamesPath: path.join(flashpointPath, 'Data', 'Games'), + cgiPath: path.join(flashpointPath, 'Legacy', 'cgi-bin'), // ... etc }; ``` ### Access Configuration +Configuration is initialized in `backend/src/server.ts`: + ```typescript -import { ConfigManager } from './config'; +import { ConfigManager } from '@/game/config'; // Load configuration (once on startup) -await ConfigManager.loadConfig(flashpointPath); +await ConfigManager.loadConfig(process.env.FLASHPOINT_PATH!); // Access settings const settings = ConfigManager.getSettings(); -console.log(`Proxy port: ${settings.proxyPort}`); +console.log(`HTDOCS path: ${settings.legacyHTDOCSPath}`); ``` ## Validation @@ -587,18 +670,19 @@ npm test ## Docker Configuration -### Environment Variables +Game content module is deployed as part of the backend container. Configuration +is in `backend/.env`. + +### Backend Service Configuration (docker-compose.yml) ```yaml -# docker-compose.yml services: - game-service: + backend: environment: - FLASHPOINT_PATH=/flashpoint - - PROXY_PORT=22500 - - GAMEZIPSERVER_PORT=22501 - LOG_LEVEL=info - NODE_ENV=production + - JWT_SECRET=your-secret-here ``` ### Volume Mounts @@ -606,17 +690,10 @@ services: ```yaml volumes: - ${FLASHPOINT_HOST_PATH}:/flashpoint:ro - - ./game-service:/app + - ./backend:/app ``` -### Build Args - -```dockerfile -ARG PROXY_PORT=22500 -ARG GAMEZIPSERVER_PORT=22501 -ENV PROXY_PORT=${PROXY_PORT} -ENV GAMEZIPSERVER_PORT=${GAMEZIPSERVER_PORT} -``` +Game content module uses the same Flashpoint volume mount as backend. ## Troubleshooting @@ -626,12 +703,13 @@ ENV GAMEZIPSERVER_PORT=${GAMEZIPSERVER_PORT} **Debug steps**: -1. Check .env file exists in correct location -2. Verify dotenv is loading: `console.log(process.env.PROXY_PORT)` +1. Check .env file exists in backend root directory (not game-service) +2. Verify dotenv is loading: `console.log(process.env.FLASHPOINT_PATH)` 3. Check for typos in variable names 4. Verify file encoding (UTF-8) +5. Restart backend to reload .env -**Solution**: Ensure .env file is in game-service root directory +**Solution**: Ensure .env file is in backend root directory with correct values ### Path Not Found @@ -648,15 +726,16 @@ ENV GAMEZIPSERVER_PORT=${GAMEZIPSERVER_PORT} ### Port Conflict -**Symptom**: `Port 22500 already in use` +**Symptom**: Backend fails to start, `Port 3100 already in use` **Debug steps**: -1. Find process: `netstat -ano | findstr :22500` -2. Kill process or change port -3. Check for multiple game-service instances +1. Find process: `netstat -ano | findstr :3100` +2. Kill process or change port in backend .env +3. Check for multiple backend instances -**Solution**: Change PROXY_PORT to alternative port +**Solution**: Change backend port (not game content specific) or kill conflicting +process ### External URLs Not Working diff --git a/docs/05-game-service/gamezip-server.md b/docs/05-game-service/gamezip-server.md index 57a41e8..6f4a053 100644 --- a/docs/05-game-service/gamezip-server.md +++ b/docs/05-game-service/gamezip-server.md @@ -1,22 +1,23 @@ -# GameZip Server (Port 22501) +# GameZip Server -The GameZip Server mounts ZIP archives in-memory and serves files directly using -streaming, eliminating disk extraction needs. +The GameZip Server is integrated into the backend as a singleton that mounts ZIP +archives in-memory and serves files directly using streaming, eliminating disk +extraction needs. ## Overview Mounts ZIP archives and serves files using `node-stream-zip` for streaming -without extraction. This saves disk space and improves performance. +without extraction. This saves disk space and improves performance. No longer +runs as a separate HTTP server - now accessed via direct method calls from +within the backend. ## Architecture ``` ┌──────────────────────────────────────────────┐ -│ GameZip Server (22501) │ +│ GameZip Server (Singleton) │ ├──────────────────────────────────────────────┤ -│ HTTP Server (Express) │ -│ ↓ │ -│ GameZipServer Class │ +│ GameZipServer Class (Direct Method API) │ │ ↓ │ │ ZipManager (singleton) │ │ ↓ │ @@ -24,97 +25,205 @@ without extraction. This saves disk space and improves performance. └──────────────────────────────────────────────┘ ``` -## API Endpoints +Called directly from: +- Backend GameService on startup +- Proxy router when serving game content +- Backend routes when managing mounts + +## Direct API Methods + +Instead of HTTP endpoints, GameZip Server provides direct method calls: ### Mount ZIP Archive -```http -POST /mount/:id HTTP/1.1 -Content-Type: application/json +```typescript +// Called from backend +await gameZipServer.mountZip(id: string, params: MountParams) +``` -{"zipPath": "D:/Flashpoint/Data/Games/Flash/G/game.zip"} +Parameters: + +```typescript +interface MountParams { + zipPath: string; // Required: Absolute path to ZIP file + gameId?: string; // Optional: Game UUID (enables auto-download) + dateAdded?: string; // Optional: ISO date (used for filename) + sha256?: string; // Optional: Expected SHA256 hash +} ``` **Response:** -```json -{ "success": true, "id": "game-123", "zipPath": "..." } +Throws on error, returns void on success. + +```typescript +// Example usage +try { + await gameZipServer.mountZip('game-abc123', { + zipPath: 'D:/Flashpoint/Data/Games/Flash/G/game.zip', + gameId: 'game-abc123', + dateAdded: '2024-01-15T10:30:00.000Z', + sha256: 'ABC123...' + }); + console.log('ZIP mounted successfully'); +} catch (error) { + console.error('Mount failed:', error.message); +} +``` + +**Mount Process:** + +On mount, the ZipManager builds an entry index for O(1) file lookups: + +``` +1. Check if already mounted (prevent duplicates) +2. Verify ZIP file exists on disk +3. Create StreamZip.async instance +4. Build Set of all entry names +5. Store in LRU cache with metadata and entryIndex +6. Log file count for debugging ``` **Errors:** -- `400`: Missing mount ID or zipPath -- `500`: ZIP file not found or mount failed +- `ZIP file not found`: Invalid zipPath, ZIP doesn't exist +- `ZIP path outside allowed directory`: Security violation +- `Auto-download failed`: All sources failed to download, 503 if too many downloads +- `Corrupted ZIP`: File read error -### Unmount ZIP Archive +### Auto-Download Feature -```http -DELETE /mount/:id HTTP/1.1 +When a ZIP file doesn't exist locally but `gameId` and `dateAdded` are provided, +the GameZip server automatically downloads it from configured `gameDataSources` +in `preferences.json`. + +**Download flow:** + +``` +1. Check if ZIP exists locally + ↓ Not found +2. Read gameDataSources from preferences.json +3. Construct filename: {gameId}-{timestamp}.zip +4. Try each source sequentially: + ├─ Download to temp file + ├─ Verify SHA256 (if provided) + ├─ Move to Data/Games/ + └─ If failed, try next source +5. Mount the downloaded ZIP +6. Return success with downloaded: true ``` -**Response:** +**Download Reliability Improvements:** -```json -{ "success": true, "id": "game-123" } -``` +- `findActiveDownload()` now correlates downloads with hostname from the request + path instead of returning the first arbitrary download (ensures correct + download status on concurrent requests) +- Stale download cleanup extracted to shared `cleanupStaleDownloads()` method + (DRY principle, shared between functions) +- `new URL()` calls wrapped in try-catch for proper 400 error handling on + malformed URLs (prevents 500 errors) -### List Mounted ZIPs - -```http -GET /mounts HTTP/1.1 -``` - -**Response:** +**Configuration** (`preferences.json`): ```json { - "mounts": [ + "gameDataSources": [ { - "id": "game-123", - "zipPath": "D:/Flashpoint/Data/Games/Flash/G/game.zip", - "mountTime": "2025-01-18T10:30:00.000Z", - "fileCount": 0 + "type": "raw", + "name": "Flashpoint Project", + "arguments": ["https://download.flashpointarchive.org/gib-roms/Games/"] } ] } ``` -### Serve File from ZIP +This matches the Flashpoint Launcher's configuration format, so existing +installations should work automatically. -```http -GET http://www.example.com/path/file.swf HTTP/1.1 +### Unmount ZIP Archive + +```typescript +// Called from backend +const success = await gameZipServer.unmountZip(id: string): Promise ``` -or +**Returns:** -```http -GET /http://www.example.com/path/file.swf HTTP/1.1 +- `true` if unmounted successfully +- `false` if ZIP not mounted or error occurred + +**Example:** + +```typescript +const success = await gameZipServer.unmountZip('game-123'); +if (success) { + console.log('ZIP unmounted'); +} else { + console.log('ZIP not mounted or error'); +} ``` -**Response:** +### List Mounted ZIPs -```http -HTTP/1.1 200 OK -Content-Type: application/x-shockwave-flash -Content-Length: 123456 -Cache-Control: public, max-age=86400 -X-Source: gamezipserver:game-123 -Access-Control-Allow-Origin: * +```typescript +// Called from backend or admin API +const mounts = gameZipServer.listMounts(): Array ``` -## Request Processing +**Returns:** -### URL Format Support +```typescript +interface MountInfo { + id: string; // Mount ID + zipPath: string; // Absolute path to ZIP + mountTime: Date; // When ZIP was mounted + fileCount: number; // Number of files in ZIP +} +``` -GameZip supports three URL formats: +**Example:** -1. **Proxy-style**: `GET http://domain.com/path` -2. **Path-based**: `GET /http://domain.com/path` -3. **Standard**: `GET /path` with Host header +```typescript +const mounts = gameZipServer.listMounts(); +mounts.forEach(mount => { + console.log(`${mount.id}: ${mount.zipPath} (${mount.fileCount} files)`); +}); +``` + +### Find File in Mounted ZIPs + +```typescript +// Called from proxy router +const result = await gameZipServer.findFile(relPath: string): Promise +``` + +**Returns:** + +```typescript +interface FileData { + data: Buffer; // File contents + mountId: string; // Which ZIP it came from +} +``` + +Returns `null` if file not found in any mounted ZIP. + +**Example:** + +```typescript +const result = await gameZipServer.findFile('www.example.com/game.swf'); +if (result) { + console.log(`Found in ${result.mountId}, ${result.data.length} bytes`); +} +``` + +## File Access Process ### Path Search Strategy -For request `www.example.com/path/file.swf`, tries: +When finding a file, GameZip searches for multiple path variations: + +For relative path `www.example.com/path/file.swf`, tries: 1. `content/www.example.com/path/file.swf` (most common) 2. `htdocs/www.example.com/path/file.swf` (standard) @@ -123,38 +232,50 @@ For request `www.example.com/path/file.swf`, tries: First match wins and returns immediately. -## File Serving +### Multi-ZIP Searching with Entry Index Optimization -### Streaming Access +If multiple ZIPs are mounted, GameZip searches all of them efficiently: -Files read directly from ZIP archives without extraction: - -```typescript -async getFile(id: string, filePath: string): Promise { - const mounted = this.mountedZips.get(id); - if (!mounted) return null; - try { - const data = await mounted.zip.entryData(filePath); - return data; - } catch (error) { - return null; - } -} +``` +For each path variation: + For each mounted ZIP: + Check entryIndex.has(variation) - O(1) lookup + If found: + Call zip.entryData(variation) to read file + Return immediately +Return null if not found in any ZIP ``` -**Benefits:** +**Performance Benefits:** -- No disk extraction required -- No temporary files created -- Direct memory buffer access -- Fast random access +- Entry index is a `Set` built at mount time +- O(1) existence checks eliminate sequential I/O +- Failures return quickly (404 responses are fast) +- Failed lookups don't trigger unnecessary ZIP reads + +This allows multiple game ZIPs to be mounted simultaneously with efficient searching. + +## Response Processing + +When files are found and served from GameZip: ### MIME Type Detection -Detected from file extensions with 199+ supported types. +Detected from file extensions with 199+ supported types. See +[mime-types.md](./mime-types.md) for complete list. + +### HTML Polyfill Injection + +HTML files are automatically processed to inject compatibility polyfills: + +- Unity WebGL support (polyfills for older game loaders) +- General browser compatibility shims +- See [html-polyfills.md](./html-polyfills.md) for details ## Response Headers +When serving through the proxy router, responses include: + ### Standard Headers ```http @@ -173,21 +294,14 @@ Access-Control-Allow-Methods: GET, POST, OPTIONS Access-Control-Allow-Headers: * ``` -## HTML Polyfill Injection - -HTML files are automatically processed to inject compatibility polyfills: - -- Unity WebGL support -- General browser compatibility shims - ## Integration with Backend **Workflow:** -1. Backend mounts ZIP: `POST /mount/{game.id}` -2. Backend returns launch config with gameZip URLs -3. Frontend loads game from proxy server -4. Proxy checks GameZip first for fastest access +1. Backend calls: `gameZipServer.mountZip('game-id', params)` +2. Backend returns launch config pointing to `/game-proxy/` URLs +3. Frontend loads game from proxy endpoint +4. Proxy calls: `gameZipServer.findFile(relPath)` 5. GameZip serves file from mounted ZIP ## Error Handling @@ -207,52 +321,58 @@ HTML files are automatically processed to inject compatibility polyfills: ## Testing +GameZip Server is not directly accessible via HTTP. Test through the backend: + ```bash -# Mount ZIP -curl -X POST http://localhost:22501/mount/test-game \ - -H "Content-Type: application/json" \ - -d '{"zipPath": "D:/Flashpoint/Data/Games/test.zip"}' +# Request file through backend proxy +curl http://localhost:3100/game-proxy/http://www.example.com/test.swf -# List mounts -curl http://localhost:22501/mounts +# Check response headers +curl -I http://localhost:3100/game-proxy/http://www.example.com/test.swf +``` -# Request file -curl http://localhost:22501/http://www.example.com/test.swf +For unit testing, you can test GameZip methods directly in backend tests: -# Check headers -curl -I http://localhost:22501/http://www.example.com/test.swf +```typescript +import { gameZipServer } from '@/game'; -# Unmount -curl -X DELETE http://localhost:22501/mount/test-game +// Mount a ZIP +await gameZipServer.mountZip('test-game', { + zipPath: 'D:/Flashpoint/Data/Games/test.zip' +}); + +// Find a file +const result = await gameZipServer.findFile('www.example.com/test.swf'); + +// List mounts +const mounts = gameZipServer.listMounts(); + +// Unmount +await gameZipServer.unmountZip('test-game'); ``` ## Troubleshooting -### Port Already in Use - -```bash -GAMEZIPSERVER_PORT=22502 -``` - ### ZIP Mount Fails 1. Verify path is absolute 2. Check file permissions 3. Ensure ZIP file exists +4. Check backend logs for mount errors ### File Not Found in ZIP -1. List mounted ZIPs: `curl http://localhost:22501/mounts` -2. Check if correct ZIP is mounted -3. Verify file exists in ZIP (extract and check) -4. Check ZIP structure (content/, htdocs/, or root?) +1. Check backend logs to see which ZIP is mounted +2. Verify file exists in ZIP (extract and check) +3. Check ZIP structure (content/, htdocs/, or root?) +4. Verify correct path variations are tried ### Memory Issues -1. Limit number of mounted ZIPs -2. Implement auto-unmount for inactive ZIPs -3. Add file size limits -4. Use streaming for large files +1. Limit number of mounted ZIPs (max 100 default) +2. Implement auto-unmount for inactive ZIPs (feature) +3. Monitor backend memory usage +4. Unmount ZIPs when not needed ## Security @@ -264,6 +384,59 @@ File paths are normalized and validated to prevent directory traversal. Only explicitly mounted ZIPs are accessible. No arbitrary ZIP path access. +### ZIP Entry Size Limits + +Files larger than 50MB are rejected to prevent memory exhaustion attacks: + +```typescript +const entry = entries[normalizedPath]; +if (entry && entry.size > 50 * 1024 * 1024) { + logger.warn(`[ZipManager] File too large to buffer: ${entry.size} bytes`); + return null; +} +``` + +**Protection**: Prevents DoS attacks that attempt to load massive files into +memory. + +### Mount Endpoint Authentication + +POST `/api/game-zip/mount/:id`, DELETE `/api/game-zip/mount/:id`, and +GET `/api/game-zip/mounts` endpoints require: + +- Valid JWT authentication (Bearer token) +- `settings.update` permission +- Valid mount ID (validated via Zod schema) + +**Implementation**: Mount IDs must not contain path traversal sequences (`../`, +`..\\`, or null bytes). + +### XSS Prevention in Download Loading Page + +All values interpolated into the download loading page HTML are sanitized via +`escapeHtml()`: + +```typescript +function escapeHtml(str: string): string { + return str + .replace(/&/g, '&') + .replace(//g, '>') + .replace(/"/g, '"') + .replace(/'/g, '''); +} +``` + +**Protected values**: + +- Progress values: `0-100` +- Source names: From `gameDataSources` configuration +- Elapsed time: `{hours}h {minutes}m {seconds}s` +- File size: Human-readable format + +All values are escaped before rendering in the HTML template to prevent +injection attacks. + ## Performance ### Memory Usage @@ -282,9 +455,9 @@ Only explicitly mounted ZIPs are accessible. No arbitrary ZIP path access. ## Logging ``` -[GameZipServer] POST /mount/game-123 [ZipManager] Mounting ZIP: game-123 → D:/Flashpoint/Data/Games/game.zip [ZipManager] ✓ Mounted ZIP: game-123 (1234 files) +[ZipManager] ✓ Found in game-123: content/www.example.com/game.swf ``` ## Related Documentation diff --git a/docs/05-game-service/html-polyfills.md b/docs/05-game-service/html-polyfills.md index d23c7df..5e1c696 100644 --- a/docs/05-game-service/html-polyfills.md +++ b/docs/05-game-service/html-polyfills.md @@ -419,10 +419,10 @@ Create test HTML file: ``` -Request file: +Request file via backend proxy: ```bash -curl http://localhost:22500/test-unity.html +curl http://localhost:3100/game-proxy/http://test.com/test-unity.html ``` Expected output (excerpt): @@ -463,10 +463,10 @@ Create test HTML file: ``` -Request file: +Request file via backend proxy: ```bash -curl http://localhost:22500/test-regular.html +curl http://localhost:3100/game-proxy/http://test.com/test-regular.html ``` Expected output (excerpt): @@ -487,7 +487,7 @@ Expected output (excerpt): ### Test Non-HTML File ```bash -curl http://localhost:22500/test.swf +curl http://localhost:3100/game-proxy/http://test.com/test.swf ``` Expected: No polyfills injected (binary file returned as-is) diff --git a/docs/05-game-service/legacy-server.md b/docs/05-game-service/legacy-server.md index 1c8a76d..356dcb0 100644 --- a/docs/05-game-service/legacy-server.md +++ b/docs/05-game-service/legacy-server.md @@ -169,6 +169,16 @@ if (settings.enableBrotli && filePath.endsWith('.br')) { - Decompressed: `file.js` - MIME: `text/javascript` +## Dynamic Configuration Loading + +Settings are now fetched fresh via `ConfigManager.getSettings()` at the start of +each `serveLegacy()` call instead of being captured once at construction. This +ensures that configuration changes (such as enabling/disabling Brotli, updating +external URLs, or changing override paths) take effect immediately without +requiring a server restart. + +**Benefit**: Admin users can update proxy settings via the UI without downtime. + ## Error Handling **File Not Found:** If no source has the file, returns 404 after all sources @@ -248,8 +258,8 @@ External fallback URLs are configured here (not via environment variables): mkdir -p "D:/Flashpoint/Legacy/htdocs/test.com" echo "Hello World" > "D:/Flashpoint/Legacy/htdocs/test.com/index.html" -# Request file -curl http://localhost:22500/http://test.com/index.html +# Request file via backend +curl http://localhost:3100/game-proxy/http://test.com/index.html # Expected: Hello World ``` @@ -267,8 +277,44 @@ All paths normalized to prevent directory traversal: ### External CDN - HTTPS enforced -- Timeouts prevent DoS +- Timeouts prevent DoS (15 second timeout) - User-Agent header for tracking +- Maximum 5 redirects allowed (prevents SSRF chains) +- Response bodies properly drained before following redirects + +## Timeout Management + +The `tryExternalSource()` method properly manages timeouts for external requests: + +```typescript +private async tryExternalSource(baseUrl: string, relPath: string) { + // Create abort controller for timeout enforcement + const abortController = new AbortController(); + const timeoutId = setTimeout(() => { + abortController.abort(); + }, EXTERNAL_REQUEST_TIMEOUT_MS); + + try { + const response = await axiosInstance.get(fullUrl, { + signal: abortController.signal, + timeout: EXTERNAL_REQUEST_TIMEOUT_MS, + maxRedirects: EXTERNAL_REQUEST_MAX_REDIRECTS, // 5 max + // ... other options + }); + + // CRITICAL: Clear timeout on success path, not just on error + clearTimeout(timeoutId); + return { data: Buffer.from(response.data), contentType }; + } catch (error) { + clearTimeout(timeoutId); + // ... error handling + } +} +``` + +**Timeout Leak Prevention**: The timeout is properly cleared on the success path, +not just on error. Without this, successful responses could leave dangling +timeouts that would still fire even though the request completed. ## Related Documentation diff --git a/docs/05-game-service/mime-types.md b/docs/05-game-service/mime-types.md index c8236ca..1f03b50 100644 --- a/docs/05-game-service/mime-types.md +++ b/docs/05-game-service/mime-types.md @@ -490,10 +490,11 @@ Unknown file types default to `application/octet-stream`: ```bash # Create test file -echo "test" > test.swf +mkdir -p "D:/Flashpoint/Legacy/htdocs/test.com" +echo "test" > "D:/Flashpoint/Legacy/htdocs/test.com/test.swf" -# Request file -curl -I http://localhost:22500/test.swf +# Request file via backend +curl -I http://localhost:3100/game-proxy/http://test.com/test.swf # Expected headers: # Content-Type: application/x-shockwave-flash @@ -503,34 +504,34 @@ curl -I http://localhost:22500/test.swf ```bash # Flash -curl -I http://localhost:22500/file.swf +curl -I http://localhost:3100/game-proxy/http://test.com/file.swf # Expected: application/x-shockwave-flash # Director -curl -I http://localhost:22500/file.dcr +curl -I http://localhost:3100/game-proxy/http://test.com/file.dcr # Expected: application/x-director # Unity -curl -I http://localhost:22500/file.unity3d +curl -I http://localhost:3100/game-proxy/http://test.com/file.unity3d # Expected: application/vnd.unity # HTML -curl -I http://localhost:22500/file.html +curl -I http://localhost:3100/game-proxy/http://test.com/file.html # Expected: text/html # JavaScript -curl -I http://localhost:22500/file.js +curl -I http://localhost:3100/game-proxy/http://test.com/file.js # Expected: text/javascript # Image -curl -I http://localhost:22500/file.png +curl -I http://localhost:3100/game-proxy/http://test.com/file.png # Expected: image/png ``` ### Test Unknown Type ```bash -curl -I http://localhost:22500/file.unknown +curl -I http://localhost:3100/game-proxy/http://test.com/file.unknown # Expected: application/octet-stream ``` diff --git a/docs/05-game-service/proxy-server.md b/docs/05-game-service/proxy-server.md index 23073c6..8755e63 100644 --- a/docs/05-game-service/proxy-server.md +++ b/docs/05-game-service/proxy-server.md @@ -1,47 +1,27 @@ -# HTTP Proxy Server (Port 22500) +# Game Proxy Router -The HTTP Proxy Server handles all HTTP requests for game content, implementing a -multi-level fallback chain to ensure content availability. +The Game Proxy Router is integrated into the backend Express application and +handles all game content requests, implementing a multi-level fallback chain to +ensure content availability. ## Overview -Replicates the functionality of the original FlashpointGameServer, providing -legacy web content serving with intelligent fallback mechanisms. +The proxy router is now part of the backend on port 3100. It provides legacy web +content serving with intelligent fallback mechanisms, replacing the original +standalone FlashpointGameServer. -## Server Configuration +## Integration + +The proxy router is registered in `backend/src/server.ts`: ```typescript -import { createHTTPProxyServer } from './http-proxy-server'; +import { createGameProxyRouter } from '@/game'; -await createHTTPProxyServer({ - proxyPort: 22500, - legacyHTDOCSPath: 'D:/Flashpoint/Legacy/htdocs', - gameDataPath: 'D:/Flashpoint/Data/Games', - externalFilePaths: [ - 'https://infinity.flashpointarchive.org/Flashpoint/Legacy/htdocs', - ], - allowCrossDomain: true, - chunkSize: 8192, -}); +// Mount proxy router +app.use('/game-proxy', await createGameProxyRouter(config)); ``` -### Server Settings - -```typescript -interface ProxyServerOptions { - proxyPort?: number; // Default: 22500 - legacyHTDOCSPath: string; // Required - gameDataPath?: string; // Optional - externalFilePaths?: string[]; // Optional - allowCrossDomain?: boolean; // Default: true - chunkSize?: number; // Default: 8192 bytes -} -``` - -Server timeouts: - -- `timeout: 120000` (2 minutes for slow downloads) -- `keepAliveTimeout: 65000` (65 seconds keep-alive) +This makes game content accessible at `http://localhost:3100/game-proxy/*` ## Request Handling @@ -49,57 +29,58 @@ Server timeouts: The proxy supports three URL formats: -**1. Proxy-Style Requests** +**1. Proxy-Style Requests (via /game-proxy)** ```http -GET http://www.example.com/path/file.swf HTTP/1.1 -Host: localhost:22500 +GET /game-proxy/http://www.example.com/path/file.swf HTTP/1.1 +Host: localhost:3100 ``` -**2. Path-Based Requests** +**2. Path-Based Requests (via /game-proxy)** ```http -GET /http://www.example.com/path/file.swf HTTP/1.1 -Host: localhost:22500 +GET /game-proxy//http://www.example.com/path/file.swf HTTP/1.1 +Host: localhost:3100 ``` -**3. Standard Requests** +**3. Standard Requests (via /game-proxy)** ```http -GET /path/file.swf HTTP/1.1 -Host: www.example.com +GET /game-proxy/path/file.swf HTTP/1.1 +Host: localhost:3100 ``` +Note: All requests must go through `/game-proxy/` prefix as they're routed by +Express. + ### URL Parsing Logic ```typescript -async handleRequest(req: IncomingMessage, res: ServerResponse) { +router.get('/*', async (req: Request, res: Response) => { + // req.url contains the path after /game-proxy/ + const url = req.params[0]; // Everything after /game-proxy/ + let hostname: string; let urlPath: string; - if (req.url.startsWith('http://') || req.url.startsWith('https://')) { - const targetUrl = new URL(req.url); - hostname = targetUrl.hostname; - urlPath = targetUrl.pathname + targetUrl.search; - } - else if (req.url.startsWith('/http://') || req.url.startsWith('/https://')) { - const urlWithoutSlash = req.url.substring(1); - const targetUrl = new URL(urlWithoutSlash); + if (url.startsWith('http://') || url.startsWith('https://')) { + const targetUrl = new URL(url); hostname = targetUrl.hostname; urlPath = targetUrl.pathname + targetUrl.search; } else { hostname = req.headers.host || 'localhost'; - urlPath = req.url; + urlPath = '/' + url; } -} + // ... process request +}); ``` ## Request Routing **Routing Chain:** -1. Try GameZip Server (if available) - fast path +1. Try GameZip Server (direct method call) - fast path 2. Try Legacy Server (local + external) 3. Send response or error @@ -110,9 +91,7 @@ async handleRequest(req: IncomingMessage, res: ServerResponse) { ```typescript handleOptionsRequest(req, res) { if (settings.allowCrossDomain) { - res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', '*'); + setCorsHeaders(res, settings); res.setHeader('Access-Control-Max-Age', '86400'); } res.writeHead(204); @@ -126,29 +105,39 @@ All responses include CORS headers when enabled: ```http Access-Control-Allow-Origin: * -Access-Control-Allow-Methods: GET, POST, OPTIONS +Access-Control-Allow-Methods: GET, POST, DELETE, OPTIONS Access-Control-Allow-Headers: * ``` +**CORS Headers Utility**: CORS headers now use the shared `setCorsHeaders` +utility from `utils/cors.ts` for consistency. This utility includes all required +methods (GET, POST, DELETE, OPTIONS) and is reused across both proxy servers, +eliminating duplication. + **Why CORS required:** Flash games and web content make cross-domain requests for assets and API calls. ## HTML Polyfill Injection -HTML files are automatically processed to inject compatibility polyfills: +HTML files are automatically processed to inject compatibility polyfills **only +in `sendResponse()`**, not during the GameZip server lookup: ```typescript -private sendResponse(res: ServerResponse, data: Buffer, contentType: string) { - let fileData = data; +private sendResponse(res: ServerResponse, result: LegacyFileResponse, source: string): void { + // ... other headers ... - if (contentType.includes('text/html')) { - fileData = injectPolyfills(data); - logger.info('Injected polyfills into HTML file'); + if (result.data) { + // Buffered response (HTML files that need polyfill injection) + let fileData = result.data; + if (contentType.includes('text/html')) { + fileData = injectPolyfills(result.data); // Only here + logger.info(`Injected polyfills into HTML file`); + } + + res.setHeader('Content-Type', contentType); + res.setHeader('Content-Length', fileData.length); + res.end(fileData); } - - res.setHeader('Content-Type', contentType); - res.setHeader('Content-Length', fileData.length); - res.end(fileData); } ``` @@ -157,6 +146,10 @@ private sendResponse(res: ServerResponse, data: Buffer, contentType: string) { - Unity WebGL support - General browser compatibility shims +**Double Injection Prevention**: HTML files from ZIPs are never processed in +`tryGameZipServer()`. Injection happens only once in `sendResponse()`, preventing +duplicate polyfill scripts in the final response. + ## Response Headers ### Standard Headers @@ -171,6 +164,34 @@ X-Source: local-htdocs Access-Control-Allow-Origin: * ``` +### Stream Cleanup + +The proxy automatically cleans up streams when clients disconnect: + +```typescript +private sendResponse(res: ServerResponse, data: Buffer, contentType: string) { + let sent = false; + + // Clean up on client disconnect + res.on('close', () => { + if (!sent) { + logger.debug('Client disconnected, destroying stream'); + } + }); + + res.setHeader('Content-Type', contentType); + res.setHeader('Content-Length', data.length); + res.end(data); + sent = true; +} +``` + +**Benefits:** + +- Prevents zombie streams on client disconnect +- Frees memory and file handles immediately +- Reduces resource leaks from abandoned connections + ### Custom Headers **X-Source** - Indicates where file was served from: @@ -220,25 +241,22 @@ browsers to hide the actual error message. ## GameZip Integration -Before trying the legacy fallback chain, check if file exists in mounted ZIP: +Before trying the legacy fallback chain, check if file exists in mounted ZIP +via direct method call with entry index optimization: ```typescript private async tryGameZipServer(hostname: string, urlPath: string) { try { - const gameZipServerUrl = `http://localhost:${settings.gameZipPort}`; - const targetUrl = `http://${hostname}${urlPath}`; - const requestUrl = `${gameZipServerUrl}/${targetUrl}`; + const relPath = `${hostname}${urlPath}`; + const result = await gameZipServer.findFile(relPath); - const response = await axios.get(requestUrl, { - responseType: 'arraybuffer', - timeout: 5000, - validateStatus: (status) => status === 200, - }); - - return { - data: Buffer.from(response.data), - contentType: response.headers['content-type'] || 'application/octet-stream' - }; + if (result) { + return { + data: result.data, + contentType: getMimeType(path.extname(urlPath).substring(1)) + }; + } + return null; } catch (error) { return null; } @@ -247,25 +265,60 @@ private async tryGameZipServer(hostname: string, urlPath: string) { **Performance:** -- Local request: <5ms +- Entry index lookups: <0.1ms (O(1) hash set operations) +- File read from ZIP: 1-10ms (streaming via node-stream-zip) - External request: 100-500ms -- Speedup: 20-100x faster for ZIP-served content +- Speedup: 100-5000x faster for ZIP-served content (vs external CDN) +- Failed lookups: <0.1ms (no I/O if entry not in index) ## Performance Optimization -### Keep-Alive Connections +### Connection Management + +The HTTP server is configured with strict connection limits to prevent resource exhaustion: ```typescript -server.keepAliveTimeout = 65000; +server.keepAliveTimeout = 65000; // 65 seconds +server.headersTimeout = 66000; // 66 seconds +server.timeout = 120000; // 120 seconds +server.maxConnections = 500; // Max concurrent connections res.setHeader('Connection', 'keep-alive'); ``` -Reuses TCP connections, reduces handshake overhead. +**Benefits:** + +- Reuses TCP connections, reduces handshake overhead +- Graceful socket management prevents connection leaks +- Bounded resource usage prevents DoS +- Timeouts prevent long-running requests from blocking + +### External Request Connection Pooling + +For external CDN requests, a dedicated axios instance uses connection pooling: + +```typescript +const externalAxios = axios.create({ + httpAgent: new http.Agent({ + maxSockets: 10, // Max 10 sockets per host + keepAlive: true + }), + httpsAgent: new https.Agent({ + maxSockets: 10, + keepAlive: true + }) +}); +``` + +**Benefits:** + +- Reuses connections to CDNs (reduces TLS handshakes) +- Per-host limits prevent overwhelming single CDN +- Bounded by total server max connections (500) ## Logging ``` -[ProxyHandler] GET http://www.example.com/game.swf +[ProxyHandler] GET /game-proxy/http://www.example.com/game.swf [ProxyHandler] Parsed - Host: www.example.com, Path: /game.swf [ProxyHandler] Trying GameZip server... [ProxyHandler] ✓ Served from GameZip @@ -284,54 +337,30 @@ Reuses TCP connections, reduces handshake overhead. 1. Frontend requests `/api/games/123/launch` 2. Backend returns: - `{"launchUrl": "http://localhost:22500/http://www.example.com/game.swf"}` + `{"launchUrl": "http://localhost:3100/game-proxy/http://www.example.com/game.swf"}` 3. Frontend loads game in player iframe 4. Proxy serves content with CORS headers -The proxy operates independently; backend only provides URLs. +The proxy is fully integrated into the backend; configuration comes from backend +`.env`. ## Testing ```bash # Test proxy-style request -curl "http://localhost:22500/http://www.example.com/test.html" - -# Test standard request -curl -H "Host: www.example.com" "http://localhost:22500/test.html" +curl "http://localhost:3100/game-proxy/http://www.example.com/test.html" # Test CORS preflight -curl -X OPTIONS "http://localhost:22500/test.html" -v +curl -X OPTIONS "http://localhost:3100/game-proxy/http://www.example.com/test.html" -v # Check response headers -curl -I "http://localhost:22500/http://www.example.com/test.html" +curl -I "http://localhost:3100/game-proxy/http://www.example.com/test.html" ``` -## Configuration Examples +## Configuration -### Basic - -```typescript -await createHTTPProxyServer({ - proxyPort: 22500, - legacyHTDOCSPath: 'D:/Flashpoint/Legacy/htdocs', - allowCrossDomain: true, -}); -``` - -### Production - -```typescript -await createHTTPProxyServer({ - proxyPort: parseInt(process.env.PROXY_PORT || '22500'), - legacyHTDOCSPath: config.htdocsPath, - gameDataPath: config.gamesPath, - allowCrossDomain: process.env.ALLOW_CROSS_DOMAIN !== 'false', - chunkSize: parseInt(process.env.PROXY_CHUNK_SIZE || '8192'), -}); -``` - -> **Note:** External fallback URLs are loaded from `proxySettings.json` in the -> Flashpoint directory, not from environment variables. +Game proxy configuration is part of the backend configuration. See +[configuration.md](./configuration.md) for details. ## Security @@ -359,6 +388,29 @@ res.setHeader('Access-Control-Allow-Origin', 'https://trusted-domain.com'); - HTTPS enforced - Timeouts prevent DoS +## Download Redirect Limiting + +External CDN requests are limited to a maximum of 5 redirects to prevent SSRF +and infinite redirect loops: + +```typescript +const response = await axiosInstance.get(fullUrl, { + responseType: 'arraybuffer', + timeout: EXTERNAL_REQUEST_TIMEOUT_MS, + maxRedirects: EXTERNAL_REQUEST_MAX_REDIRECTS, // 5 max + // ... other options +}); +``` + +**Feature**: When a redirect is followed, the response body is properly drained: + +```typescript +response.resume(); // Properly drain response before following redirect +``` + +**Protection**: Prevents attackers from creating chains of redirects that waste +resources or bypass security controls. + ## Troubleshooting ### Port Already in Use diff --git a/docs/05-game-service/zip-manager.md b/docs/05-game-service/zip-manager.md index e69bc81..f5f2ec5 100644 --- a/docs/05-game-service/zip-manager.md +++ b/docs/05-game-service/zip-manager.md @@ -244,6 +244,15 @@ async getFile(id: string, filePath: string): Promise { } ``` +**Performance Improvement - findFile() Refactoring:** + +The `findFile()` method has been optimized for single-pass lookup: + +- Entries are read directly without redundant delegation through `getFile()` +- Process: index check → entry existence check → size check → read +- Eliminates intermediate function calls and extra lookups +- Result: Faster file discovery with fewer function invocations + ### File Access Examples ```typescript @@ -627,6 +636,13 @@ const data = await zipManager.getFile('not-mounted', 'file.swf'); // Returns: null (not an error) ``` +**File too large**: + +```typescript +const data = await zipManager.getFile('game-123', 'huge-file.bin'); +// Returns: null if file > 50MB (prevents memory exhaustion) +``` + ### Unmount Errors **ZIP not mounted**: @@ -645,27 +661,22 @@ const success = await zipManager.unmount('game-123'); ## Testing -### Manual Testing +### Testing with Backend + +ZipManager is not directly accessible. Test through the backend proxy: ```bash -# 1. Start game-service -npm run dev +# 1. Start backend +cd backend && npm run dev -# 2. Mount a ZIP -curl -X POST http://localhost:22501/mount/test \ - -H "Content-Type: application/json" \ - -d '{"zipPath": "D:/Flashpoint/Data/Games/test.zip"}' +# 2. Request file through proxy (GameZip is called internally) +curl http://localhost:3100/game-proxy/http://www.example.com/file.swf -# 3. Check mounts -curl http://localhost:22501/mounts - -# 4. Request file -curl http://localhost:22501/http://www.example.com/file.swf - -# 5. Unmount -curl -X DELETE http://localhost:22501/mount/test +# Backend logs will show ZipManager operations ``` +For unit testing, test GameZip methods directly: + ### Unit Testing (Future) ```typescript @@ -744,18 +755,64 @@ Typical performance on modern SSD: - Per ZIP: 50KB - 500KB (index) - Per file: File size + decompression overhead +## Race Condition Protection + +The ZIP manager prevents concurrent mount race conditions through the +`mountingInProgress` Set: + +```typescript +private mountingInProgress = new Set(); + +async mount(id: string, zipPath: string): Promise { + // Check if already mounting + if (this.mountingInProgress.has(id)) { + logger.debug(`ZIP mount already in progress: ${id}`); + return; + } + + this.mountingInProgress.add(id); + try { + // Mount the ZIP + } finally { + this.mountingInProgress.delete(id); + } +} +``` + +**Protection**: Prevents duplicate file handles from being created when multiple +concurrent mount requests arrive for the same ZIP. The second request returns +immediately instead of creating redundant ZipManager entries. + +## ZIP Entry Size Limits + +The `getFile()` method checks entry size before reading to prevent memory +exhaustion: + +```typescript +const entry = entries[normalizedPath]; +if (entry && entry.size > 50 * 1024 * 1024) { + logger.warn(`File too large to buffer: ${normalizedPath} (${entry.size} bytes)`); + return null; +} +``` + +**Limit**: Files exceeding 50MB are rejected. + +**Rationale**: Large files are buffered entirely into memory. Without limits, +malicious or misconfigured ZIPs could exhaust available memory. + +**Future**: Implement streaming for large files instead of buffering. + ## Future Enhancements -1. **Auto-Unmount**: Unmount inactive ZIPs after timeout +1. **Auto-Unmount**: Unmount inactive ZIPs after timeout (already implemented via + LRU TTL) 2. **LRU Cache**: Cache frequently accessed files in memory -3. **File Index**: Pre-index files on mount for O(1) lookup +3. **Streaming API**: Stream large files instead of loading into memory 4. **Compression Detection**: Skip decompression for stored (uncompressed) files -5. **Streaming API**: Stream large files instead of loading into memory -6. **Concurrent Limits**: Limit concurrent file reads -7. **Memory Limits**: Enforce maximum memory usage -8. **Health Monitoring**: Track memory usage, cache hit rate -9. **Metrics**: Expose mount count, file access count, cache hit rate -10. **Pre-Warming**: Mount popular ZIPs on startup +5. **Health Monitoring**: Track memory usage, cache hit rate +6. **Metrics**: Expose mount count, file access count, cache hit rate +7. **Pre-Warming**: Mount popular ZIPs on startup ## References diff --git a/docs/06-api-reference/admin-api.md b/docs/06-api-reference/admin-api.md index 5f96bd7..5d747ce 100644 --- a/docs/06-api-reference/admin-api.md +++ b/docs/06-api-reference/admin-api.md @@ -41,7 +41,7 @@ Returns `200 OK` (healthy) or `503 Service Unavailable` (unhealthy) }, "memory": { "heapUsed": 125829120, "heapTotal": 536870912 }, "services": { - "gameService": { "available": true, "url": "http://localhost:22500" } + "gameService": { "status": "integrated" } } } ``` @@ -237,8 +237,7 @@ maxLoginAttempts: 1-10). Returns file content with appropriate MIME type and CORS headers. -Uses fallback chain: Game Service → Game Service GameZip → External CDN → Local -cache +Uses fallback chain: Backend game-proxy → game-zip → External CDN → Local cache Query parameter `cache=false` bypasses caching. diff --git a/docs/06-api-reference/authentication-api.md b/docs/06-api-reference/authentication-api.md index 743c5ae..49a6d51 100644 --- a/docs/06-api-reference/authentication-api.md +++ b/docs/06-api-reference/authentication-api.md @@ -8,7 +8,8 @@ User authentication, registration, and token management. Body: `{ "username": "string (3-50)", "password": "string (min 6)" }` -Returns `200 OK` with user object and tokens: +Returns `200 OK` with user object and access token. **Refresh token is set as +HTTP-only cookie automatically**. ```json { @@ -21,12 +22,16 @@ Returns `200 OK` with user object and tokens: }, "tokens": { "accessToken": "jwt...", - "refreshToken": "jwt...", - "expiresIn": 3600 + "expiresIn": 900 } } ``` +Response headers also include: +``` +Set-Cookie: fp_refresh=jwt...; Path=/api/auth; HttpOnly; Secure; SameSite=Lax; Max-Age=2592000 +``` + Error: `401 Unauthorized` (invalid credentials) or `429 Too Many Requests` (account locked after failed attempts) @@ -37,7 +42,8 @@ Error: `401 Unauthorized` (invalid credentials) or `429 Too Many Requests` Body: `{ "username": "string (3-50)", "email": "valid-email", "password": "string (min 6)" }` -Returns `201 Created` with user and tokens (same format as login). +Returns `201 Created` with user and tokens (same format as login). **Refresh token +is set as HTTP-only cookie automatically**. Error: `403 Forbidden` (registration disabled) or `409 Conflict` (username/email exists) @@ -46,21 +52,49 @@ exists) `POST /api/auth/refresh` - No auth required -Body: `{ "refreshToken": "jwt..." }` +Body: Empty object `{}` or can be omitted -Returns new accessToken, refreshToken (rotated), expiresIn. +**Refresh token is read from HTTP-only cookie `fp_refresh` automatically.** +The request must include credentials (cookies) to send the refresh token. + +```bash +curl -X POST https://api.example.com/api/auth/refresh \ + -H "Content-Type: application/json" \ + -d '{}' \ + --cookie "fp_refresh=jwt..." +``` + +Returns new `accessToken` and `expiresIn`. **New refresh token is set as HTTP-only +cookie automatically**. + +```json +{ + "accessToken": "jwt...", + "expiresIn": 900 +} +``` Old refresh token automatically revoked for security (prevents token reuse). -Error: `401 Unauthorized` (invalid or expired refresh token) +Error: `401 Unauthorized` (invalid or expired refresh token, or no refresh token +in cookie) ## Logout `POST /api/auth/logout` - Optional auth -Body: `{ "refreshToken": "jwt..." }` +Body: Empty object `{}` or can be omitted -Revokes the refresh token. +**Refresh token is read from HTTP-only cookie `fp_refresh` automatically.** + +```bash +curl -X POST https://api.example.com/api/auth/logout \ + -H "Content-Type: application/json" \ + -d '{}' \ + --cookie "fp_refresh=jwt..." +``` + +Revokes the refresh token and clears the HTTP-only cookie. Returns `{ "success": true }` @@ -74,45 +108,60 @@ Error: `401 Unauthorized` ## Usage Pattern -```javascript +**Frontend (with Axios client configured with `withCredentials: true`)**: + +```typescript // Login -const { data } = await api.post('/auth/login', { +const { data } = await api.post('/api/auth/login', { username: 'john_doe', password: 'password123', }); +// Response contains accessToken only +// Refresh token is in HTTP-only cookie (automatic, not visible to JS) -localStorage.setItem('accessToken', data.tokens.accessToken); -localStorage.setItem('refreshToken', data.tokens.refreshToken); +// Store access token in Zustand store (memory only) +useAuthStore.setState({ accessToken: data.tokens.accessToken }); -// Subsequent requests +// Subsequent requests (cookies sent automatically) const response = await api.get('/api/games'); -// On 401 - refresh token -const refreshed = await api.post('/auth/refresh', { - refreshToken: localStorage.getItem('refreshToken'), -}); +// On 401 - refresh automatically (cookies sent automatically) +const refreshed = await api.post('/api/auth/refresh', {}); +// Response contains new accessToken only +// New refresh token is in HTTP-only cookie (automatic) -localStorage.setItem('accessToken', refreshed.data.accessToken); -localStorage.setItem('refreshToken', refreshed.data.refreshToken); +useAuthStore.setState({ accessToken: refreshed.data.accessToken }); -// Logout -await api.post('/auth/logout', { - refreshToken: localStorage.getItem('refreshToken'), -}); +// Logout (cookies sent automatically) +await api.post('/api/auth/logout', {}); +// Refresh token cookie is cleared by backend -localStorage.removeItem('accessToken'); -localStorage.removeItem('refreshToken'); +// Clear access token from store +useAuthStore.setState({ accessToken: null }); ``` +**axios configuration (required)**: + +```typescript +const api = axios.create({ + baseURL: 'http://localhost:3100', + withCredentials: true, // Enable cookies to be sent with requests +}); +``` + +This ensures cookies are sent with every request to `/api/auth/*` endpoints. + ## Security Best Practices - Use HTTPS in production -- Store tokens securely (httpOnly cookies recommended over localStorage) -- Access tokens expire in 1 hour (configurable) -- Refresh tokens rotate on each use +- Access tokens (15 minutes) stored in memory only via Zustand store +- Refresh tokens (30 days) stored in HTTP-only cookies - not accessible to JavaScript +- HTTP-only cookies prevent XSS attacks from stealing refresh tokens +- Refresh tokens rotate on each use to prevent token reuse attacks - Failed login attempts trigger rate limiting and account lockout - Enforce strong passwords in production (minimum 6 shown, recommend 12+) -- Implement automatic token refresh on 401 responses +- Automatic token refresh on 401 responses via axios interceptor +- CORS with `credentials: true` ensures cookies are sent cross-origin (if applicable) ## Permission System @@ -137,22 +186,55 @@ Common permissions returned in login/me: ## Interceptor Pattern -```javascript +```typescript // Axios interceptor for automatic token refresh api.interceptors.response.use( (response) => response, async (error) => { - if (error.response?.status === 401) { - const refreshToken = localStorage.getItem('refreshToken'); - const { data } = await axios.post('/api/auth/refresh', { refreshToken }); + const originalRequest = error.config; - localStorage.setItem('accessToken', data.accessToken); - localStorage.setItem('refreshToken', data.refreshToken); + if (error.response?.status === 401 && !originalRequest._retry) { + originalRequest._retry = true; - error.config.headers['Authorization'] = `Bearer ${data.accessToken}`; - return axios(error.config); + try { + // Refresh token is sent in HTTP-only cookie automatically + const { data } = await api.post('/api/auth/refresh', {}); + + // Update access token in store (memory only) + useAuthStore.getState().updateAccessToken(data.accessToken); + + // Retry original request with new token + originalRequest.headers['Authorization'] = `Bearer ${data.accessToken}`; + return api(originalRequest); + } catch (refreshError) { + // Refresh failed - clear auth and redirect to login + useAuthStore.getState().clearAuth(); + window.location.href = '/login'; + return Promise.reject(refreshError); + } } + return Promise.reject(error); } ); ``` + +## Cookie Details + +The `fp_refresh` HTTP-only cookie is set with the following attributes: + +| Attribute | Value | Purpose | +| --------- | ----- | ------- | +| `name` | `fp_refresh` | Identifies the refresh token | +| `value` | JWT token | Refresh token | +| `httpOnly` | `true` | Not accessible to JavaScript (XSS protection) | +| `secure` | `true` (prod) | Only sent over HTTPS | +| `sameSite` | `'lax'` | CSRF protection | +| `path` | `/api/auth` | Only sent to auth endpoints | +| `maxAge` | `2592000` | 30 days (in seconds) | +| `domain` | (implicit) | Same domain as backend | + +Example Set-Cookie header: +``` +Set-Cookie: fp_refresh=eyJhbGc...; Path=/api/auth; HttpOnly; Secure; SameSite=Lax; Max-Age=2592000 +``` diff --git a/docs/06-api-reference/games-api.md b/docs/06-api-reference/games-api.md index bd4e131..c32fa15 100644 --- a/docs/06-api-reference/games-api.md +++ b/docs/06-api-reference/games-api.md @@ -13,8 +13,12 @@ Query params: `search` (title/developer/publisher), `platform`, `series`, default: asc), `page` (default: 1), `limit` (default: 50, max: 100), `showBroken` (default: false), `showExtreme` (default: false) -Returns paginated array with standard game fields (id, title, developer, -publisher, platformName, releaseDate, tags, description, etc.). +Returns paginated array with essential game fields (id, title, developer, +platformName, library, tags, orderTitle). + +**Important:** Flash games without `.swf` launchCommands are excluded from +browse/search results. Only Flash games with direct SWF launch commands appear +in results and platform counts. ## Get Filter Options @@ -23,6 +27,9 @@ publisher, platformName, releaseDate, tags, description, etc.). Returns platforms, developers, publishers, series, playModes, languages arrays with name/code and count. +**Note:** Flash platform count reflects only games with `.swf` launchCommands +(playable Flash games). Non-SWF Flash games are excluded from counts. + ## Get Random Game `GET /api/games/random?library=arcade` - No auth required @@ -50,7 +57,44 @@ Error: `404 Not Found` `GET /api/games/:id/launch` - No auth required Returns gameId, title, platform, launchCommand, contentUrl (proxied through -game-service), applicationPath, playMode, canPlayInBrowser. +backend on `/game-proxy/` and `/game-zip/` routes), applicationPath, playMode, +canPlayInBrowser, and downloading status. + +**Response:** + +```json +{ + "gameId": "550e8400-e29b-41d4-a716-446655440000", + "title": "Super Mario Flash", + "platform": "Flash", + "launchCommand": "http://uploads.ungrounded.net/396000/396724_DAplayer_V6.swf", + "contentUrl": "http://localhost:3100/game-proxy/http://uploads.ungrounded.net/396000/396724_DAplayer_V6.swf", + "applicationPath": ":http: :message:", + "playMode": "Single Player", + "canPlayInBrowser": true, + "downloading": false +} +``` + +**Response (while downloading):** + +```json +{ + "gameId": "...", + "title": "Some Game", + "platform": "Flash", + "launchCommand": "...", + "contentUrl": "...", + "canPlayInBrowser": true, + "downloading": true +} +``` + +**Notes:** + +- `downloading: true` indicates game ZIP is being downloaded from configured gameDataSources +- Frontend should poll this endpoint every 2 seconds until `downloading` becomes false +- Backend awaits the mount to complete before returning (not fire-and-forget) ## Get Related Games @@ -89,6 +133,20 @@ await api.get('/games', { ## Game Data Fields +**Search Results (GET /api/games):** + +| Field | Type | Description | +| ------------ | ------------------- | ------------------------------------------ | +| id | string (uuid) | Unique game identifier | +| title | string | Game name | +| developer | string | Developer name | +| platformName | string | Platform (Flash, HTML5, etc.) | +| tagsStr | string | Semicolon-separated tags | +| library | string | "arcade" (games) or "theatre" (animations) | +| orderTitle | string | Title for sorting | + +**Game Details (GET /api/games/:id):** + | Field | Type | Description | | ---------------- | ------------------- | ------------------------------------------ | | id | string (uuid) | Unique game identifier | @@ -96,7 +154,7 @@ await api.get('/games', { | platformName | string | Platform (Flash, HTML5, etc.) | | library | string | "arcade" (games) or "theatre" (animations) | | developer | string | Developer name | -| publisher | string | Publisher name | +| publisher | string | Publisher name (optional in list views) | | releaseDate | string (YYYY-MM-DD) | Original release date | | tags | string | Semicolon-separated tags | | description | string | Full description | @@ -114,3 +172,4 @@ await api.get('/games', { - Cache results locally (important for repeated searches) - Check `canPlayInBrowser` before attempting to play - Use exact tag names from `/api/tags` endpoint +- Note: Flash games require `.swf` launchCommands to appear in browse/search results diff --git a/docs/08-development/README.md b/docs/08-development/README.md index 73fd144..dbc4ba6 100644 --- a/docs/08-development/README.md +++ b/docs/08-development/README.md @@ -44,20 +44,18 @@ npm run build ## Architecture Overview -The project is a monorepo with three independent services: +The project is a monorepo with two independent services: -- **Backend** (port 3100): REST API with Express + TypeScript +- **Backend** (port 3100): REST API with Express + TypeScript (includes game service) - **Frontend** (port 5173): React SPA with Vite + TypeScript -- **Game Service** (ports 22500, 22501): Game content proxy and ZIP server -Each service can be developed, tested, and deployed independently. +Each service can be developed, tested, and deployed independently. The game service is now integrated into the backend as the `/game-proxy/*` and `/game-zip/*` routes. ## Getting Help - Review existing documentation in this directory - Check the main [CLAUDE.md](../../CLAUDE.md) for project overview -- Consult service-specific docs in `docs/03-backend`, `docs/04-frontend`, and - `docs/05-game-service` +- Consult service-specific docs in `docs/03-backend` and `docs/04-frontend` - Review architecture documentation in `docs/02-architecture` ## Contributing diff --git a/docs/08-development/commands.md b/docs/08-development/commands.md index ea2ac3d..8970364 100644 --- a/docs/08-development/commands.md +++ b/docs/08-development/commands.md @@ -12,7 +12,6 @@ Run from repository root. npm run dev # Start all services (concurrent) npm run dev:backend # Backend only npm run dev:frontend # Frontend only -npm run dev:game-service # Game service only ``` ### Installation @@ -21,7 +20,6 @@ npm run dev:game-service # Game service only npm run install:all # Install all service dependencies npm run install:backend # Backend only npm run install:frontend # Frontend only -npm run install:game-service # Game service only ``` ### Building @@ -30,7 +28,6 @@ npm run install:game-service # Game service only npm run build # Build all services for production npm run build:backend npm run build:frontend -npm run build:game-service ``` ### Type Checking @@ -39,15 +36,13 @@ npm run build:game-service npm run typecheck # Type check all services npm run typecheck:backend npm run typecheck:frontend -npm run typecheck:game-service ``` ### Production ```bash -npm start # Start backend + game-service in production +npm start # Start backend in production npm run start:backend -npm run start:game-service ``` ### Code Formatting @@ -57,7 +52,6 @@ npm run format # Format all files with Prettier npm run format:check # Check without changes npm run format:backend npm run format:frontend -npm run format:game-service ``` ### Cleanup @@ -66,7 +60,6 @@ npm run format:game-service npm run clean # Remove all build artifacts + node_modules npm run clean:backend npm run clean:frontend -npm run clean:game-service ``` --- @@ -142,33 +135,6 @@ npm run copy-ruffle # Copy Ruffle emulator to public/ruffle/ --- -## Game Service Commands - -Run from `game-service/` directory. - -### Development - -```bash -npm run dev # Start with hot reload - # Proxy: http://localhost:22500 - # GameZip: http://localhost:22501 -``` - -### Building & Production - -```bash -npm run build # Compile TypeScript to dist/ -npm start # Run production build -``` - -### Code Quality - -```bash -npm run typecheck # Type check only -``` - ---- - ## Docker Commands Run from repository root. @@ -222,8 +188,7 @@ $env:FLASHPOINT_HOST_PATH = "D:\Flashpoint" ```bash npm run install:all cp backend/.env.example backend/.env -cp game-service/.env.example game-service/.env -# Edit .env files with Flashpoint path +# Edit .env with Flashpoint path npm run typecheck npm run dev ``` @@ -259,21 +224,13 @@ npm start ### Backend (.env) -| Variable | Required | Default | Notes | -| ------------------- | -------- | --------------------- | ------------------------------------- | -| `FLASHPOINT_PATH` | Yes | - | All other paths derived automatically | -| `JWT_SECRET` | Yes\* | - | \*Required in production | -| `DOMAIN` | No | http://localhost:5173 | Frontend URL for CORS | -| `GAME_SERVICE_HOST` | No | localhost | Game service hostname | -| `LOG_LEVEL` | No | info | error, warn, info, debug | -| `NODE_ENV` | No | development | production in production | - -### Game Service (.env) - -| Variable | Required | Default | Notes | -| ----------------- | -------- | ------- | ------------------------------------- | -| `FLASHPOINT_PATH` | Yes | - | All other paths derived automatically | -| `LOG_LEVEL` | No | info | error, warn, info, debug | +| Variable | Required | Default | Notes | +| ----------------- | -------- | --------------------- | ------------------------------------- | +| `FLASHPOINT_PATH` | Yes | - | All other paths derived automatically | +| `JWT_SECRET` | Yes\* | - | \*Required in production | +| `DOMAIN` | No | http://localhost:5173 | Frontend URL for CORS | +| `LOG_LEVEL` | No | info | error, warn, info, debug | +| `NODE_ENV` | No | development | production in production | > **Note:** External fallback URLs are configured in `proxySettings.json`. diff --git a/docs/08-development/common-pitfalls.md b/docs/08-development/common-pitfalls.md index 24df934..6a19d85 100644 --- a/docs/08-development/common-pitfalls.md +++ b/docs/08-development/common-pitfalls.md @@ -231,14 +231,14 @@ api.interceptors.response.use( error.config._retry = true; try { - const { data } = await api.post('/api/auth/refresh', { - refreshToken: localStorage.getItem('refreshToken'), - }); - localStorage.setItem('accessToken', data.accessToken); + // Refresh token sent automatically via HTTP-only cookie + const { data } = await api.post('/api/auth/refresh', {}); + // Store new access token in memory (Zustand store) + useAuthStore.getState().updateAccessToken(data.accessToken); error.config.headers['Authorization'] = `Bearer ${data.accessToken}`; return api(error.config); } catch { - localStorage.clear(); + useAuthStore.getState().clearAuth(); window.location.href = '/login'; } } @@ -277,7 +277,7 @@ const { data } = await api.get('/api/games'); # ✓ Has auth headers --- -## Game Service Issues +## Game Loading Issues ### Game Files Not Loading @@ -288,20 +288,68 @@ const { data } = await api.get('/api/games'); # ✓ Has auth headers **Solution:** ```bash -# Verify game-service is running -curl http://localhost:22500/health +# Verify backend is running +curl http://localhost:3100/api/health # Check logs -cd game-service && npm run dev +npm run dev:backend # Verify Flashpoint paths ls $FLASHPOINT_PATH/Legacy/htdocs ls $FLASHPOINT_PATH/Data/Games/ -# Test file access -curl http://localhost:22500/file.swf +# Test game proxy access +curl http://localhost:3100/game-proxy/file.swf ``` +### Game Download Polling Issues + +**Symptom:** + +- Download status doesn't update, stuck showing "Downloading..." +- Download completes but UI doesn't reflect it + +**Root Causes:** + +1. **Polling not activated:** TanStack Query's `refetchInterval` only works if condition is checked properly +2. **Wrong response checking:** Backend returns `downloading: false` but frontend still polls +3. **Query cache not updated:** Old cached value persists after download + +**Solution:** + +```typescript +// CORRECT: Conditional polling based on data state +const { data: launchData } = useQuery({ + queryKey: ['game', id, 'launch'], + queryFn: () => gamesApi.getGameLaunchData(id), + refetchInterval: (query) => { + // Poll every 2 seconds if downloading + return query.state.data?.downloading ? 2000 : false; + }, +}); + +// WRONG: Static polling +const { data: launchData } = useQuery({ + queryKey: ['game', id, 'launch'], + queryFn: () => gamesApi.getGameLaunchData(id), + refetchInterval: 2000, // ❌ Polls forever! +}); + +// WRONG: Not checking response +if (launchData?.downloading) { + // Show downloading UI +} else if (launchData) { + // Show player - but might have cached old response +} +``` + +**Prevention:** + +- Always check `query.state.data?.downloading` in refetchInterval callback +- Use query cache properly - don't manually track state +- Test by monitoring network tab for polling requests +- Verify responses show `downloading: false` when complete + --- ### Ruffle Not Loading @@ -333,6 +381,166 @@ npm run copy-ruffle --- +## Game Service Resource Management Issues + +### Fire-and-Forget Mount Pattern + +**Symptom:** + +- Race conditions when game starts before ZIP is mounted +- Requests fail with "ZIP not mounted" errors +- Intermittent game loading failures + +**Root Cause:** + +Backend calls `mountGameZip()` but doesn't await it: + +```typescript +// WRONG: Fire-and-forget +this.gameZipServer.mountGameZip(gameId, zipPath, dateAdded, sha256); + +// Returns immediately, but mount hasn't completed yet! +return { mounted: true }; +``` + +**Solution:** + +Always await the mount operation: + +```typescript +// CORRECT: Wait for mount to complete +const result = await this.gameZipServer.mountGameZip( + gameId, + zipPath, + dateAdded, + sha256 +); + +if (!result.success) { + throw new Error(`Failed to mount ZIP: ${result.error}`); +} + +return { mounted: true }; +``` + +**Prevention:** + +- Review all mount operations - ensure they're awaited +- Test game launch immediately after mount (no delays) +- Check server logs for "mount timeout" errors + +--- + +### `node-stream-zip` Memory Usage with Large Files + +**Symptom:** + +- High memory consumption when accessing large files in ZIPs +- Application crashes with "out of memory" errors +- Slow performance when extracting file data + +**Root Cause:** + +`entryData()` reads entire file into memory Buffer: + +```typescript +// WRONG: Reads entire file into memory +const data = zip.entryData(filename); +// If file is 500MB, consumes 500MB+ RAM! +``` + +**Solution:** + +Check file size before reading and stream large files: + +```typescript +// CORRECT: Check size first +const entry = zip.entries()[filename]; + +if (!entry) { + throw new Error(`File not found: ${filename}`); +} + +const MAX_BUFFER_SIZE = 100 * 1024 * 1024; // 100MB limit + +if (entry.size > MAX_BUFFER_SIZE) { + // Stream large files instead + return zip.createReadStream(filename); +} else { + // Safe to buffer small files + return zip.entryData(filename); +} +``` + +**Prevention:** + +- Document file size limits for ZIP access +- Monitor memory usage in production +- Set reasonable buffer limits per deployment environment +- Test with various ZIP sizes + +--- + +### LRU Cache File Handle Accumulation + +**Symptom:** + +- "Too many open files" errors after many game loads +- File descriptor exhaustion +- System runs out of available file handles + +**Root Cause:** + +LRU cache's `dispose` callback is async but eviction is sync: + +```typescript +// WRONG: Async cleanup on sync eviction +this.cache.set(cacheKey, zipManager, { + dispose: async (zipManager) => { + // This is async... + await zipManager.close(); // ...but eviction won't wait! + } +}); + +// zipManager.close() fires but doesn't complete +// File handle stays open +// Multiple instances accumulate +``` + +**Solution:** + +Use synchronous cleanup or properly track promises: + +```typescript +// CORRECT: Synchronous cleanup +const closePromises = new Map(); + +this.cache.set(cacheKey, zipManager, { + dispose: (zipManager) => { + // Close synchronously if possible + if (zipManager.close) { + zipManager.close(); // Synchronous close + } + } +}); + +// OR: Track async closures separately +async handleCacheEviction() { + const pendingClosures = Array.from(closePromises.values()); + await Promise.all(pendingClosures); +} +``` + +**Prevention:** + +- Implement synchronous close methods where possible +- Monitor file descriptor count: `lsof -p | wc -l` +- Set appropriate cache size limits +- Manually evict unused entries during maintenance +- Add metrics for open file handles + +--- + ## Frontend Build Issues ### Build Fails with Type Errors @@ -415,6 +623,159 @@ npm run dev ## Performance Issues +### Multiple COUNT Queries Instead of Scalar Subqueries + +**Symptom:** + +- Slow paginated responses with separate COUNT queries +- Multiple roundtrips to database for pagination results +- Example: `/api/users?page=1` makes 2 queries (SELECT + COUNT) + +**Root Cause:** + +Separate queries for data and total count: + +```typescript +// WRONG: 2 queries (inefficient) +const users = db.all('SELECT * FROM users LIMIT 50 OFFSET 0'); +const total = db.get('SELECT COUNT(*) as total FROM users'); +return { users, total: total.total }; +``` + +**CORRECT - Merge with window function or scalar subquery:** + +```typescript +// CORRECT: 1 query with window function +const result = db.all( + `SELECT u.*, + COUNT(*) OVER() as total_count + FROM users u + ORDER BY u.created_at DESC + LIMIT ? OFFSET ?`, + [limit, offset] +); + +const total = result.length > 0 ? result[0].total_count : 0; +const users = result.map(u => { delete u.total_count; return u; }); + +// OR: Scalar subquery +const result = db.all( + `SELECT u.*, + (SELECT COUNT(*) FROM users) as total_count + FROM users u + ORDER BY u.created_at DESC + LIMIT ? OFFSET ?`, + [limit, offset] +); +``` + +**Prevention:** + +- Always fetch count in the same query as data +- Use window functions (`COUNT(*) OVER()`) for paginated queries +- Use scalar subqueries for single column aggregates +- Profile queries: `EXPLAIN QUERY PLAN` should show one main scan, not two + +--- + +### O(N²) Data Merging with .find() in Loops + +**Symptom:** + +- Slow API responses when returning joined data (favorites with game details, playlists with games) +- Response time degrades proportionally with O(N²) +- Example: Getting 100 favorites with 1000 total games is 100,000 comparisons + +**Root Cause:** + +Using `.find()` loop to match related data: + +```typescript +// WRONG: O(N²) complexity +const favoriteGames = favorites.map(fav => { + const game = games.find(g => g.id === fav.gameId); // Searches through entire games array + return { ...fav, game }; +}); +``` + +**CORRECT - Use Map for O(1) lookups:** + +```typescript +// CORRECT: O(N) with Map +const gamesMap = new Map(games.map(g => [g.id, g])); // Build index once + +const favoriteGames = favorites.map(fav => { + const game = gamesMap.get(fav.gameId); // O(1) lookup + return { ...fav, game }; +}); +``` + +**Prevention:** + +- Build a `Map` when merging data from two lists +- Never use `.find()` inside loops over potentially large datasets +- Profile response times: should be linear O(N+M), not quadratic O(N*M) + +--- + +### Directory Scans Instead of Direct File Access + +**Symptom:** + +- File operations are slow (playlists, user data files) +- Speed degrades as file count increases +- All filesystem lookups take O(N) time + +**Root Cause:** + +Always scanning the directory instead of trying direct path first: + +```typescript +// WRONG: Always scans directory +async getPlaylist(id: string) { + const files = await fs.readdir(playlistsPath); + for (const file of files) { + const content = await fs.readFile(path.join(playlistsPath, file)); + const playlist = JSON.parse(content); + if (playlist.id === id) return playlist; + } +} +``` + +**CORRECT - Try direct path first:** + +```typescript +// CORRECT: O(1) fast path, O(N) fallback +async getPlaylist(id: string) { + // Try direct path first (most common case) + try { + const content = await fs.readFile( + path.join(playlistsPath, `${id}.json`) + ); + return JSON.parse(content); + } catch (e) { + // Fallback: scan only if direct access fails + const files = await fs.readdir(playlistsPath); + for (const file of files) { + if (file === `${id}.json`) continue; // Already tried + const content = await fs.readFile(path.join(playlistsPath, file)); + const playlist = JSON.parse(content); + if (playlist.id === id) return playlist; + } + } + return null; +} +``` + +**Prevention:** + +- For file-based data, assume filename matches ID +- Try direct file access before falling back to directory scan +- Use extracted helper methods (e.g., `findPlaylistFile()`) to avoid code duplication +- Use `Promise.allSettled()` for parallel file reads instead of sequential loops + +--- + ### Slow Database Queries **Symptom:** @@ -466,18 +827,1377 @@ const GamePlayer = lazy(() => import('./GamePlayer')); --- +## Code Quality Issues + +### Duplicate Utility Functions + +**Symptom:** + +Confusion about which utility to import. Inconsistent behavior across codebase. + +**Root Cause:** + +Similar utility functions exist in multiple files: + +```typescript +// WRONG: Same logic in multiple files +// frontend/src/lib/utils/errorUtils.ts +export function getErrorMessage(error: unknown): string { ... } + +// frontend/src/lib/api/api-error.ts +export function getErrorMessage(error: unknown): string { ... } + +// Which one should components import? +``` + +**CORRECT - Single source of truth:** + +```typescript +// CORRECT: One centralized utility +// frontend/src/lib/utils/errorUtils.ts +export function getErrorMessage(error: unknown): string { ... } + +// frontend/src/lib/api/api-error.ts +import { getErrorMessage } from '../utils/errorUtils'; +export { getErrorMessage }; // Re-export if needed for compatibility +``` + +**Prevention:** + +- Search for duplicate function names before creating new utilities +- Consolidate similar functions into a single module +- Document which utilities are canonical in team conventions +- Use linters to detect duplicate code + +--- + +### parseInt without NaN Validation + +**Symptom:** + +API returns 404 for numeric IDs instead of 400 Bad Request. Logs show NaN passed +to queries. + +**Root Cause:** + +`parseInt()` returns `NaN` for non-numeric strings, which silently passes to SQL +queries and matches no rows: + +```typescript +// WRONG: No NaN check +const id = parseInt(req.params.id, 10); +const user = UserDatabaseService.get('SELECT * FROM users WHERE id = ?', [id]); +// If req.params.id is 'abc', id is NaN +// Query runs: WHERE id = NaN (matches nothing) +// User gets 404 instead of 400 +``` + +**CORRECT - Always check isNaN():** + +```typescript +// CORRECT: Validate after parsing +const id = parseInt(req.params.id, 10); +if (isNaN(id)) { + return res.status(400).json({ error: 'Invalid ID' }); +} +const user = UserDatabaseService.get('SELECT * FROM users WHERE id = ?', [id]); +``` + +**Why this matters:** + +- NaN passes silently to SQL, creating misleading 404 responses +- Makes debugging harder - looks like data doesn't exist +- Security issue: No validation feedback helps attackers probe +- Client sees confusing "not found" instead of "invalid format" + +**Prevention:** + +- Always check `isNaN()` after `parseInt()` +- Unit test with non-numeric IDs +- Review logs for NaN values in queries + +--- + +### Unbounded Query Limits + +**Symptom:** + +Memory spikes and database slowdowns. Clients request `limit=999999` and dump +entire tables. + +**Root Cause:** + +Query limits not capped: + +```typescript +// WRONG: No limit cap +const limit = req.query.limit ? parseInt(req.query.limit) : 10; +const games = GameService.getGames(limit); +// Client requests: /api/games?limit=999999 +// Server executes: SELECT * FROM game LIMIT 999999 +// Returns entire database contents +``` + +**CORRECT - Cap query limits:** + +```typescript +// CORRECT: Cap limit with Math.min +const limit = Math.min( + req.query.limit ? parseInt(req.query.limit) : 10, + 100 // Maximum 100 results +); +const games = GameService.getGames(limit); +``` + +**Recommended Caps**: + +- Games/favorites/users: 100 per request +- Playlists: 50 per request +- Logs: 50 per request + +**Prevention:** + +- Add `Math.min()` for all user-provided limits +- Document API limit in OpenAPI/Swagger specs +- Monitor slow queries in production logs +- Test with extreme values: `?limit=999999` + +--- + +### Server-Sent Events (SSE) Error Handling + +**Symptom:** + +Backend crashes when errors occur during SSE streaming. Error: "Cannot set +headers after they are sent to the client." + +**Root Cause:** + +Once SSE headers are sent, `response.status().json()` fails because headers are +already in the response: + +```typescript +// WRONG: Tries to set status after headers sent +const sendGameStream = (req, res) => { + res.writeHead(200, { 'Content-Type': 'text/event-stream' }); // Headers sent + res.write(`data: ${JSON.stringify(data)}\n\n`); + + try { + // Some operation + } catch (error) { + res.status(500).json({ error: 'Server error' }); // ❌ Crashes! + } +}; +``` + +**CORRECT - Check headersSent before responding:** + +```typescript +// CORRECT: Check if headers already sent +const sendGameStream = (req, res) => { + res.writeHead(200, { 'Content-Type': 'text/event-stream' }); + res.write(`data: ${JSON.stringify(data)}\n\n`); + + try { + // Some operation + } catch (error) { + if (res.headersSent) { + // Headers already sent, use SSE format for error + res.write(`data: {"error": "Server error"}\n\n`); + res.end(); + } else { + // Headers not sent, use normal JSON response + res.status(500).json({ error: 'Server error' }); + } + } +}; +``` + +**Prevention:** + +- Always check `res.headersSent` before calling `res.status()` +- Wrap error handling around entire response lifecycle +- Use appropriate error format for SSE (events, not JSON) +- Test error paths during development + +--- + +### PerformanceMetrics Path Normalization + +**Symptom:** + +Metrics memory grows unbounded. `/api/games/:id` with different UUIDs creates +separate entries for each unique game. + +**Root Cause:** + +Parametric routes create unbounded map entries if paths aren't normalized: + +```typescript +// WRONG: Records raw request paths +GET /api/games/550e8400-e29b-41d4-a716-446655440000 // Separate entry +GET /api/games/6ba7b810-9dad-11d1-80b4-00c04fd430c8 // Separate entry +GET /api/games/6ba7b811-9dad-11d1-80b4-00c04fd430c8 // Separate entry +// After 10k unique games, metrics map has 10k entries! +``` + +**CORRECT - Normalize paths before recording:** + +```typescript +// CORRECT: Replace UUIDs/numbers with :id placeholder +const normalizePath = (path: string): string => { + return path + .replace(/\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi, '/:id') // UUID + .replace(/\/\d+/g, '/:id'); // Numeric ID +}; + +// Usage in metrics middleware +const normalizedPath = normalizePath(req.path); +recordMetric(normalizedPath, duration); +``` + +**Prevention:** + +- Always normalize parametric segments before recording metrics +- Monitor metrics map size: `metrics.size` should equal number of unique routes +- Set `maxMapSize` with warnings if exceeded +- Use middleware to normalize paths consistently + +--- + +### Cache Clearing Logic with Overlapping Conditions + +**Symptom:** + +Clearing one specific cache type also clears other caches unexpectedly. + +**Root Cause:** + +Overlapping `if` blocks instead of mutually exclusive branching: + +```typescript +// WRONG: Overlapping if blocks +if (!cacheType || cacheType === 'gameSearch') { clearGameSearch(); } +if (!cacheType || cacheType === 'permissions') { clearPermissions(); } +// When cacheType is 'gameSearch', the first block runs but so does... nothing else. +// But when cacheType is undefined, BOTH blocks run (double clear). +// And unknown types silently do nothing. +``` + +**CORRECT - Use switch for mutual exclusivity:** + +```typescript +switch (cacheType) { + case 'gameSearch': + clearGameSearch(); + break; + case 'permissions': + clearPermissions(); + break; + case 'all': + default: + clearGameSearch(); + clearPermissions(); + break; +} +``` + +**Prevention:** + +Use `switch` or `if/else if` chains for mutually exclusive options. Validate inputs with Zod enum schemas. + +--- + +### Fire-and-Forget Async Operations Without Error Handling + +**Symptom:** + +Unhandled promise rejection warnings in logs. Background operations fail silently. + +**Root Cause:** + +Calling async functions without `await` or `.catch()`: + +```typescript +// WRONG: No error handling +this.downloadInBackground(id, zipPath); // Promise returned but discarded + +// WRONG: IIFE without catch +(async () => { + await fs.writeFile(cachePath, data); +})(); +``` + +**CORRECT - Always handle the promise:** + +```typescript +// Option 1: .catch() on the call +this.downloadInBackground(id, zipPath) + .catch(err => logger.error('Background download failed:', err)); + +// Option 2: .catch() on IIFE +(async () => { + await fs.writeFile(cachePath, data); +})().catch(err => logger.warn('Cache write failed:', err)); +``` + +**Prevention:** + +Always add `.catch()` to fire-and-forget promises. Linters can flag floating promises. + +--- + +### Exporting Internal Constants + +**Symptom:** + +Confusion about which exports are part of the public API. Accidental coupling between modules. + +**Root Cause:** + +Exporting constants that are only used within the module: + +```typescript +// WRONG: Unnecessary export +export const CUSTOM_MIME_TYPES = { ... }; // Only used in this file + +export function getMimeType(filename: string): string { + // Uses CUSTOM_MIME_TYPES internally +} +``` + +**CORRECT - Keep internals private:** + +```typescript +// CORRECT: Don't export internal constants +const CUSTOM_MIME_TYPES = { ... }; // Private to this module + +export function getMimeType(filename: string): string { + // Uses CUSTOM_MIME_TYPES internally +} +``` + +**Why this matters:** + +- Reduces public API surface +- Makes it clear what's part of the module contract +- Prevents other code from depending on implementation details +- Makes refactoring easier (no external dependencies on internal constants) + +**Prevention:** + +- Only export what's needed by other modules +- Review exports before committing +- Use linters to detect unused exports +- Document public API explicitly + +--- + +### setInterval Blocking Graceful Shutdown + +**Symptom:** + +Process doesn't exit after SIGTERM. Node.js event loop stays alive. + +**Root Cause:** + +`setInterval` keeps the event loop alive by default: + +```typescript +// WRONG: Interval keeps process alive +this.cleanupInterval = setInterval(() => { + this.cleanup(); +}, 60000); +// Even after http.close(), this timer prevents process.exit() +``` + +**CORRECT - Use .unref() on cleanup intervals:** + +```typescript +this.cleanupInterval = setInterval(() => { + this.cleanup(); +}, 60000); +this.cleanupInterval.unref(); // Won't keep process alive +``` + +**Prevention:** + +Call `.unref()` on all non-critical intervals (cleanup jobs, background tasks). Only keep critical intervals ref'd. + +--- + +### Hostname Includes Port Number + +**Symptom:** + +Hostname validation fails. External lookups don't match expected patterns. + +**Root Cause:** + +`req.headers.host` includes the port (e.g., `localhost:3100`): + +```typescript +// WRONG: Port included in hostname +const hostname = req.headers.host || 'localhost'; +// hostname = 'localhost:3100' — validation regex rejects colons +``` + +**CORRECT - Strip port before using hostname:** + +```typescript +const hostname = (req.headers.host || 'localhost').split(':')[0]; +// hostname = 'localhost' — clean hostname without port +``` + +**Prevention:** + +Always strip port from `req.headers.host` before hostname validation or filesystem lookups. + +--- + +### Redundant Hostname Subdomain Variations + +**Symptom:** + +File system lookups try nonsensical hostnames like `www.www.example.com` or `cdn.cdn.example.com`. + +**Root Cause:** + +Generating subdomain variations without checking if hostname already has that prefix: + +```typescript +// WRONG: Always prepends subdomains +for (const sub of ['www', 'cdn', 'core']) { + variations.push(`${sub}.${hostname}`); +} +// If hostname is 'www.example.com', generates 'www.www.example.com' +``` + +**CORRECT - Skip existing prefixes:** + +```typescript +for (const sub of ['www', 'cdn', 'core']) { + if (!hostname.startsWith(`${sub}.`)) { + variations.push(`${sub}.${hostname}`); + } +} +``` + +**Prevention:** + +Check for existing prefixes before adding subdomain variations. + +--- + +### Flash Games Must Have .swf Launch Commands to Appear in Browse + +**Symptom:** + +Flash games exist in the database but don't appear in search results or browse +pages. Platform filters show zero Flash games. + +**Root Cause:** + +The backend filters Flash games whose `launchCommand` doesn't end in `.swf`. This +excludes Flash games with HTML wrapper pages or missing launch commands from +appearing in browse/search results. + +**CORRECT - Flash games with .swf launchCommands:** + +```typescript +// These appear in browse/search results +launchCommand: "http://uploads.ungrounded.net/396000/396724_DAplayer_V6.swf" +launchCommand: "Games/Flash/game.swf" +``` + +**WRONG - Flash games without .swf launchCommands:** + +```typescript +// These are FILTERED OUT and won't appear in results +launchCommand: "Game.html" // HTML wrapper +launchCommand: "wrapper.php" // PHP wrapper +launchCommand: "https://example.com/play" // Web wrapper +``` + +**Prevention:** + +- Check `launchCommand` ends with `.swf` for browseable Flash games +- Use detail view (`GET /api/games/:id`) for full game information +- Note: Other platforms (HTML5, Windows) are not affected by this filter + +--- + +### Route Ordering Matters in Express + +**Symptom:** + +Static routes like `/api/games/random` return wrong results or 404 errors. + +**Root Cause:** + +Route registration order matters in Express. Parameterized routes like `/:id` +match anything and will shadow (hide) more specific static routes if registered +first. + +**WRONG - Parameterized route registered first:** + +```typescript +// WRONG: This route matches EVERYTHING +app.get('/api/games/:id', (req, res) => { + // /api/games/random will match here with id='random' + // Then database query for game.id='random' returns nothing +}); + +// This static route is shadowed +app.get('/api/games/random', (req, res) => { + // This is NEVER reached! +}); +``` + +**CORRECT - Static route registered first:** + +```typescript +// CORRECT: Specific route first +app.get('/api/games/random', (req, res) => { + // /api/games/random matches here + // This is reached! +}); + +// Parameterized route after +app.get('/api/games/:id', (req, res) => { + // Only matches if path doesn't match static routes + // /api/games/uuid-123 matches here +}); +``` + +**Prevention:** + +- Always register static/specific routes BEFORE parameterized routes +- Test special routes explicitly: + `curl http://localhost:3100/api/games/random` +- Review route registration order when adding new endpoints + +--- + +### Use Nullish Coalescing (`??`) Instead of OR (`||`) for Config Defaults + +**Symptom:** + +Settings like `homeRecentHours = 0` or `maintenanceMode = false` fall through +to defaults when they shouldn't. + +**Root Cause:** + +The `||` operator treats `0`, `""`, and `false` as falsy and uses the default +even when explicitly set: + +```typescript +// WRONG: Using || +const homeRecentHours = settingsValue || 24; // If settingsValue is 0: +// Result: 24 (wrong! 0 was explicitly set) + +const maintenanceMode = settingsValue || false; // If settingsValue is false: +// Result: false (happens to work, but for wrong reason) +``` + +**CORRECT - Using nullish coalescing (`??`):** + +```typescript +// CORRECT: Using ?? +const homeRecentHours = settingsValue ?? 24; // If settingsValue is 0: +// Result: 0 (correct! Only uses default if null or undefined) + +const maintenanceMode = settingsValue ?? false; // If settingsValue is false: +// Result: false (correct! Only uses default if null or undefined) +``` + +**Key difference:** + +- `||` checks truthiness: `0`, `""`, `false`, `null`, `undefined` all trigger + default +- `??` checks nullishness: Only `null` and `undefined` trigger default + +**Prevention:** + +- Always use `??` for config/settings defaults +- Use `||` only when you actually want to treat falsy values as missing + +--- + +### Array `.sort()` Mutates in Place + +**Symptom:** + +Building cache keys from arrays returns inconsistent results. Cache misses +increase after sorting. + +**Root Cause:** + +JavaScript's `.sort()` mutates the original array: + +```typescript +// WRONG: Mutates the caller's array +function getCacheKey(tags: string[]): string { + const sorted = tags.sort(); // This MUTATES tags! + return sorted.join(':'); +} + +const myTags = ['z', 'a', 'b']; +const key1 = getCacheKey(myTags); // myTags is now ['a', 'b', 'z'] +const key2 = getCacheKey(myTags); // myTags is still ['a', 'b', 'z'] +// Cache might miss if code relies on original order +``` + +**CORRECT - Spread operator creates copy:** + +```typescript +// CORRECT: Creates copy before sorting +function getCacheKey(tags: string[]): string { + const sorted = [...tags].sort(); // Sorts the COPY + return sorted.join(':'); +} + +const myTags = ['z', 'a', 'b']; +const key1 = getCacheKey(myTags); // myTags is still ['z', 'a', 'b'] +const key2 = getCacheKey(myTags); // myTags is still ['z', 'a', 'b'] +// Consistent cache key regardless of mutations +``` + +**Prevention:** + +- Always use `[...array].sort()` or `Array.from(array).sort()` +- If mutation is intentional, explicitly assign: `array = array.sort()` +- Use linters that warn about accidental mutations + +--- + +### Token Refresh Subscriber Queue Cleanup + +**Symptom:** + +Memory leaks in token refresh mechanism. Promise queues grow unbounded over +time. + +**Root Cause:** + +When implementing a token refresh queue, subscribers waiting for the refresh +might not be properly cleaned up on failure: + +```typescript +// WRONG: Queued promises never rejected on failure +private refreshPromise: Promise | null = null; +private refreshSubscribers: ((token: string) => void)[] = []; + +async refreshToken(): Promise { + if (!this.refreshPromise) { + this.refreshPromise = this.performRefresh(); + } + + return new Promise((resolve, reject) => { + this.refreshSubscribers.push((token) => { + resolve({ accessToken: token }); + }); + }); + // If refreshPromise rejects, subscribers are never notified + // They wait forever in memory! +} +``` + +**CORRECT - Clean up on failure:** + +```typescript +// CORRECT: Reject subscribers on refresh failure +private refreshPromise: Promise | null = null; +private refreshSubscribers: Array<{ + resolve: (token: string) => void; + reject: (error: Error) => void; +}> = []; + +async refreshToken(): Promise { + if (!this.refreshPromise) { + this.refreshPromise = this.performRefresh().then( + (token) => { + // Success: resolve all subscribers + this.refreshSubscribers.forEach((sub) => + sub.resolve(token.accessToken) + ); + this.refreshSubscribers = []; + this.refreshPromise = null; + return token; + }, + (error) => { + // IMPORTANT: Reject all subscribers on failure + this.refreshSubscribers.forEach((sub) => + sub.reject(error) + ); + this.refreshSubscribers = []; + this.refreshPromise = null; + throw error; + } + ); + } + + return new Promise((resolve, reject) => { + this.refreshSubscribers.push({ + resolve, + reject, // Subscriber can be rejected now + }); + }); +} +``` + +**Prevention:** + +- Always handle both success and failure in Promise chains +- Reject/cleanup pending subscribers on failure +- Monitor queue sizes in production for growth patterns + +--- + +## Frontend-Specific Pitfalls + +### Callback Props in useEffect Dependencies + +**Symptom:** + +Infinite re-render loops. Component re-renders constantly even when data hasn't changed. + +**Root Cause:** + +Passing callback props directly in `useEffect` dependency arrays. Parent re-renders create new function references, triggering the effect: + +```typescript +// WRONG: Callback in dependency array +interface Props { + onLoad?: () => void; + onError?: (error: Error) => void; +} + +function Component({ onLoad, onError }: Props) { + useEffect(() => { + loadData() + .then(() => onLoad?.()) + .catch((err) => onError?.(err)); + }, [onLoad, onError]); // Parent re-renders create new functions → effect re-runs +} +``` + +**CORRECT - Use useRef pattern:** + +```typescript +// CORRECT: Store callbacks in refs +interface Props { + onLoad?: () => void; + onError?: (error: Error) => void; +} + +function Component({ onLoad, onError }: Props) { + const onLoadRef = useRef(onLoad); + const onErrorRef = useRef(onError); + + // Update refs when callbacks change + useEffect(() => { + onLoadRef.current = onLoad; + }, [onLoad]); + + useEffect(() => { + onErrorRef.current = onError; + }, [onError]); + + // Main effect references .current (stable) + useEffect(() => { + loadData() + .then(() => onLoadRef.current?.()) + .catch((err) => onErrorRef.current?.(err)); + }, []); // Empty deps - only runs once +} +``` + +**Why this pattern works:** + +- Ref `.current` is stable across re-renders +- Main effect doesn't depend on callback props +- Separate effects update refs when callbacks change +- No infinite loops + +**Prevention:** + +- Never put callback props in `useEffect` dependencies +- Use `useRef` to store callback references +- Update refs in separate effects +- Reference `ref.current` in the main effect + +--- + +### Hardcoded Tailwind Colors vs Theme Tokens + +**Symptom:** + +UI doesn't adapt to dark/light mode changes. Colors look out of place or are hard to read in certain themes. + +**Root Cause:** + +Using hardcoded Tailwind color classes instead of theme-aware tokens: + +```typescript +// WRONG: Hardcoded colors that don't adapt to theme +
+

Error message

+
+``` + +**CORRECT - Use theme tokens:** + +```typescript +// CORRECT: Theme-aware tokens +
+

Error message

+
+``` + +**Common theme tokens:** + +| Instead of | Use | +| --- | --- | +| `text-gray-900`, `text-white` | `text-foreground` | +| `text-gray-400`, `text-gray-600` | `text-muted-foreground` | +| `bg-white`, `bg-gray-900` | `bg-background` | +| `bg-gray-100`, `bg-gray-800` | `bg-muted` | +| `border-gray-200`, `border-gray-700` | `border-border` | +| `text-blue-600`, `text-blue-400` | `text-primary` | +| `bg-blue-600`, `bg-blue-500` | `bg-primary` | +| `text-red-600`, `text-red-400` | `text-destructive` | +| `bg-red-600`, `bg-red-500` | `bg-destructive` | + +**Why this matters:** + +- Theme tokens automatically adapt to dark/light mode +- Consistent color usage across the app +- Easier to maintain and update themes +- Better accessibility (theme-aware contrast ratios) + +**Prevention:** + +- Use theme tokens from the design system +- Review components in both light and dark modes +- Document theme token conventions +- Use linters to detect hardcoded color usage + +--- + +### Image Preload Without Cleanup Causes Memory Leaks + +**Symptom:** + +Multiple images load simultaneously when `src` changes quickly. Stale callbacks fire with outdated URLs. + +**Root Cause:** + +Creating `new Image()` objects without cleanup: + +```typescript +// WRONG: No cleanup on src change +useEffect(() => { + const img = new Image(); + img.onload = () => setImageSrc(src); + img.onerror = () => setError(true); + img.src = src; +}, [src]); +// When src changes, old Image keeps loading and may call stale handlers +``` + +**CORRECT - Add cancellation and use refs for callbacks:** + +```typescript +const onLoadRef = useRef(onLoad); +useEffect(() => { onLoadRef.current = onLoad; }, [onLoad]); + +useEffect(() => { + let cancelled = false; + const img = new Image(); + img.onload = () => { if (!cancelled) setImageSrc(src); }; + img.onerror = () => { if (!cancelled) setError(true); }; + img.src = src; + return () => { cancelled = true; img.onload = null; img.onerror = null; }; +}, [src]); +``` + +**Prevention:** + +Always return cleanup functions from useEffect. Store callbacks in refs to avoid dependency churn. + +--- + +### setTimeout Closure Leaks in useEffect + +**Symptom:** + +Old timeouts fire with stale state. Multiple competing timeouts when effect re-runs. + +**Root Cause:** + +`setTimeout` stored in a `let` variable inside `useEffect` leaks across re-runs: + +```typescript +// WRONG: Timeout in closure variable +useEffect(() => { + let timeout: NodeJS.Timeout; + timeout = setTimeout(() => setStatus('slow'), 10000); + return () => clearTimeout(timeout); +}, [status]); // Re-runs on status change, old timeout may already fire +``` + +**CORRECT - Use useRef for timeouts:** + +```typescript +const timeoutRef = useRef(null); + +useEffect(() => { + timeoutRef.current = setTimeout(() => setStatus('slow'), 10000); + return () => { + if (timeoutRef.current) clearTimeout(timeoutRef.current); + }; +}, []); // Stable deps, ref survives re-renders +``` + +**Prevention:** + +Store timeout/interval IDs in `useRef`, not `let` variables. Remove state values from dependency arrays when they only need to be read, not trigger re-runs. + +--- + +### Filter Chip IDs with Comma-Separated Values + +**Symptom:** + +Removing a filter chip removes the wrong filter. Multiple filters with commas get deleted instead of just the selected one. + +**Root Cause:** + +When building filter chips from comma-separated values, using array index as the chip identifier breaks when values contain commas: + +```typescript +// WRONG: Index-based chip IDs +const filterChips = values.split(',').map((value, index) => ({ + id: index, // ❌ Breaks when values contain commas + label: value, +})); + +// If tags = "action,puzzle,puzzle-adventure" +// chips[2] could match "puzzle-adventure" but index 2 is ambiguous +``` + +**Solution:** + +Use `encodeURIComponent` as the chip identifier: + +```typescript +// CORRECT: Unique identifiers for each chip +const filterChips = values.split(',').map((value) => ({ + id: encodeURIComponent(value), // ✅ Unique per value + label: value, +})); + +// When removing chip with id "puzzle-adventure" +// Only that exact value is removed +const updatedValues = filterChips + .filter((chip) => chip.id !== chipIdToRemove) + .map((chip) => decodeURIComponent(chip.id)) + .join(','); +``` + +**Prevention:** + +- Always use content-based identifiers for dynamic collections +- Never rely on array indices for removable items +- Use `encodeURIComponent` when building IDs from user content + +--- + +### Server-Sent Events (SSE) with Axios + +**Symptom:** + +SSE endpoints work in simple fetch examples but fail when using the axios API client. + +**Root Cause:** + +Axios doesn't support EventSource or streaming responses. It buffers the entire response, breaking the streaming contract of SSE. + +```typescript +// WRONG: Axios doesn't work with SSE +const response = await api.get('/api/stream/events'); // ❌ Won't stream +// Axios buffers entire response instead of opening EventSource +``` + +**Solution:** + +Use native `fetch()` for SSE endpoints and manually attach authentication: + +```typescript +// CORRECT: Use fetch for SSE +export async function subscribeToEvents( + onMessage: (data: any) => void +): Promise { + const token = useAuthStore.getState().accessToken; + + return new EventSource('/api/stream/events', { + headers: { + Authorization: `Bearer ${token}`, // ✅ Manually attach token + }, + }); +} + +// Usage in component +useEffect(() => { + let eventSource: EventSource; + + subscribeToEvents((data) => { + console.log('Event received:', data); + }).then((es) => { + eventSource = es; + eventSource.addEventListener('message', (e) => { + const event = JSON.parse(e.data); + onMessage(event); + }); + }); + + return () => { + eventSource?.close(); + }; +}, []); +``` + +**Prevention:** + +- Document SSE endpoints as exceptions to the "always use axios" rule +- Implement token refresh handling if tokens expire during long-lived SSE connections +- Include authentication token in EventSource constructor headers + +--- + +### Ruffle Window Type Safety + +**Symptom:** + +TypeScript errors when accessing Ruffle methods: `Property 'RufflePlayer' does not exist on type 'Window'` + +**Root Cause:** + +Ruffle is loaded via `