dotcom-arcade/docs/05-game-service/gamezip-server.md

GameZip Server (Port 22501)

The GameZip Server provides ZIP archive mounting and streaming without extraction, allowing efficient serving of game files packed in ZIP archives.

Overview

The GameZip server mounts ZIP archives in-memory and serves files directly using streaming, eliminating the need for disk extraction. This approach saves disk space and improves performance for frequently accessed games.

Architecture

┌──────────────────────────────────────────────┐
│           GameZip Server (22501)             │
├──────────────────────────────────────────────┤
│  HTTP Server (Express)                       │
│       ↓                                      │
│  GameZipServer Class                         │
│       ↓                                      │
│  ZipManager (singleton)                      │
│       ↓                                      │
│  node-stream-zip (streaming ZIP access)      │
│       ↓                                      │
│  ZIP files in Data/Games/                    │
└──────────────────────────────────────────────┘

API Endpoints

Mount ZIP Archive

Mount a ZIP file for serving:

POST /mount/:id HTTP/1.1
Content-Type: application/json

{
  "zipPath": "D:/Flashpoint/Data/Games/Flash/G/game.zip"
}

Response:

{
  "success": true,
  "id": "game-123",
  "zipPath": "D:/Flashpoint/Data/Games/Flash/G/game.zip"
}

Parameters:

• id (path): Unique identifier for this mount (e.g., game data ID)
• zipPath (body): Absolute path to ZIP file

Errors:

• 400: Missing mount ID or zipPath
• 500: ZIP file not found or mount failed

Unmount ZIP Archive

Unmount a previously mounted ZIP:

DELETE /mount/:id HTTP/1.1

Response:

{
  "success": true,
  "id": "game-123"
}

Errors:

• 400: Missing mount ID
• 404: ZIP not mounted (success: false)

List Mounted ZIPs

Get list of all currently mounted ZIPs:

GET /mounts HTTP/1.1

Response:

{
  "mounts": [
    {
      "id": "game-123",
      "zipPath": "D:/Flashpoint/Data/Games/Flash/G/game.zip",
      "mountTime": "2025-01-18T10:30:00.000Z",
      "fileCount": 0
    }
  ]
}

Serve File from ZIP

Get a file from a mounted ZIP archive:

GET http://www.example.com/path/file.swf HTTP/1.1

or

GET /http://www.example.com/path/file.swf HTTP/1.1

or

GET /path/file.swf HTTP/1.1
Host: www.example.com

Response:

HTTP/1.1 200 OK
Content-Type: application/x-shockwave-flash
Content-Length: 123456
Cache-Control: public, max-age=86400
X-Source: gamezipserver:game-123
Access-Control-Allow-Origin: *

[file data]

Errors:

• 404: File not found in any mounted ZIP

Request Processing

URL Parsing

GameZip server supports the same three URL formats as the proxy server:

private async handleFileRequest(req: http.IncomingMessage, res: http.ServerResponse) {
  let hostname: string;
  let urlPath: string;

  if (req.url.startsWith('http://') || req.url.startsWith('https://')) {
    // Proxy-style: GET http://domain.com/path HTTP/1.1
    const targetUrl = new URL(req.url);
    hostname = targetUrl.hostname;
    urlPath = targetUrl.pathname + targetUrl.search;
  }
  else if (req.url.startsWith('/http://') || req.url.startsWith('/https://')) {
    // Path-based: GET /http://domain.com/path HTTP/1.1
    const urlWithoutSlash = req.url.substring(1);
    const targetUrl = new URL(urlWithoutSlash);
    hostname = targetUrl.hostname;
    urlPath = targetUrl.pathname + targetUrl.search;
  }
  else {
    // Regular: GET /path HTTP/1.1
    hostname = req.headers.host || 'localhost';
    urlPath = req.url;
  }

  // Build relative path
  const relPath = path.posix.join(hostname, urlPath);

  // Search mounted ZIPs
  const result = await zipManager.findFile(relPath);
  // ...
}

Query String Handling

Query strings are stripped before ZIP lookup:

// Example: file.swf?token=xyz → file.swf
const pathWithoutQuery = urlPath.split('?')[0];
const relPath = path.posix.join(hostname, pathWithoutQuery);

Rationale: Files in ZIP archives don't have query strings, but games may append them for cache busting.

File Serving

Path Search Strategy

For a request to www.example.com/path/file.swf, the ZipManager tries multiple path variations:

1. content/www.example.com/path/file.swf (most common)
2. htdocs/www.example.com/path/file.swf (standard)
3. www.example.com/path/file.swf (no prefix)
4. Legacy/htdocs/www.example.com/path/file.swf (full path)

First match wins - returns immediately when file is found.

Multi-ZIP Search

If multiple ZIPs are mounted, all are searched in mount order:

async findFile(relPath: string): Promise<{data, mountId} | null> {
  const pathsToTry = [
    `content/${relPath}`,
    `htdocs/${relPath}`,
    relPath,
    `Legacy/htdocs/${relPath}`
  ];

  // Search all mounted ZIPs
  for (const [id, mounted] of this.mountedZips) {
    for (const pathVariant of pathsToTry) {
      const data = await this.getFile(id, pathVariant);
      if (data) {
        return { data, mountId: id };
      }
    }
  }

  return null;
}

Streaming Access

Files are read directly from ZIP archives using node-stream-zip:

async getFile(id: string, filePath: string): Promise<Buffer | null> {
  const mounted = this.mountedZips.get(id);
  if (!mounted) return null;

  try {
    // Stream file data from ZIP
    const data = await mounted.zip.entryData(filePath);
    return data;
  } catch (error) {
    return null;
  }
}

Benefits:

• No disk extraction required
• No temporary files created
• Direct memory buffer access
• Fast random access

MIME Type Detection

MIME types are detected from file extensions:

const ext = path.extname(pathWithoutQuery).substring(1).toLowerCase();
const contentType = getMimeType(ext);

res.setHeader('Content-Type', contentType);

See mime-types.md for supported types.

HTML Polyfill Injection

HTML files from ZIPs are automatically processed to inject polyfills:

let fileData = result.data;

// Process HTML files
if (ext === 'html' || ext === 'htm') {
  fileData = injectPolyfills(result.data);
  logger.info(`Injected polyfills into HTML file: ${relPath}`);
}

See html-polyfills.md for details.

Response Headers

Standard Headers

Content-Type: application/x-shockwave-flash
Content-Length: 123456
Cache-Control: public, max-age=86400
X-Source: gamezipserver:game-123
Connection: keep-alive

CORS Headers

Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: *

Custom Headers

X-Source: Indicates the ZIP mount ID that served the file

X-Source: gamezipserver:game-123

Use case: Debugging, monitoring, performance analysis

CORS Handling

Preflight Requests

OPTIONS /path/file.swf HTTP/1.1

Response:

HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, DELETE, OPTIONS
Access-Control-Allow-Headers: *
Access-Control-Max-Age: 86400

CORS on Errors

CORS headers are included even on error responses:

private sendError(res: http.ServerResponse, statusCode: number, message: string) {
  // Set CORS headers even for errors
  if (this.settings.allowCrossDomain) {
    res.setHeader('Access-Control-Allow-Origin', '*');
    res.setHeader('Access-Control-Allow-Methods', 'GET, POST, DELETE, OPTIONS');
    res.setHeader('Access-Control-Allow-Headers', '*');
  }

  res.setHeader('Content-Type', 'text/plain');
  res.writeHead(statusCode);
  res.end(message);
}

Integration with Backend

Typical Workflow

1. User requests game:

   GET /api/games/123/launch
   

2. Backend checks for ZIP:

   const game = await GameService.getGameById(123);
   const zipPath = path.join(FLASHPOINT_GAMES_PATH, game.zippedPath);
   

3. Backend mounts ZIP:

   await axios.post(`http://localhost:22501/mount/${game.id}`, {
     zipPath: zipPath
   });
   

4. Backend returns launch config:

   {
     "launchUrl": "http://localhost:22500/http://www.example.com/game.swf",
     "mountId": "game-123"
   }
   

5. Frontend loads game:

   <iframe src="http://localhost:22500/http://www.example.com/game.swf">
   

6. Proxy server checks GameZip:

   const result = await tryGameZipServer('www.example.com', '/game.swf');
   

7. GameZip serves file:

   • Search mounted ZIPs
   • Find in content/www.example.com/game.swf
   • Stream from ZIP
   • Return to proxy server
   • Proxy returns to frontend

Auto-Unmount (Future)

Current implementation: Manual unmount required

Future enhancement: Auto-unmount after game session ends or inactivity timeout

// Future: Auto-unmount after 1 hour of inactivity
const UNMOUNT_TIMEOUT = 60 * 60 * 1000; // 1 hour

setInterval(() => {
  for (const [id, mounted] of mountedZips) {
    const inactiveTime = Date.now() - mounted.lastAccess;
    if (inactiveTime > UNMOUNT_TIMEOUT) {
      await unmount(id);
    }
  }
}, 5 * 60 * 1000); // Check every 5 minutes

Performance Considerations

Memory Usage

ZIP Index: Each mounted ZIP loads its file index into memory:

• Small ZIP (~100 files): ~1MB RAM
• Medium ZIP (~1,000 files): ~5MB RAM
• Large ZIP (~10,000 files): ~50MB RAM
• Very large ZIP (>100,000 files): ~500MB RAM

File Buffers: Each request loads file into memory:

• Small file (<1MB): Negligible
• Medium file (1-10MB): Moderate impact
• Large file (>10MB): Significant memory spike

Recommendation: Limit concurrent large file requests or implement streaming for files >10MB

Access Speed

ZIP File Access:

• Index lookup: O(1) hash table
• File read: Random access, depends on file position in ZIP
• Decompression: If file is compressed (deflate), CPU overhead

Typical Performance:

• Small files (<1MB): 2-5ms
• Medium files (1-10MB): 10-50ms
• Large files (>10MB): 50-500ms

Comparison to Extracted Files:

• Extracted: 5-20ms (SSD), 20-100ms (HDD)
• ZIP: 10-50ms (adds decompression overhead)
• Verdict: ZIP access is competitive with HDD, slightly slower than SSD

Optimization Strategies

1. Limit Mounted ZIPs:

   const MAX_MOUNTED_ZIPS = 100;
   if (mountedZips.size >= MAX_MOUNTED_ZIPS) {
     await unmountOldest();
   }
   

2. Pre-mount Popular Games:

   // Mount top 10 games on startup
   const popularGames = await getPopularGames(10);
   for (const game of popularGames) {
     await zipManager.mount(game.id, game.zipPath);
   }
   

3. File Caching (Future):

   const cache = new LRU({ max: 100 }); // Cache 100 most accessed files
   

Error Handling

Mount Errors

ZIP file not found:

POST /mount/game-123
{"zipPath": "D:/invalid/path.zip"}

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain

ZIP file not found: D:/invalid/path.zip

ZIP already mounted:

// Silent success - returns immediately
if (this.mountedZips.has(id)) {
  logger.debug(`ZIP already mounted: ${id}`);
  return;
}

Corrupted ZIP:

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain

Failed to read ZIP file

File Access Errors

File not in ZIP:

GET http://www.example.com/missing.swf

HTTP/1.1 404 Not Found
Content-Type: text/plain
Access-Control-Allow-Origin: *

File not found in mounted ZIPs

No ZIPs mounted:

HTTP/1.1 404 Not Found
Content-Type: text/plain

File not found in mounted ZIPs

Graceful Degradation

If GameZip server fails to start, the proxy server continues without it:

try {
  await createGameZipServer();
  logger.info('GameZip Server started');
} catch (error) {
  logger.error('Failed to start GameZip Server:', error);
  logger.warn('Continuing without GameZip - ZIP files will not be available');
}

Testing

Mount and Serve

# 1. Mount a ZIP
curl -X POST http://localhost:22501/mount/test-game \
  -H "Content-Type: application/json" \
  -d '{"zipPath": "D:/Flashpoint/Data/Games/test.zip"}'

# Expected: {"success":true,"id":"test-game","zipPath":"..."}

# 2. List mounts
curl http://localhost:22501/mounts

# Expected: {"mounts":[{"id":"test-game",...}]}

# 3. Request file
curl http://localhost:22501/http://www.example.com/test.swf

# Expected: File content with proper MIME type

# 4. Check headers
curl -I http://localhost:22501/http://www.example.com/test.swf

# Expected:
# Content-Type: application/x-shockwave-flash
# X-Source: gamezipserver:test-game
# Access-Control-Allow-Origin: *

# 5. Unmount
curl -X DELETE http://localhost:22501/mount/test-game

# Expected: {"success":true,"id":"test-game"}

Error Cases

# Missing zipPath
curl -X POST http://localhost:22501/mount/test \
  -H "Content-Type: application/json" \
  -d '{}'

# Expected: 400 Missing zipPath

# Invalid ZIP path
curl -X POST http://localhost:22501/mount/test \
  -H "Content-Type: application/json" \
  -d '{"zipPath": "/invalid/path.zip"}'

# Expected: 500 ZIP file not found

# File not in ZIP
curl http://localhost:22501/http://www.example.com/missing.swf

# Expected: 404 File not found in mounted ZIPs

# Unmount non-existent
curl -X DELETE http://localhost:22501/mount/missing

# Expected: {"success":false,"id":"missing"}

Logging

Mount Operations

[GameZipServer] POST /mount/game-123
[ZipManager] Mounting ZIP: game-123 -> D:/Flashpoint/Data/Games/game.zip
[ZipManager] ✓ Mounted ZIP: game-123 (1234 files)

File Requests

[GameZipServer] GET http://www.example.com/game.swf
[GameZipServer] Looking for: www.example.com/game.swf
[ZipManager] Reading from ZIP game-123: content/www.example.com/game.swf
[ZipManager] ✓ Read 123456 bytes from game-123:content/www.example.com/game.swf
[GameZipServer] ✓ Serving from ZIP game-123: www.example.com/game.swf (123456 bytes)

Unmount Operations

[GameZipServer] DELETE /mount/game-123
[ZipManager] Unmounting ZIP: game-123
[ZipManager] ✓ Unmounted ZIP: game-123

Error Logging

[GameZipServer] Sending error 404: File not found in mounted ZIPs
[ZipManager] ZIP not mounted: game-999
[ZipManager] Error unmounting ZIP: game-123 [Error details]

Security Considerations

Path Traversal Prevention

File paths are normalized to prevent directory traversal:

const normalizedPath = filePath.startsWith('/')
  ? filePath.substring(1)
  : filePath;

This prevents attacks like:

• ../../../etc/passwd
• ..\\..\\windows\\system32

Mount Access Control

Only explicitly mounted ZIPs are accessible:

• Mount IDs are controlled by backend
• No arbitrary ZIP path access from frontend
• Frontend can only request files from mounted ZIPs

ZIP Bomb Protection (Future)

Current implementation loads entire files into memory, vulnerable to ZIP bombs.

Future protection:

const MAX_FILE_SIZE = 100 * 1024 * 1024; // 100MB

if (fileSize > MAX_FILE_SIZE) {
  throw new Error('File too large');
}

Troubleshooting

Port Already in Use

Error: Port 22501 is already in use

Solution: Change port in .env

GAMEZIPSERVER_PORT=22502

ZIP Mount Fails

Error: ZIP file not found: D:/path/to/game.zip

Debug steps:

1. Verify path is absolute
2. Check file permissions
3. Ensure ZIP file exists
4. Try mounting manually with curl

File Not Found in ZIP

Error: 404: File not found in mounted ZIPs

Debug steps:

1. List mounted ZIPs: curl http://localhost:22501/mounts
2. Check if correct ZIP is mounted
3. Verify file exists in ZIP (extract and check)
4. Enable DEBUG logging to see path variations tried
5. Check ZIP structure (content/, htdocs/, or root?)

Memory Issues

Symptoms: High memory usage, slow responses, crashes

Solutions:

1. Limit number of mounted ZIPs
2. Implement auto-unmount for inactive ZIPs
3. Add file size limits
4. Use streaming for large files
5. Monitor memory with process.memoryUsage()

Future Enhancements

1. Auto-Unmount: Unmount inactive ZIPs after timeout
2. Pre-Warming: Mount popular games on startup
3. Streaming: Stream large files instead of loading into memory
4. Caching: LRU cache for frequently accessed files
5. Compression: Transparent Brotli compression for responses
6. Metrics: Expose mount count, hit rate, memory usage
7. ZIP Validation: Verify ZIP integrity on mount
8. Concurrent Access: Optimize for multiple simultaneous requests
9. Health Check: /health endpoint for monitoring
10. Admin API: List files in mounted ZIPs, search across ZIPs

References

• node-stream-zip: https://github.com/antelle/node-stream-zip
• ZIP file format: https://en.wikipedia.org/wiki/ZIP_(file_format)
• Flashpoint ZIP structure standards