Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ builds
buildmeta.*
*.swp
*.tsbuildinfo
*.routes.latest.json
scripts/seed_map_ratings_from_csv.js
55 changes: 54 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,10 +58,63 @@ export SESSION_SECRET=foobar
Apply database migrations with `npm run migrate`.
You can then start up a server with `npm run dev`.

### Move Calculator Performance Harness

There is an engine-level benchmark test for the move calculator route search:

```bash
npm run test -- searcher_perf_test
```

To export a complex real game into a reproducible fixture:

```bash
npm run export-game-fixture -- 716 src/e2e/goldens/move_perf_716.json
```

Named flags are also supported when your shell forwards them correctly:

```bash
npm run export-game-fixture -- --game-id 716 --output src/e2e/goldens/move_perf_716.json
```

Then run the benchmark against that fixture:

```bash
MOVE_SEARCH_PERF_FIXTURE=src/e2e/goldens/move_perf_716.json npm run test -- searcher_perf_test
```

To enforce a soft regression gate (for example 30% over a known baseline):

```bash
MOVE_SEARCH_PERF_FIXTURE=src/e2e/goldens/move_perf_716.json \
MOVE_SEARCH_PERF_BASELINE_MS=47000 \
MOVE_SEARCH_PERF_ALLOWED_REGRESSION=0.3 \
npm run test -- searcher_perf_test
```

Defaults:

- Fixture: `src/e2e/goldens/create_game_after.json`
- Runs: 1 warmup + 3 measured iterations
- Output: route count and timing stats (min/median/max)

Optional environment variables:

- `MOVE_SEARCH_PERF_FIXTURE`: workspace-relative JSON fixture path
- `MOVE_SEARCH_PERF_GAME_KEY`: required only when fixture is raw serialized game data without top-level metadata
- `MOVE_SEARCH_PERF_VARIANT`: JSON string for special variants (for example Puerto Rico)
- `MOVE_SEARCH_PERF_WARMUP_RUNS`: warmup iteration count
- `MOVE_SEARCH_PERF_RUNS`: measured iteration count
- `MOVE_SEARCH_PERF_BASELINE_MS`: baseline median to compare against
- `MOVE_SEARCH_PERF_ALLOWED_REGRESSION`: soft budget ratio (default `0.3` = 30%)
- `MOVE_SEARCH_PERF_MAX_MS`: optional absolute median cap

For e2e smoke coverage, `src/e2e/move_good_test.ts` opens Move Calculator and times completion. You can set an optional soft timeout with `MOVE_CALCULATOR_E2E_TIMEOUT_MS`.

## Ratings Data Attribution

Map player-count ratings are derived from community-maintained sources:

- BoardGameGeek discussion and source list: https://boardgamegeek.com/thread/2930352/article/41563109#41563109
- Community spreadsheet: https://docs.google.com/spreadsheets/d/1zrBKRb3I3SJX2EvinB2U5D_wt0LPWMYAMf1FYeeq9fY/edit?gid=0#gid=0

90 changes: 90 additions & 0 deletions REGRESSION_BASELINES.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
{
"description": "MoveSearcher Algorithm - Comprehensive Regression Test Baselines. Both route counts AND actual routes are captured to ensure optimizations don't break the algorithm.",
"timestamp": "2026-03-05T21:20:00Z",
"approach": {
"overview": "Two-level regression verification",
"level1": "Route count - Must match exactly (74, 668, 1982)",
"level2": "Route content - Full routes exported to .routes.json files for detailed comparison",
"why": "Catches both algorithmic breaks (wrong count) AND subtle changes (different routes returned)"
},
"baselines": [
{
"fixture": "move_perf_2096.json",
"map": "madagascar",
"gameId": 2096,
"gameStatus": "ACTIVE",
"goods": 38,
"expectedRoutes": 74,
"routesFile": "src/e2e/goldens/move_perf_2096.routes.json",
"duration_ms": 13670,
"duration_pretty": "13.7s",
"routes_per_good": 1.95
},
{
"fixture": "move_perf_2026.json",
"map": "southern-us",
"gameId": 2026,
"gameStatus": "ACTIVE",
"goods": 38,
"expectedRoutes": 668,
"routesFile": "src/e2e/goldens/move_perf_2026.routes.json",
"duration_ms": 590436,
"duration_pretty": "9m 50s",
"routes_per_good": 17.6,
"note": "43x slower than Madagascar despite same good count - network complexity"
},
{
"fixture": "move_perf_716.json",
"map": "poland",
"gameId": 716,
"gameStatus": "ENDED",
"goods": 150,
"expectedRoutes": 1982,
"routesFile": "src/e2e/goldens/move_perf_716.routes.json",
"duration_ms": 1170609,
"duration_pretty": "19m 30s",
"routes_per_good": 13.2,
"note": "Most complex case - exponential route explosion on highly connected network"
}
],
"usage": {
"description": "How to use these regression baselines",
"steps": [
"1. Run performance test with optimization: npx ts-node src/scripts/measure_move_search.ts <FIXTURE>",
"2. NEW .routes.json file will be generated automatically",
"3. Diff the new .routes.json against baseline:",
" git diff src/e2e/goldens/move_perf_2096.routes.json",
"4. If routes changed, optimization may have broken the algorithm",
"5. Compare count first (faster), then full routes (comprehensive)"
],
"example_level1_regression": {
"output": "[MoveSearch] Complete! Found 75 routes in 12000ms",
"check": "75 !== expectedRoutes (74) ✗ REGRESSION DETECTED",
"action": "STOP - Algorithm is broken. Revert changes."
},
"example_level2_regression": {
"scenario": "Count matches (74) but route content differs",
"detection": "git diff shows different endingStop coordinates or owner values",
"impact": "Algorithm may be finding same number but different routes",
"action": "INVESTIGATE - Check if optimization changed search order/logic"
}
},
"command_reference": {
"generate_baselines": "MOVE_SEARCH_DEBUG=true npx ts-node src/scripts/measure_move_search.ts src/e2e/goldens/move_perf_2096.json",
"compare_routes": "git diff src/e2e/goldens/move_perf_2096.routes.json",
"check_all_fixtures": [
"npx ts-node src/scripts/measure_move_search.ts src/e2e/goldens/move_perf_2096.json",
"npx ts-node src/scripts/measure_move_search.ts src/e2e/goldens/move_perf_2026.json",
"npx ts-node src/scripts/measure_move_search.ts src/e2e/goldens/move_perf_716.json"
]
},
"notes": [
"Route counts MUST NEVER CHANGE - if they do, algorithm is broken",
"Route content should NOT change for the same algorithm logic",
"If both count and content match, optimization is safe (only timing improved)",
"If count matches but content differs, the algorithm logic may have changed subtly",
"Use full .routes.json files as ground truth in git diffs during development",
"These baselines should be committed to git with the fixture files",
"Add route validation to CI/CD to prevent regressions in production"
]
}
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
"deploy-server": "./src/scripts/deploy_server.sh",
"create-migration": "./src/scripts/create_migration.sh",
"create-map": "ts-node ./src/scripts/create_map.ts",
"export-game-fixture": "ts-node ./src/scripts/export_game_fixture.ts",
"package": "npm run build && mkdir ./builds && tar -cvzf ./builds/server.tar.gz ./bin",
"lint": "eslint src/ && knip",
"format": "prettier src --write",
Expand Down
2 changes: 2 additions & 0 deletions src/e2e/e2e_test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { buildingTrack } from "./build_track_test";
import { creatingGame } from "./create_game_test";
import { movingGoods } from "./move_good_test";
import { setUpServer } from "./util/server";
import { setTestTimeout } from "./util/timeout";
import { Driver, setUpWebDriver } from "./util/webdriver";
Expand All @@ -19,4 +20,5 @@ describe("e2e tests", () => {

describe("Building track", () => buildingTrack(driver));
describe("creating game", () => creatingGame(driver));
describe("moving goods", () => movingGoods(driver));
});
Loading
Loading