forked from NASA-AMMOS/MMGIS
-
Notifications
You must be signed in to change notification settings - Fork 0
Run the test suite against both deployment modes (full and lean) in CI #154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
a540feb
Run the test suite against both deployment modes (full and lean) in C…
CarsonDavis 281f5a9
Drop dangling acceptance-criteria cross-refs from CI comments
CarsonDavis 66d41b4
Trim the assert-gated-tables docblock down to the two real gotchas
CarsonDavis 24bdc63
Trim the deployment-mode spec header; drop the last acceptance cross-ref
CarsonDavis ac0b043
Lead the assert-gated-tables docblock with the rule, not an abstract …
CarsonDavis d25ded8
Reframe both-modes tables justification: pins the unconditional-sync …
CarsonDavis 9b4f69f
Merge remote-tracking branch 'origin/development' into tests-both-mod…
CarsonDavis cbf2d42
chore: bump version to 4.2.12-20260630 [version bump]
github-actions[bot] 108613a
simplify instructions to agents about modes
CarsonDavis 1d606f5
chore: bump version to 4.2.13-20260701 [version bump]
github-actions[bot] b01eebb
improve comments
CarsonDavis 8d60218
Tighten docstrings in deployment-mode spec and gated-tables check
CarsonDavis 51acd05
Load .env in Playwright config so the mode reaches the test process
CarsonDavis 9aefadc
ci(e2e): keep legacy smoke/eventbus specs gating
CarsonDavis File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,9 +7,40 @@ on: | |
| branches: [master, main, development] | ||
|
|
||
| jobs: | ||
| # Unit tests (Vitest + jsdom, no DB/browser). Run once, outside the matrix: | ||
| # they already cover both modes in a single run by reloading the mode module | ||
| # per test. See tests/unit/deploymentMode.spec.js and bakeGuards.spec.js. | ||
| unit: | ||
| timeout-minutes: 10 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: "20" | ||
| cache: "npm" | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci | ||
|
|
||
| - name: Run Unit Tests | ||
| run: npx vitest run | ||
|
|
||
| # E2e tests boot the real app. Mode is fixed at process start, so we boot once | ||
| # per mode via the matrix. MMGIS_DEPLOYMENT_MODE is written into .env below. | ||
| test: | ||
| name: e2e (${{ matrix.mode }}) | ||
| timeout-minutes: 20 | ||
| runs-on: ubuntu-latest | ||
| needs: unit | ||
|
|
||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| mode: [full, lean] | ||
|
|
||
| services: | ||
| postgres: | ||
|
|
@@ -39,12 +70,6 @@ jobs: | |
| - name: Install dependencies | ||
| run: npm ci | ||
|
|
||
| # Unit tests run under Vitest in a jsdom environment — hermetic, so they | ||
| # need neither the database nor a browser. Run them before the e2e-only | ||
| # setup so a DB/browser problem can't mask a unit result. | ||
| - name: Run Unit Tests | ||
| run: npx vitest run | ||
|
|
||
| - name: Install Playwright Browsers | ||
| run: npx playwright install --with-deps chromium | ||
|
|
||
|
|
@@ -62,39 +87,65 @@ jobs: | |
| echo "ENABLE_MMGIS_WEBSOCKETS=false" >> .env | ||
| echo "ENABLE_CONFIG_WEBSOCKETS=false" >> .env | ||
| echo "HIDE_CONFIG=true" >> .env | ||
| echo "MMGIS_DEPLOYMENT_MODE=${{ matrix.mode }}" >> .env | ||
|
|
||
| # Lean deploys no sidecars. sample.env ships them ON; left on, init-db would | ||
| # try to create the mmgis-stac catalog DB. Turn them off so the lean env | ||
| # matches a real lean deployment. | ||
| - name: Disable sidecar services for the lean leg | ||
| if: matrix.mode == 'lean' | ||
| run: | | ||
| echo "WITH_STAC=false" >> .env | ||
| echo "WITH_TIPG=false" >> .env | ||
| echo "WITH_TITILER=false" >> .env | ||
| echo "WITH_TITILER_PGSTAC=false" >> .env | ||
| echo "WITH_VELOSERVER=false" >> .env | ||
|
Comment on lines
+98
to
+102
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. just a heads up to other reviewers, this is being simplified in #155, which depends on this PR for it's tests |
||
|
|
||
| # In lean, also confirms the mmgis-stac catalog DB is not created (full-only). | ||
| - name: Initialize Database | ||
| run: node scripts/init-db.js | ||
|
|
||
| # The e2e suite drives the real front-end at `/`, which the server renders | ||
| # from build/index.pug. Without this build step the route 500s, every test | ||
| # times out waiting for window.mmgisAPI, and the job is cancelled at the | ||
| # 20-minute cap. Build must run before the tests start the server. | ||
| # A feature gated OFF in the current mode still keeps its DB tables (both | ||
| # directions: datasets/geodatasets/draw/shortener off in lean, deployments | ||
| # off in full). Models always register; only routes are gated (ADR D2). | ||
| # Fails CI if an expected table is missing. | ||
| - name: Assert gated tables exist | ||
| run: node tests/ci/assert-gated-tables.js | ||
|
|
||
| # Must build before the server starts: the e2e tests hit `/`, which 500s | ||
| # without build/index.pug. | ||
| - name: Build frontend | ||
| # GitHub Actions auto-sets CI=true, which makes react-scripts promote all | ||
| # build warnings to errors. MMGIS ships several intentional warnings (the | ||
| # optional dist/ theme assets and the unshipped AOI counties/cities geojson | ||
| # dirs loaded via require.context). The production Dockerfile builds with CI | ||
| # unset, so mirror that here — otherwise the build fails on non-fatal warnings. | ||
| # CI=false: GitHub sets CI=true, which makes react-scripts fail on MMGIS's | ||
| # intentional build warnings. The production Dockerfile builds with CI unset. | ||
| run: npm run build | ||
| env: | ||
| CI: false | ||
|
|
||
| - name: Run E2E Tests | ||
| run: npx playwright test tests/e2e --project=chromium | ||
| # Legacy smoke / event-bus specs. Gating (no continue-on-error), matching | ||
| # their pre-matrix behavior on development — a real regression here fails | ||
| # the leg. They boot the same app per mode, so they also confirm smoke + | ||
| # event bus pass under lean, not just full. | ||
| - name: Run legacy E2E Tests | ||
| run: npx playwright test tests/e2e/smoke.spec.js tests/e2e/eventbus-integration.spec.js --project=chromium | ||
|
|
||
| # Gating check: each feature's route must be mounted (non-404) in its mode | ||
| # and absent (404) otherwise. Checks mount vs. 404 only, not behavior. No | ||
| # continue-on-error, so mis-gating fails the leg. | ||
| - name: Run deployment-mode shape check (gating) | ||
| run: npx playwright test tests/e2e/deployment-mode.spec.js --project=chromium | ||
|
|
||
| - name: Upload test results | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: playwright-report | ||
| name: playwright-report-${{ matrix.mode }} | ||
| path: playwright-report/ | ||
| retention-days: 30 | ||
|
|
||
| - name: Upload test artifacts | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: test-results | ||
| name: test-results-${{ matrix.mode }} | ||
| path: test-results/ | ||
| retention-days: 7 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| /** | ||
| * assert-gated-tables.js | ||
| * | ||
| * A feature gated OFF in the current mode still keeps its DB tables: models | ||
| * register and sync() runs unconditionally on boot, only route mounts are gated | ||
| * (ADR D2). This pins that — dropping a model, or gating its registration, fails | ||
| * CI in the leg where the feature is off. | ||
| * | ||
| * Boots the backend setups (registering models), runs sync(), and exits non-zero | ||
| * if any gated-feature table is missing. | ||
| * | ||
| * Must run AFTER scripts/init-db.js: sync() creates PostGIS geometry columns, so | ||
| * the postgis extension must exist first. | ||
| */ | ||
|
|
||
| require("dotenv").config({ path: __dirname + "/../../.env" }); | ||
|
|
||
| const { MODE } = require("../../API/Backend/Utils/deploymentMode"); | ||
| const setups = require("../../API/setups"); | ||
| const { sequelize } = require("../../API/connection"); | ||
|
|
||
| // Hand-written: tables behind features gated off in one mode or the other; they | ||
| // must exist in BOTH. Accept either spelling where Sequelize may pluralize. | ||
| const REQUIRED_TABLE_GROUPS = [ | ||
| ["datasets"], // geodata management (datasets) | ||
| ["geodatasets"], // geodata management (geodatasets) | ||
| ["user_files"], // on-disk mission filesystem / drawing | ||
| ["user_features"], // drawing (vector features) | ||
| ["url_shorteners", "url_shortener"], // link shortener | ||
| ["deployments"], // lean-only dashboard publish flow | ||
| ]; | ||
|
|
||
| async function main() { | ||
| await new Promise((resolve) => { | ||
| // Loading the setups requires each feature's models, registering them. | ||
| setups.getBackendSetups(() => resolve()); | ||
| }); | ||
|
|
||
| await sequelize.authenticate(); | ||
| await sequelize.sync(); | ||
|
|
||
| const tables = await sequelize | ||
| .getQueryInterface() | ||
| .showAllTables(); | ||
| const present = new Set(tables.map((t) => String(t).toLowerCase())); | ||
|
|
||
| const missing = []; | ||
| for (const group of REQUIRED_TABLE_GROUPS) { | ||
| const found = group.some((name) => present.has(name.toLowerCase())); | ||
| if (!found) missing.push(group.join(" | ")); | ||
| } | ||
|
|
||
| if (missing.length > 0) { | ||
| console.error( | ||
| `[assert-gated-tables] MODE=${MODE}: missing expected table(s): ${missing.join( | ||
| ", " | ||
| )}` | ||
| ); | ||
| console.error( | ||
| `[assert-gated-tables] tables present: ${[...present] | ||
| .sort() | ||
| .join(", ")}` | ||
| ); | ||
| process.exit(1); | ||
| } | ||
|
|
||
| console.log( | ||
| `[assert-gated-tables] MODE=${MODE}: all ${REQUIRED_TABLE_GROUPS.length} gated-feature table groups present.` | ||
| ); | ||
| process.exit(0); | ||
| } | ||
|
|
||
| main().catch((err) => { | ||
| console.error("[assert-gated-tables] Unexpected failure:", err); | ||
| process.exit(1); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| import { test, expect, request } from '@playwright/test' | ||
|
|
||
| /** | ||
| * Deployment-mode present/absent checks. Boots once per mode in CI; per feature, | ||
| * asserts its route is reachable in the mode it belongs to and gone otherwise. | ||
| * | ||
| * The on/off mapping below is hand-written from the deployment ADR. Do NOT | ||
| * rewrite it to read expectations from a capability table — a test that takes its | ||
| * answers from the code under test can't catch a wrong entry. | ||
| * | ||
| * Discriminator: a mounted route answers from its handler (never 404); an | ||
| * unmounted route hits the catch-all `app.all('*')` -> 404. | ||
| */ | ||
|
|
||
| const MODE = process.env.MMGIS_DEPLOYMENT_MODE || 'full' | ||
|
|
||
| // Each feature: which mode it belongs to, and one HTTP probe against a route it | ||
| // owns. We only care whether that route is mounted (non-404) or absent (404). | ||
| const FEATURES = [ | ||
| // --- Full-only: present in full, absent in lean --- | ||
| { | ||
| name: 'geodata management (datasets)', | ||
| mode: 'full', | ||
| method: 'post', | ||
| // ensureAdmin allow-lists /get, so it reaches the router without auth. | ||
| path: '/api/datasets/get', | ||
| }, | ||
| { | ||
| name: 'geodata management (geodatasets)', | ||
| mode: 'full', | ||
| method: 'get', | ||
| // ensureAdmin allow-lists /api/geodatasets/get. | ||
| path: '/api/geodatasets/get', | ||
| }, | ||
| { | ||
| name: 'drawing (draw API)', | ||
| mode: 'full', | ||
| method: 'post', | ||
| path: '/api/draw/add', | ||
| }, | ||
| { | ||
| name: 'on-disk mission filesystem (files API)', | ||
| mode: 'full', | ||
| method: 'post', | ||
| path: '/api/files/getfiles', | ||
| }, | ||
| { | ||
| name: 'link shortener', | ||
| mode: 'full', | ||
| method: 'post', | ||
| path: '/api/shortener/shorten', | ||
| }, | ||
| { | ||
| name: 'server-side raster utilities', | ||
| mode: 'full', | ||
| method: 'post', | ||
| // /api/utils mounts in both modes, but its raster endpoints are isFull()-only. | ||
| path: '/api/utils/getbands', | ||
| }, | ||
| { | ||
| name: 'bundled sidecar services / proxy (titiler)', | ||
| mode: 'full', | ||
| method: 'get', | ||
| // /titiler proxy needs isFull() AND WITH_TITILER=true; lean has neither. | ||
| path: '/titiler/healthz', | ||
| }, | ||
|
|
||
| // --- Lean-only: present in lean, absent in full --- | ||
| { | ||
| name: 'dashboard publish flow (deployments)', | ||
| mode: 'lean', | ||
| method: 'get', | ||
| // Mounted only when isLean(); ensureAdmin rejects with 200 (not 404). | ||
| path: '/api/deployments', | ||
| }, | ||
| ] | ||
|
|
||
| async function probe(api, feature) { | ||
| const res = | ||
| feature.method === 'post' | ||
| ? await api.post(feature.path, { data: {} }) | ||
| : await api.get(feature.path) | ||
| return res.status() | ||
| } | ||
|
|
||
| test.describe(`Deployment mode present/absent — MODE=${MODE}`, () => { | ||
| let api | ||
|
|
||
| test.beforeAll(async () => { | ||
| api = await request.newContext({ | ||
| baseURL: process.env.TEST_BASE_URL || 'http://localhost:8888', | ||
| ignoreHTTPSErrors: true, | ||
| }) | ||
| }) | ||
|
|
||
| test.afterAll(async () => { | ||
| await api.dispose() | ||
| }) | ||
|
|
||
| for (const feature of FEATURES) { | ||
| const belongsToRunningMode = feature.mode === MODE | ||
|
|
||
| test(`${feature.name} is ${belongsToRunningMode ? 'present' : 'absent' | ||
| } in ${MODE}`, async () => { | ||
| const status = await probe(api, feature) | ||
|
|
||
| if (belongsToRunningMode) { | ||
| // Mounted: any status but the catch-all 404. | ||
| expect( | ||
| status, | ||
| `${feature.name} should be MOUNTED in ${MODE} (got ${status}); a 404 means the route is gone` | ||
| ).not.toBe(404) | ||
| } else { | ||
| // Absent: falls through to the app catch-all 404. | ||
| expect( | ||
| status, | ||
| `${feature.name} should be ABSENT in ${MODE} (got ${status}); anything but 404 means the route is still mounted` | ||
| ).toBe(404) | ||
| } | ||
| }) | ||
| } | ||
| }) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we upgrade this version?