Add listing tests by tintinthong · Pull Request #251 · cardstack/boxel-catalog

tintinthong · 2026-03-24T11:13:26Z

No description provided.

Copilot

Pull request overview

This PR refactors the CI setup for the catalog repo by switching from the previous catalog acceptance/integration test execution to a new “live test” workflow in the Boxel monorepo, and introduces a sample card + live-test module alongside a shared copy script.

Changes:

Removed the existing catalog acceptance tests and the listing-update-specs integration test from this repo.
Added a reusable copy-catalog-to-monorepo.sh script and updated CI workflows to use it.
Added sample-command-card.gts and sample-command-card-test.gts for live-test coverage.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/integration/listing-update-specs-test.gts	Deletes the `listing-update-specs` integration test.
tests/acceptance/real-catalog-app-test.gts	Deletes the “real catalog app” acceptance test.
tests/acceptance/catalog-app-test.gts	Deletes the large catalog acceptance suite.
scripts/copy-catalog-to-monorepo.sh	Adds a shared script to wipe/sync catalog files into the Boxel monorepo.
sample-command-card.gts	Adds a sample realm card, but currently includes embedded test-only imports/code.
sample-command-card-test.gts	Adds a live-test module that exercises saving + search indexing for the sample card.
.github/workflows/ci-test.yaml	Renames job and switches CI to run `pnpm test:live` plus uploads a junit artifact.
.github/workflows/ci-lint.yaml	Switches the copy step to use the new script.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-25T09:18:14Z

scripts/copy-catalog-to-monorepo.sh

+echo "[copy-catalog] Syncing catalog source into monorepo..."
+rsync -av \
+  --exclude='.git/' \
+  --include='index.json' \
+  --exclude='*.json' \
+  --exclude='.gitignore' \
+  --exclude='tests/' \
+  --exclude='scripts/' \
+  --exclude='system-card/' \
+  "$CATALOG_SRC/" "$CATALOG_REALM/"


The rsync excludes all *.json files (except index.json) after the script has removed most of packages/catalog-realm. That combination will wipe out the catalog realm’s JSON card content (e.g. Category/Tag/Theme/Listing instances) in the Boxel checkout and not restore it, which will likely break the realm and any tests that depend on those cards. Consider either (a) copying JSON cards from boxel-catalog-src (remove --exclude='*.json' and instead exclude only the config JSONs you don’t want to overwrite), or (b) if JSON content should remain from the monorepo, update the find keep-list to preserve those directories so they aren’t deleted.

Copilot · 2026-03-25T09:18:14Z

sample-command-card.gts

+// ── Tests (imports resolved via loader.shimModule in live-test.js) ────────────
+import { on } from '@ember/modifier'; import { service } from '@ember/service';
+import { click, render, waitUntil } from '@ember/test-helpers';
+import GlimmerComponent from '@glimmer/component';
+import { module, test } from 'qunit';
+
+import { type Store } from '@cardstack/runtime-common';
+import {
+  setupCardTest,
+  setupIntegrationTestRealm,
+  testRealmURL,
+  withCachedRealmSetup,
+  type TestContextWithSave,
+} from '@cardstack/host/tests/helpers';
+import { TestRealmAdapter } from '@cardstack/host/tests/helpers/adapter';
+


sample-command-card.gts mixes a realm card module with test-only dependencies (@ember/test-helpers, qunit, @cardstack/host/tests/helpers, etc.). Outside the live-test harness these imports won’t resolve for a normal realm loader/indexer, so this card module can fail to load (and it also pulls test-only code into the realm bundle). Suggest splitting this so sample-command-card.gts contains only the card definition (and any runtime UI), and keep all QUnit/Ember-test code in sample-command-card-test.gts (or another test-only file that won’t be shipped/loaded by the realm).

Copilot · 2026-03-25T09:18:14Z

sample-command-card.gts

+} from 'https://cardstack.com/base/card-api';
+import StringField from 'https://cardstack.com/base/string';
+// ── Tests (imports resolved via loader.shimModule in live-test.js) ────────────
+import { on } from '@ember/modifier'; import { service } from '@ember/service';


Two separate import statements are on the same line, which is likely to fail formatting/lint rules and makes diffs harder to read. Please split these into separate lines (and let the formatter handle ordering).

Suggested change

import { on } from '@ember/modifier'; import { service } from '@ember/service';

import { on } from '@ember/modifier';

import { service } from '@ember/service';

Copilot · 2026-03-25T09:18:15Z

sample-command-card.gts

+  static isolated = class Isolated extends Component<typeof SampleCommandCard> {
+    <template>
+      <h1><@fields.title /></h1>
+      <button type='button'>Create Card</button>


The card’s isolated template renders a “Create Card” button, but it isn’t wired to any action (the click handler exists only in the separate CreateCardButton test component). If this button is meant to demonstrate the command behavior, consider rendering CreateCardButton (or adding an {{on 'click' ...}} handler) in the isolated template; otherwise remove the button to avoid a misleading UI.

Suggested change

<button type='button'>Create Card</button>

Copilot · 2026-03-25T09:18:15Z

sample-command-card-test.gts

+      await render(<template><CreateCardButton /></template>);
+      await click('button');
+
+      assert.ok(savedUrl, 'card was saved to realm');
+      let relativePath = `${savedUrl!.href.substring(testRealmURL.length)}.json`;
+      let file = await testRealmAdapter.openFile(relativePath);
+      assert.ok(file, 'card JSON file exists in the realm adapter');
+    });


This test asserts savedUrl immediately after click('button'), but the save callback can run asynchronously depending on how store.add flushes writes. That can make the test flaky and can lead to savedUrl still being undefined when computing relativePath. Suggest waiting until savedUrl is set (as you already do in the next test) before the assertions and file lookup.

Copilot · 2026-03-25T09:18:15Z

sample-command-card-test.gts

+      await render(<template><CreateCardButton /></template>);
+      await click('button');
+


Using click('button') is brittle if additional buttons get introduced by the test harness or template changes. Prefer targeting a specific button via a data-test-* attribute (or a more specific selector) so the test intent stays stable.

Copilot · 2026-03-25T09:18:15Z

.github/workflows/ci-test.yaml

+  live-test:
+    name: Live Tests (realm)
    runs-on: ubuntu-latest


The PR title says "Add listing tests", but this change set removes the existing catalog acceptance/integration test suites and switches CI to a different pnpm test:live workflow. If the intent is to migrate away from the previous listing tests, consider updating the PR title/description to match, or reintroducing the listing-specific coverage in the new live test suite so coverage doesn’t regress.

tintinthong added 18 commits March 24, 2026 15:38

try to run live test from ci

028c96b

attempt to run test which is colocated

9627919

try again to copy .gts

1af61d5

temporarily remove test

5cf0d5f

cleanup the catalog tests

820b4ca

improve copy sh

7223bec

try re-including some files

b2b609a

try adding sample command card

a357b67

try sample-command-card again

ce76c37

add sample command card test

a83bf7c

try to not copy everything

7aab4a2

try remove -test

149c936

include index.json

927e963

try again with listing-create-test

7b61267

try to fix the import and index

5ab1315

remove listing test for now

cf83096

remove is-live-test ref

61fc314

Merge remote-tracking branch 'origin/main' into add-listing-create-test

9341883

richardhjtan requested a review from Copilot March 25, 2026 09:10

Copilot started reviewing on behalf of richardhjtan March 25, 2026 09:11 View session

richardhjtan approved these changes Mar 25, 2026

View reviewed changes

Copilot AI reviewed Mar 25, 2026

View reviewed changes

tintinthong merged commit ddf517f into main Mar 25, 2026
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add listing tests#251

Add listing tests#251
tintinthong merged 18 commits intomainfrom
add-listing-create-test

tintinthong commented Mar 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	import { on } from '@ember/modifier'; import { service } from '@ember/service';
	import { on } from '@ember/modifier';
	import { service } from '@ember/service';

		await render(<template><CreateCardButton /></template>);
		await click('button');

Conversation

tintinthong commented Mar 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants