feat(Example): add checkmark script for tests by sgaczol · Pull Request #4087 · software-mansion/react-native-screens

sgaczol · 2026-05-22T14:43:00Z

Description

This PR adds a new utility script that automatically collects information about test scenarios and generates readable Markdown tables based on their metadata.
The script processes the collected data and outputs it into two separate sections: Smoke Tests and Non-Smoke Tests. Currently, the generated tables include the following columns: Mark (rendered as an empty checkbox [ ]), Test Name, E2E Coverage and Key. The entries in both tables are sorted first by E2E coverage priority, and then alphabetically by their file path. This ensures that tests related to the same features or areas are grouped logically together in the list.

If you want to add a new property or column to the table in the future, you simply need to add a new object to the TABLE_COLUMNS array inside the script.

To execute the script, simply run it using Node.js:
node path-to/generate-test-checklist.ts

By default, the script searches the standard test directory, but you can override this behavior by passing a custom path using the --test-path <path> flag.

Note

Adding tsconfig.json and .eslintrc.js files to scripts directory will be handled in a separate PR.

Changes

add generate-test-checklist.ts in scripts/release directory

Visual documentation

That's how tables are meant to look like:

Test plan

N/A

Checklist

Included code example that can be used to test this change.
For visual changes, included screenshots / GIFs / recordings documenting the change.
For API changes, updated relevant public types.
Ensured that CI passes

Copilot

Pull request overview

Adds a release utility script intended to generate a Markdown checklist/table of test scenarios by discovering scenario-description.ts files under the app tests directory and grouping them into Smoke vs Non-Smoke sections.

Changes:

Introduces scripts/release/generate-test-checklist.ts to crawl the tests tree and collect scenario metadata.
Adds Markdown table generation with configurable columns and sorting/grouping logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kkafar · 2026-05-25T09:10:08Z

+async function parseScenarioFile(
+  filePath: string,
+): Promise<ScenarioDescription | null> {
+  try {
+    const module = require(filePath);
+    return module[OBJECT_NAME];
+  } catch (e) {
+    console.error(
+      `Warning: could not import ${filePath}: ${(e as Error).message}`,


This is something I recall you told me last week. Don't we need the scenario-description files to migrate to non-default exports first?

Yes, the change regarding exports is already merged to main branch. I've just rebased so it's all good

kkafar · 2026-05-25T09:06:12Z

+const fs = require('fs');
+const path = require('path');
+import type { ScenarioDescription } from '../../apps/src/tests/shared/helpers.ts';
+
+type E2eCoverage = ScenarioDescription['e2eCoverage'];
+


Inaccurate. Newer node versions (which we use) run TS directly.

Only thing I'm concerned here is that indeed, we mix require style imports for modules with import directives for types. This looks weird, however I dunno whether we can workaround this one. Please do 10-min of research here.

@sgaczol any results of the research?

Mixing require with import is generally incorrect, but in this case it is admissible as we only use import for a type (which does not exist in runtime). However, if we switch to "type": "module" in the future, we should change all require calls to import.

kkafar · 2026-05-25T09:02:14Z

@@ -0,0 +1,166 @@
+const fs = require('fs');
+const path = require('path');
+import type { ScenarioDescription } from '../../apps/src/tests/shared/helpers.ts';


Bot is right here.

Additionally, its smelly to import things across modules like this, but I guess it'll do for now. Otherwise we'd have to duplicate the ScenarioDescription definition which is not great either.

The proper solution here would be to move these symbols to some third, shared module. This will be doable easily after we migrate to monorepo setup.

Let's create ticket for the above and link it here, please ☝🏻

@sgaczol can we have the ticket linked?

Ticket: https://github.com/software-mansion/react-native-screens-labs/issues/1492
Commit regarding import: b303bdb

kkafar · 2026-05-25T09:11:21Z

+function compareEntries(a: Entry, b: Entry): number {
+  const coverageDiff =
+    COVERAGE_ORDER[a.e2eCoverage] - COVERAGE_ORDER[b.e2eCoverage];
+  if (coverageDiff !== 0) {
+    return coverageDiff;
+  }
+
+  if (a.subdirectory !== b.subdirectory) {
+    return a.subdirectory.localeCompare(b.subdirectory);
+  }
+
+  return a.name.localeCompare(b.name);
+}


I think +1 here. We should sort by key instead of name.

0b64c63 + f2f4c0c

sgaczol · 2026-05-25T09:48:54Z

+  const cellValue = (col: TableColumn, entry: Entry) =>
+    col.prop
+      ? String(entry[col.prop as keyof Entry] ?? 'N/A')
+      : col.defaultValue!;
+


kkafar

Hey! This looks good. Good job.

I have few remarks. Please answer them before proceeding.

kkafar · 2026-05-25T09:02:14Z

@@ -0,0 +1,166 @@
+const fs = require('fs');
+const path = require('path');
+import type { ScenarioDescription } from '../../apps/src/tests/shared/helpers.ts';


Bot is right here.

Additionally, its smelly to import things across modules like this, but I guess it'll do for now. Otherwise we'd have to duplicate the ScenarioDescription definition which is not great either.

The proper solution here would be to move these symbols to some third, shared module. This will be doable easily after we migrate to monorepo setup.

Let's create ticket for the above and link it here, please ☝🏻

kkafar · 2026-05-25T09:02:36Z

@@ -0,0 +1,166 @@
+const fs = require('fs');
+const path = require('path');
+import type { ScenarioDescription } from '../../apps/src/tests/shared/helpers.ts';


Bot is right here.

Additionally, its smelly to import things across modules like this, but I guess it'll do for now. Otherwise we'd have to duplicate the ScenarioDescription definition which is not great either.

The proper solution here would be to move these symbols to some third, shared module. This will be doable easily after we migrate to monorepo setup.

Let's create ticket for the above and link it here, please ☝🏻

kkafar · 2026-05-25T09:06:12Z

+const fs = require('fs');
+const path = require('path');
+import type { ScenarioDescription } from '../../apps/src/tests/shared/helpers.ts';
+
+type E2eCoverage = ScenarioDescription['e2eCoverage'];
+


Inaccurate. Newer node versions (which we use) run TS directly.

Only thing I'm concerned here is that indeed, we mix require style imports for modules with import directives for types. This looks weird, however I dunno whether we can workaround this one. Please do 10-min of research here.

kkafar · 2026-05-25T09:10:08Z

+async function parseScenarioFile(
+  filePath: string,
+): Promise<ScenarioDescription | null> {
+  try {
+    const module = require(filePath);
+    return module[OBJECT_NAME];
+  } catch (e) {
+    console.error(
+      `Warning: could not import ${filePath}: ${(e as Error).message}`,


This is something I recall you told me last week. Don't we need the scenario-description files to migrate to non-default exports first?

kkafar · 2026-05-25T09:11:21Z

+function compareEntries(a: Entry, b: Entry): number {
+  const coverageDiff =
+    COVERAGE_ORDER[a.e2eCoverage] - COVERAGE_ORDER[b.e2eCoverage];
+  if (coverageDiff !== 0) {
+    return coverageDiff;
+  }
+
+  if (a.subdirectory !== b.subdirectory) {
+    return a.subdirectory.localeCompare(b.subdirectory);
+  }
+
+  return a.name.localeCompare(b.name);
+}


I think +1 here. We should sort by key instead of name.

LKuchno

Looks good, thank you!
Just FYI, markdown does not support checkboxes within tables. Therefore, if we wish to mark an item in this table, we will need to manually enter an "x" inside [ ], as opposed to clicking a checkbox (as it works in the PR checklist). I will create a task to update the test release scope with information regarding this limitation and the script usage.

## Description Adding `tsconfig.json` and `.eslintrc.js` to `scripts` directory is necessary in order for the new script, introduced in #4087, to work properly. ## Changes - add `tsconfig.json` and `.eslintrc.js` to `scripts` directory ## Before & after - visual documentation N/A ## Test plan N/A ## Checklist - [ ] Included code example that can be used to test this change. - [ ] For visual changes, included screenshots / GIFs / recordings documenting the change. - [ ] For API changes, updated relevant public types. - [ ] Ensured that CI passes --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.

sgaczol · 2026-05-25T13:23:08Z

+async function parseScenarioFile(
+  filePath: string,
+): Promise<ScenarioDescription | null> {
+  try {
+    const module = require(filePath);
+    return module[OBJECT_NAME];


According to TS documentation:
.ts/.tsx/.js/.jsx/.d.ts files are ES modules if the nearest ancestor package.json file contains "type": "module", otherwise CommonJS modules
So using require(filePath) is correct

sgaczol · 2026-05-25T12:42:31Z

+  if (a.key !== b.subdirectory) {
+    return a.subdirectory.localeCompare(b.subdirectory);
+  }
+
+  return a.key.localeCompare(b.key);


sgaczol · 2026-05-25T13:09:24Z

@@ -0,0 +1,164 @@
+const fs = require('fs');
+const path = require('path');
+import type { ScenarioDescription } from '../../apps/src/tests/shared/helpers.ts';


sgaczol · 2026-05-25T13:10:08Z

+  const customTestPath = getCustomTestPath();
+  const searchDir = customTestPath
+    ? path.resolve(process.cwd(), customTestPath)
+    : path.resolve(__dirname, DEFAULT_TESTS_DIR);


…scription

kkafar

Okay, just tested it and seems to work fine, both with default settings and when specifying the --test-path option.

Since the code here is rather hard to follow, one more thing we're missing is a comment in the very top of the file, describing what this script does and documenting input it receives (--test-path parameter). Otherwise nobody is going to be able to decipher that even such option exists.

sgaczol · 2026-05-27T10:12:11Z

Okay, just tested it and seems to work fine, both with default settings and when specifying the --test-path option.

Since the code here is rather hard to follow, one more thing we're missing is a comment in the very top of the file, describing what this script does and documenting input it receives (--test-path parameter). Otherwise nobody is going to be able to decipher that even such option exists.

Added in 650b895

sgaczol requested review from LKuchno, Copilot, kkafar, kligarski, kmichalikk and t0maboro May 25, 2026 08:54

sgaczol marked this pull request as ready for review May 25, 2026 08:54

Copilot started reviewing on behalf of sgaczol May 25, 2026 08:55 View session

Copilot AI reviewed May 25, 2026

View reviewed changes

sgaczol force-pushed the @sgaczol/tests-script branch from 716b6dd to 2f40512 Compare May 25, 2026 09:12

kkafar reviewed May 25, 2026

View reviewed changes

sgaczol mentioned this pull request May 25, 2026

chore: add typescript configuration files to scripts directory #4093

Merged

4 tasks

LKuchno approved these changes May 25, 2026

View reviewed changes

sgaczol requested a review from kkafar May 25, 2026 10:59

kkafar requested a review from Copilot May 25, 2026 12:24

Copilot started reviewing on behalf of kkafar May 25, 2026 12:25 View session

Copilot AI reviewed May 25, 2026

View reviewed changes

kkafar requested changes May 25, 2026

View reviewed changes

Comment thread scripts/release/generate-test-checklist.ts Outdated

sgaczol added 6 commits May 25, 2026 14:47

add checkmark script for tests

0dd0368

defaultValue is necessary for values that are not props in ScenarioDe…

befaa33

…scription

sort by key instead of name

a5efdd7

typo fix

f2f4c0c

DEFAULT_TESTS_DIR fix

d12a21c

update import

b303bdb

sgaczol force-pushed the @sgaczol/tests-script branch from ff37bf3 to b303bdb Compare May 25, 2026 13:08

kkafar reviewed May 27, 2026

View reviewed changes

add description

650b895

sgaczol requested a review from kkafar May 27, 2026 10:12

Uh oh!

Conversation

sgaczol commented May 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

Visual documentation

Test plan

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sgaczol May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sgaczol May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sgaczol May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sgaczol May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kkafar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LKuchno left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kkafar left a comment

Choose a reason for hiding this comment

Uh oh!

sgaczol commented May 27, 2026

sgaczol commented May 22, 2026 •

edited

Loading

sgaczol May 25, 2026 •

edited

Loading

sgaczol May 25, 2026 •

edited

Loading

sgaczol May 25, 2026 •

edited

Loading

sgaczol May 25, 2026 •

edited

Loading

LKuchno left a comment •

edited

Loading