Skip to content

test(client): distinguish variant test cases#25503

Merged
jason-ha merged 4 commits intomainfrom
test/distinguish-variants
Sep 24, 2025
Merged

test(client): distinguish variant test cases#25503
jason-ha merged 4 commits intomainfrom
test/distinguish-variants

Conversation

@jason-ha
Copy link
Contributor

@jason-ha jason-ha commented Sep 20, 2025

Several test suites run with varying environments, but report tests the same way (most meaningfully by name) that are tracked as the same by infrastructure.

Update default setup to prefix test case names by variance when it is present.

Reconfigure CommonJS testing as a variant and replace MOCHA_SPEC using new FLUID_TEST_MODULE_SYSTEM environment variable. CommonJS test cases and report file are prefixed by [CJS]. ESM tests remain unprefixed.

  • This "adds" 773 test cases.
  • The 130 test cases from @fluidframework/routerlicious-driver are effectively renamed as it only tests CommonJS.

Correct @fluid-internal/devtools-browser-extension mocha configuration also use @fluid-internal/mocha-test-setup specified modules instead of ignoring them with complete require property override. Will now report package tests in telemetry and suppress console output by default (in addition to respecting the test case variant renaming).

@fluidframework/core-interfaces which is unable to leverage @fluid-internal/mocha-test-setup has a cloned subset of logic and now generates reports. (Its test spec was restored to *.spec.*js which allowed src/test/types/fluidObjectTypes.ts to be restored from #25292 changes.)

  • This adds 770 test cases to tracking.

Note: packages @fluidframework/driver-definitions and @fluid-internal/test-driver-definitions are also used by @fluid-internal/mocha-test-setup. They were not updated like @fluidframework/core-interfaces as they have no runtime tests.

@github-actions github-actions bot added base: main PRs targeted against main branch area: dds Issues related to distributed data structures area: dds: propertydds area: dds: sharedstring area: dds: tree area: dev experience Improving the experience of devs building on top of fluid area: driver Driver related issues area: examples Changes that focus on our examples area: framework Framework is a tag for issues involving the developer framework. Eg Aqueduct area: loader Loader related issues area: odsp-driver area: runtime Runtime related issues area: tests Tests to add, test infrastructure improvements, etc labels Sep 20, 2025
Several test suites run with varying environments but report tests the same way (most meaningfully by name) that are tracked as the same by infrastructure.

Update default setup to prefix test case names by variance when it is present.

Reconfigure CommonJS testing as a variant and replace MOCHA_SPEC using new FLUID_TEST_MODULE_SYSTEM environment variable. CommonJS test cases are prefixed by `[CJS]`. ESM tests remain unprefixed.

`@fluidframework/core-interfaces` which is unable to leverage `@fluid-internal/mocha-test-setup` has a cloned subset of logic and now generates reports. Its test spec was restored to `*.spec.*js` which allowed `src/test/types/fluidObjectTypes.ts` to be restored from #25292 changes.
@jason-ha jason-ha force-pushed the test/distinguish-variants branch from 477e766 to 1bcd572 Compare September 20, 2025 00:15
@jason-ha jason-ha marked this pull request as ready for review September 20, 2025 18:40
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR standardizes test case identification by introducing a new environment variable FLUID_TEST_MODULE_SYSTEM to replace MOCHA_SPEC for CommonJS testing and implementing test case prefixing to distinguish test variants in reporting infrastructure.

Key changes:

  • Replace MOCHA_SPEC=dist/test with FLUID_TEST_MODULE_SYSTEM=CJS across all package.json files
  • Add automatic [CJS] prefixing to CommonJS test case names
  • Configure @fluidframework/core-interfaces to generate test reports with variant tracking

Reviewed Changes

Copilot reviewed 70 out of 70 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/test/mocha-test-setup/src/mochaHooks.ts Add logic to prefix test titles with variant when FLUID_TEST_MODULE_SYSTEM is set
packages/test/mocha-test-setup/mocharc-common.cjs Implement CJS detection and automatic spec/prefix configuration
packages/test/mocha-test-setup/README.md Document new FLUID_TEST_MODULE_SYSTEM environment variable
packages/common/core-interfaces/.mocharc.cjs Add mocha configuration for test reporting with variant support
packages/common/core-interfaces/src/test/mochaHooks.ts Add minimal hook for CJS test prefixing
packages/common/core-interfaces/test-config.json Add test reporter configuration
packages/common/core-interfaces/src/test/types/fluidObjectTypes.ts Remove unnecessary conditional wrapper around type tests
packages/common/core-interfaces/package.json Update test scripts to use new environment variable pattern
Multiple package.json files Replace MOCHA_SPEC with FLUID_TEST_MODULE_SYSTEM=CJS in test:mocha:cjs scripts

instead of overriding all modules; specify additional as additional to config helper.
@jason-ha jason-ha requested a review from Josmithr September 21, 2025 18:38
@jason-ha jason-ha requested a review from a team as a code owner September 24, 2025 00:43
@github-actions
Copy link
Contributor

🔗 No broken links found! ✅

Your attention to detail is admirable.

linkcheck output


> fluid-framework-docs-site@0.0.0 ci:check-links /home/runner/work/FluidFramework/FluidFramework/docs
> start-server-and-test "npm run serve -- --no-open" 3000 check-links

1: starting server using command "npm run serve -- --no-open"
and when url "[ 'http://127.0.0.1:3000' ]" is responding with HTTP status code 200
running tests using command "npm run check-links"


> fluid-framework-docs-site@0.0.0 serve
> docusaurus serve --no-open

[SUCCESS] Serving "build" directory at: http://localhost:3000/

> fluid-framework-docs-site@0.0.0 check-links
> linkcheck http://localhost:3000 --skip-file skipped-urls.txt

Crawling...

Stats:
  236987 links
    1726 destination URLs
    1962 URLs ignored
       0 warnings
       0 errors


@jason-ha
Copy link
Contributor Author

I made silly mistake of trying to run performance pipeline against the test branch which gets picked up by our checks. It is failing per current dependency issues.

@jason-ha jason-ha enabled auto-merge (squash) September 24, 2025 18:09
@jason-ha jason-ha merged commit ff4cb44 into main Sep 24, 2025
40 checks passed
@jason-ha jason-ha deleted the test/distinguish-variants branch September 24, 2025 18:20
anthony-murphy-agent pushed a commit to anthony-murphy-agent/FluidFramework that referenced this pull request Jan 14, 2026
Several test suites run with varying environments, but report tests the
same way (most meaningfully by name) that are tracked as the same by
infrastructure.

Update default setup to prefix test case names by variance when it is
present.

Reconfigure CommonJS testing as a variant and replace MOCHA_SPEC using
new FLUID_TEST_MODULE_SYSTEM environment variable. CommonJS test cases
and report file are prefixed by `[CJS]`. ESM tests remain unprefixed.
- This "adds" 773 test cases.
- The 130 test cases from `@fluidframework/routerlicious-driver` are
effectively renamed as it only tests CommonJS.

Correct `@fluid-internal/devtools-browser-extension` mocha configuration
also use `@fluid-internal/mocha-test-setup` specified modules instead of
ignoring them with complete `require` property override. Will now report
package tests in telemetry and suppress console output by default (in
addition to respecting the test case variant renaming).

`@fluidframework/core-interfaces` which is unable to leverage
`@fluid-internal/mocha-test-setup` has a cloned subset of logic and now
generates reports. (Its test spec was restored to `*.spec.*js` which
allowed `src/test/types/fluidObjectTypes.ts` to be restored from microsoft#25292
changes.)
- This adds 770 test cases to tracking.

Note: packages `@fluidframework/driver-definitions` and
`@fluid-internal/test-driver-definitions` are also used by
`@fluid-internal/mocha-test-setup`. They were not updated like
`@fluidframework/core-interfaces` as they have no runtime tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: dds: propertydds area: dds: sharedstring area: dds: tree area: dds Issues related to distributed data structures area: dev experience Improving the experience of devs building on top of fluid area: driver Driver related issues area: examples Changes that focus on our examples area: framework Framework is a tag for issues involving the developer framework. Eg Aqueduct area: loader Loader related issues area: odsp-driver area: runtime Runtime related issues area: tests Tests to add, test infrastructure improvements, etc base: main PRs targeted against main branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants