mocha-summary-report

Generates summary reports of all the suites executed in a single Mocha execution. This reporter also works when run in --parallel mode.

Latest Build Report

https://cdtester.github.io/mocha-summary-report/summary-report.html

Console Summary Feature

Provides a summary for each suite and a total summary of all the suites in the terminal console.

This report can be turned off via the report options.

Text File Summary Feature

Provides a summary of all the suites in a text file. This text file can be added to an email.

This report can be turned off via the report options.

HTML Summary Feature

Provides a summary of when and where the suites were executed, a summary of all the suites and a summary of each test (including the test steps) in a HTML file.

The HTML report has a responsive web design to match most screen sizes.

Large Screens

Small Screens

Test Suite Details Section

This Section provides information about the overall details about when and where the tests were executed.

Suite Details

This sub section provides information about the:

Version of the project release that is being tested. This information comes from process.env variable, this can be set in the CI server parameters or added to npm_config_ varaibles either in your npmrc file or as a custom option on the execution line. The location can be configured in the report options.
Test Cycle that this test executed as part of. Similar to Version, this is configurable.
Test Environment that this test executed as part of. Similar to Version, this is configurable.
Test Types this shows any test phase found in all the tags in your tests that were executed.
Components this shows any tag found your tests that are not a test type/phase.

Test Types/phases is customizable, the default options are 'smoke' and 'regression'. The way this project customizes the report is via the ./config/default.json. See the reporter-options section.

Duplicate tags are removed. The tag should be placed in your test file on the Mocha Describe feature. Tags can be useful to denote what type of test (e.g. smoke, regression etc) and/or the feature of the project. The tag prefix can be configured in the reporter options.

Run Time Details

This sub section provides information about the:

Run Date of the test execution.
Start Time of the the first test execution.
End Time of the last test execution.
Duration of the total test executions.
Parallel Run, whether the tests were executed in parallel or not.

CI Server Details

This section can show some limited information about the CI/CD server the test was executed on. A custom set of functions have been created based on env-ci package, this is due to that package uses import to load modules and I could not get it to work with the mocha-multi-reporters which uses require to load modules. This version of the package works with the following CI servers:

Appveyor
Azure Devops/Pipelines
Bamboo
Bitbucket
Bitrise
Buddy
Buildkite
CircleCI
Cirrus CI
Cloudflare Pages
AWS CodeBuild
Codefresh
Codeship
Drone
GitHub Actions
GitLab CI
Jenkins
JetBrains Space
Netlify
Puppet
Sail CI
Screwdriver.cd
Scrutinizer
Semaphore
Shippable
TeamCity
Travis CI
Vela
Vercel
Wercker
Woodpecker CI

If the tests are run on a local machine, then the sub section heading will indicate this.

Summary Section

Test Suite Summary

This section provides an overall status of how many tests:

Run in total
Passed, a test that has all steps passed.
Skipped, a test that was not executed.
Failed, a test that has one or more failed step.

This section also provides metrics for:

Passed Rate of tests executed, whch uses the formula (passed tests / total executed tests) * 100.
Tests/s tests executed per second, which uses the formula (passed tests + failed tests) / total execution time in seconds. This metric can be used to comapre against manual test execution rates.

Test Step Summary

This section provides an overall status of how many test steps:

Run in total
Passed, a test step that has a passed status.
Skipped, a test step that was not executed.
Failed, a test step that has a failed status.

This section also provides metrics for:

Passed Rate of test steps executed, whch uses the formula (passed test steps / total executed Test steps) * 100.
Steps/s steps executed per second, which uses the formula (passed test steps + failed test steps) / total execution time in seconds. This metric can be used to comapre against manual test execution rates.

Risk Summary

This section details the risks/confidence of your test suite. It assesses whether you have run any flaky tests (requires your test to have a @flaky tag in the title). It assesses how many tests were skipped and whether any of those skipped tests were flaky. And finally it takes those assesments and adjusts your passed Rate percentage in to a confidence percentage.

Passed Rate = tests passed / (total tests - tests skipped) * 100
Coverage = (total tests - skipped tests) / total tests * 100
flaky percentage = flaky tests executed / (total tests) * 100
penalty ratio = 0.4
flaky penalty = flaky percentage * penalty ratio
confidence = Passed Rate * (1 - flaky penalty/100) * (1 - coverage/100)

This section is first broken down by total risk and test phase risk. Another section the breaks down the risk associated with each component (tags that are not test phases)

Suites Section

Links to other reports

If you use other test reports like Mochawesome, you will need to use a reporter like mocha-multi-reporters. This is how the Suites section is presented when no other reports are used during the Mocha test execution.

This is how the Suites section is presented when other reports are used during the Mocha test execution. A link button becomes available that opens the other report in a new browser tab.

This report can be turned off via the report options.

Test Suite run information

This section Provides detais of each test suite, including:

The Test ID. This should be a unique code that can identify your test on your test management tool. In this example, Zephyr tests are located on Jira and will have a Jira issue key. The test ID should be placed on the Mocha Describe feature where tags are detailed. The Test ID prefix can be configured in the reporter options. Also, the Test ID can be a link to your manual test. The url is configured in the reporter options, this will be the url base that precedes the Test ID.
Tags associated to the test. The tag should be placed in your test file on the Mocha Describe feature. Tags can be useful to denote what type of test (e.g. smoke, regression etc) and/or the feature of the project. The tag prefix can be configured in the reporter options.
Test Scenario title. The tags at test ID are stripped from the title.
Passed / Skipped / Failed How many steps in the tests Passed, skipped or failed.
The Duration of the test suite in seconds.

Test Step information

Each test suite can be expanded to reveal more details about the test steps status by clicking on the + icon in the Expand column. This new section shows the test step:

Step Number.
Step description.
Error message that was produced if the test step failed.
Retries the number of retries performed, not the max number of retries (unless the max limit was reached).
Status.
Duration in seconds. This duration includes the duration of each retry.

Install

Add mocha-summary-report to your project by adding it to to your package.json:

  "dependencies": {
    "mocha-summary-report": "github:CDTester/mocha-summary-report"
  },

Then install it with:

npm i

Usage

Usage examples can be found on github mocha-summary-report.

Configuring reporters

Using mocharc file

You can set up the reporter in a mocharc file and running the test with

mocha --config .mocharc.js

Where the mocharc file has the following options:

    reporter: 'mocha-summary-report',
    'reporter-option': [
        'environmentVar=testenv',
        'projectName=project_name',
        'projectVersion=project_version',
        'projectCycle=npm_test_cycle',
        'manualTestLink=manual_test_url',
        'output=test_report',
        'testIDPrefix=$',
        'tagPrefix=@',
        'consoleSummary=true',
        'textFileSummary=true',
        'htmlSummary=true',
        'passRateGreen=80',
        'passRateAmber=70'
    ],

Pointing the --reporter-option in mocharc file to a config file

This option can be used when using mocha-multi-reporters.

mocha --config .mocharc.js

Where the mocharc file has the following options:

    reporter: 'mocha-summary-report',
    'reporter-option': ['configFile=./config/default.json'],

And the config file has the following options:

{
    "reporterEnabled": "mocha-summary-report,mochawesome",
    "mochawesomeReporterOptions": {
        "reportDir": "test_report",
        "reportFilename": "test-report.html"
    },
    "mochaSummaryReportReporterOptions": {
        "environmentVar": "testenv",
        "projectName": "project_name",
        "projectVersion": "project_version",
        "projectCycle": "test_cycle",
        "manualTestLink": "manual_test_url",
        "output": "test_report",
        "testIDPrefix": "$",
        "tagPrefix": "@",
        "consoleSummary": "true",
        "textFileSummary": "true",
        "htmlSummary": "true",
        "passRateGreen": 80,
        "passRateAmber": 70,
        "otherReportLink": "test_report/test-report.html",
        "includeMochaBase": false
    }
}

Reporter Options

The summary reports can be configured in the reporter options:

Option Name	Type	Default	Description
consoleSummary	boolean	true	option to turn off the summary in the console log by setting to false.
textFileSummary	boolean	true	option to turn off the summary in a text file by setting to false.
htmlSummary	boolean	true	option to turn off the summary in a text file by setting to false.
tagPrefix	string	'@'	Prefix to be used to capture tags used in the tests.
testIDPrefix	string	'$'	Prefix to be used to capture the test ID used in the tests.
testPhases	Array[string]	['smoke', 'regression']	You can override the default values with tagged test phases in your tests. e.g ['unit', 'integration', 'e2e']
passRateGreen	string	'90'	Value used to determine the background colour of the overall success rate of which is deemed acceptable.
passRateAmber	string	'50'	Value used to determine the background colour of the overall success rate of which is deemed a warning.
output	string	'test_report'	Folder location to be used to save summary report to.
otherReportLink	string	undefined	File location of any other reports created at the end of the test execution.
includeMochaBase	true	option to turn off the Mocha Base reporter in the console. Mocha-multi-reporters includes Base reporting which duplicates this reporters output.
manualTestLink	string	undefined	process.env variable name that contains the base URL of your test management application. The testIDPrefix value is suffixed to this URL to create the full link.
projectName	string	'--'	process.env variable name that denotes the name of the project.
projectVersion	string	'--'	process.env variable name that denotes the version of the project being tested.
projectCycle	string	'--'	process.env variable name that denotes the test cycle name of the test execution.
environmentVar	string	'--'	process.env variable name that denotes the test environment.

The process.env variables can be set up in many ways:

as a command line to running your tests e.g. npm run test [spec] --project=ProjectName. (these will be set up with a prefix of npm_config_project. The 'npm_config_project' is the value that needs to be saved in the reporter option for projectName. )
in an .npmrc file (these will be set up with a prefix of npm_config_)
from your CI application.

Process Environment Variables

The summary reports can be configured to get information from the process.env, these can be added by either your CI/CD job or if you are running the tests for your local machine there is a envData section in the ./config/default.json. In the mocharc.js file, in the require section, a script runs before the tests iterate through the envData section and creates process.env variables, IF they are not already set up by you CI/CD applicaiton. You can name these whatever you like, but you must rememeber to refer to that env name in the reporter options section of this defaul.json file :

Option Name	Type	Description
TESTENV	string	the name of the test environment
PROJECT_NAME	string	the name of the project
RELEASE_VERSION	string	the name of the project version
TEST_CYCLE	string	the name of the test cycle
TMS_URL	string	the base url for you test management server

E.g.

{
       "mochaSummaryReportReporterOptions": {
        "environmentVar": "TESTENV",
        "projectName": "PROJECT_NAME",
        "projectVersion": "RELEASE_VERSION",
        "projectCycle": "TEST_CYCLE",
        "manualTestLink": "TMS_URL"
    },
    "envData": {
        "TESTENV": "dev",
        "ZEPHYR_UPDATE": "no",
        "PROJECT_NAME": "proj1",
        "RELEASE_VERSION": "1",
        "TEST_CYCLE": "Cycle 123",
        "TMS_URL": "https://company.domain/jira/browse/"
    } 
}

Name		Name	Last commit message	Last commit date
Latest commit History 45 Commits
.github/workflows		.github/workflows
config		config
docs		docs
src		src
test		test
.eslintrc.json		.eslintrc.json
.gitignore		.gitignore
.mocharc.js		.mocharc.js
.nojekyll		.nojekyll
.npmignore		.npmignore
.npmrc		.npmrc
LICENSE		LICENSE
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

mocha-summary-report

Latest Build Report

Console Summary Feature

Text File Summary Feature

HTML Summary Feature

Large Screens

Small Screens

Test Suite Details Section

Suite Details

Run Time Details

CI Server Details

Summary Section

Test Suite Summary

Test Step Summary

Risk Summary

Suites Section

Links to other reports

Test Suite run information

Test Step information

Install

Usage

Configuring reporters

Using mocharc file

Pointing the --reporter-option in mocharc file to a config file

Reporter Options

Process Environment Variables

About

Uh oh!

Releases 6

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

mocha-summary-report

Latest Build Report

Console Summary Feature

Text File Summary Feature

HTML Summary Feature

Large Screens

Small Screens

Test Suite Details Section

Suite Details

Run Time Details

CI Server Details

Summary Section

Test Suite Summary

Test Step Summary

Risk Summary

Suites Section

Links to other reports

Test Suite run information

Test Step information

Install

Usage

Configuring reporters

Using mocharc file

Pointing the --reporter-option in mocharc file to a config file

Reporter Options

Process Environment Variables

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 6

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages