Skip to content
Testomatio

CLI

The Testomat.io Reporter CLI is a powerful tool for managing test runs, parsing XML reports, and uploading artifacts. CLI can be used to start and finish test runs, run tests, parse XML reports, and upload artifacts. It can be used in CI/CD pipelines or locally.

Reporter is designed to work with Testomat.io service but not exclusively

Installation

Section titled “Installation”
Terminal window
npm install -g @testomatio/reporter

For Yarn 4 (Berry), install and run CLI wrapper:

Terminal window
yarn add -D testomatio-reporter-cli
npx testomatio-reporter <command> [options]

General Usage

Section titled “General Usage”
Terminal window
npx @testomatio/reporter <command> [options]

Available Commands

Section titled “Available Commands”

1. start

Section titled “1. start”

Starts a new test run and returns its ID. This requires an API key to be set in the TESTOMATIO environment variable.

Usage:

Terminal window
npx @testomatio/reporter start [options]

Examples:

Terminal window
npx @testomatio/reporter start
npx @testomatio/reporter start --kind manual
npx @testomatio/reporter start --kind mixed

Environment Variables:

  • TESTOMATIO: Your Testomat.io API project key in format: tstmt_* (required).

Options:

  • --env-file <envfile>: Load environment variables from a specific env file. If none specified, it will look for .env file.
  • --kind <type>: Specify run type: automated, manual, or mixed. Determines how the test run is categorized in Testomat.io.

Previously known as: npx start-test-run --launch (before 1.6.0)

2. finish

Section titled “2. finish”

Finishes a run by its ID.

Usage:

Terminal window
npx @testomatio/reporter finish [options]

Environment Variables:

  • TESTOMATIO: Your Testomat.io API key in format: tstmt_* (required).

Options:

  • --env-file <envfile>: Load environment variables from a specific env file. If none specified, it will look for .env file.

Previously known as: npx start-test-run --finish (before 1.6.0)

3. run

Section titled “3. run”

Creates a run, and runs tests with the specified command, then finishes the run. This command ensures that if test runner spawns parallel workers or processes and reports it will report to the same run.

Use npx @testomatio/reporter run if multiple run reports are created on the same launch

Usage:

Terminal window
npx @testomatio/reporter run [options]

Alias for this command – test, e.g. npx @testomatio/reporter test [options]

Environment Variables:

  • TESTOMATIO: Your Testomat.io API key in format: tstmt_* (required).

Options:

  • -c, --command <cmd>: Test runner command (required).
  • --filter <filter>: Filter executed tests by tag, label, jira, plan.
  • --filter-list <filter>: Print the list of tests matching the filter without running them. Useful for inspecting which tests would run, or for piping IDs into another command. See Coverage Pipe for examples.
  • --format <format>: Machine-readable output format for --filter-list. Supported values: grep, json, newline, ids. When set, the CLI banner is suppressed and informational logs go to stderr so stdout stays clean for piping.
  • --env-file <envfile>: Load environment variables from a specific env file.
  • --kind <type>: Specify run type: automated, manual, or mixed. Determines how the test run is categorized in Testomat.io.

Examples:

Terminal window
npx @testomatio/reporter run "npm test"
npx @testomatio/reporter run "npx jest" --filter-list "testomatio:tag-name=frontend"
npx @testomatio/reporter run "npx jest" --filter "testomatio:tag-name=frontend"
npx @testomatio/reporter run "npx jest" --filter "testomatio:label=Smoke"
npx @testomatio/reporter run "npx jest" --filter "testomatio:jira=TC-123"
npx @testomatio/reporter run "npx jest" --filter "testomatio:plan=a123fb12"
npx @testomatio/reporter run "npx jest" --filter-list "coverage:file=coverage.yml,diff=user-branch"
npx @testomatio/reporter run --filter-list "coverage:file=coverage.yml" --format grep
npx @testomatio/reporter run "npx jest" --filter "coverage:file=coverage.yml,diff=user-branch"
npx @testomatio/reporter run "npx jest" --filter "coverage:file=coverage/coverage.yml"
npx @testomatio/reporter run "mocha tests/" --env-file .env.test
npx @testomatio/reporter run "npm test" --kind manual
npx @testomatio/reporter run "npx jest" --kind mixed

3.1 Filter pipes

Section titled “3.1 Filter pipes”

--filter and --filter-list only work with the testomatio: and coverage: pipes (e.g. testomatio:tag-name=smoke, coverage:file=coverage.yml). Any other prefix is rejected.

3.2 Machine-readable output with --format

Section titled “3.2 Machine-readable output with --format”

When --filter-list is set, the matched test IDs are printed to stdout in a format suitable for piping. The CLI banner is suppressed, progress logs are silenced, and any remaining warnings/errors go to stderr — so the terminal shows only the test list, and the output is safe to copy or capture.

Set TESTOMATIO_LOG_LEVEL=INFO to bring the progress logs back for debugging.

Exit codes: --filter-list exits 0 when at least one test matched, and 1 when no tests matched or filter resolution failed. CI scripts can branch on the exit code to skip launching the runner when there’s nothing to run.

The default format is ids (comma-separated). Use --format to switch:

  • grep — alternation wrapped in parens, e.g. (t1|t2|t3)
  • json — JSON array, e.g. ["t1","t2","t3"]
  • newline — one ID per line
  • ids — comma-separated (default)

Example: pipe matched tests straight into a runner’s grep flag

Terminal window
GREP=$(npx @testomatio/reporter run --filter-list "coverage:file=coverage.yml" --format grep)
npx playwright test --grep "$GREP"

Example: capture IDs as JSON for further processing

Terminal window
npx @testomatio/reporter run --filter-list "coverage:file=coverage.yml" --format json > affected-tests.json

See the Coverage Pipe docs for more details on each format.

Previously known as: npx start-test-run -c "command" (before 1.6.0)

4. xml

Section titled “4. xml”

Parses XML reports in JUnit (NUnit, xUnit, TRX) XML format and uploads them to Testomat.io or uses GitHub / GitLab / ButBucket pipe to create a mini-report

Usage:

Terminal window
npx @testomatio/reporter xml <pattern> [options]

Environment Variables:

  • TESTOMATIO: Your Testomat.io API key in format: tstmt_* if you want to upload reports to Testomat.io (optional).

Arguments:

  • <pattern>: XML file pattern (required).

Options:

  • -d, --dir <dir>: Project directory.
  • --java-tests [java-path]: Load Java tests from path (default: src/test/java).
  • --lang <lang>: Language used (python, ruby, java).
  • --timelimit <time>: Default time limit in seconds to kill a stuck process.
  • --env-file <envfile>: Load environment variables from a specific env file.

Examples:

Terminal window
npx @testomatio/reporter xml "test-results/**.xml" --lang python
npx @testomatio/reporter xml "junit-reports/**.xml" -d ./project --lang java
npx @testomatio/reporter xml "pytest-results.xml" --timelimit 300 --env-file .env.test

Previously known as: npx report-xml (before 1.6.0)

5. upload-artifacts

Section titled “5. upload-artifacts”

Testomat.io reporter automatically uploads artifacts during run. However, either some artifacts failed to upload or you intentioanlly disabled file upload during tests to speed up reporting. In this case you can use this command to upload artifacts after the run.

It is important to have the TESTOMATIO_RUN environment variable set to the run ID.

Usage:

Terminal window
npx @testomatio/reporter upload-artifacts [options]

Environment Variables:

  • TESTOMATIO: Your Testomat.io API key (required).
  • TESTOMATIO_RUN: The previous run ID you want to upload artifacts (optional). If none set, latest run will be used.

Options:

  • --force: Re-upload artifacts even if they were uploaded before.
  • --env-file <envfile>: Load environment variables from a specific env file.

You still need S3 artifacts configuration to be set to upload artifacts to storage. In order to disable artifacts upload during tests you can use TESTOMATIO_DISABLE_ARTIFACTS=1 while running tests.

Examples:

Terminal window
npx @testomatio/reporter upload-artifacts
npx @testomatio/reporter upload-artifacts --force
npx @testomatio/reporter upload-artifacts --env-file .env.prod

With Playwright:

Terminal window
TESTOMATIO=tstmt_* TESTOMATIO_DISABLE_ARTIFACTS=1 npx playwright test
TESTOMATIO=tstmt_* npx @testomatio/reporter upload-artifacts

With webdriverio:

Terminal window
TESTOMATIO=tstmt_* TESTOMATIO_DISABLE_ARTIFACTS=1 npx @testomatio/reporter run "npx wdio"
TESTOMATIO=tstmt_* npx @testomatio/reporter upload-artifacts

You can also upload small artifacts during the run, while large files can be uploaded after. For instance, all files uploading larger than 10MB will be skipped during the run.

Terminal window
TESTOMATIO=tstmt_* TESTOMATIO_ARTIFACT_MAX_SIZE_MB=10 npx playwright test
TESTOMATIO=tstmt_* npx @testomatio/reporter upload-artifacts

However, upload-artifacts command will upload all files after the run, without blocking the final result.

6. replay

Section titled “6. replay”

The replay command allows you to re-send test data from debug files to Testomat.io. This is useful when your original test run failed to upload results properly.

Important: To make replay work, tests should be executed with DEBUG=1 variable set, to ensure they are running in debug mode and save data into a file.

Usage:

Terminal window
npx @testomatio/reporter replay [debug-file] [options]

Arguments:

  • debug-file (optional) - Path to debug file. Defaults to ./testomatio.debug.json.

Options:

  • --dry-run - Preview the data without sending to Testomat.io
  • --env-file <envfile> - Load environment variables from env file

Examples:

Terminal window
TESTOMATIO=<your-api-key> DEBUG=1 npx playwright test




TESTOMATIO=<your-api-key> npx @testomatio/reporter replay




TESTOMATIO=<your-api-key> npx @testomatio/reporter replay /path/to/debug.json




TESTOMATIO=<your-api-key> npx @testomatio/reporter replay --dry-run




npx @testomatio/reporter replay --env-file .env.staging

How it works:

The replay command uses the ReplayService class (located in src/replay.js) to:

  1. Parse the debug file line by line
  2. Extract environment variables, run parameters, test data, and finish parameters
  3. Restore environment variables (without overriding existing ones)
  4. Create a new test run using the TestomatClient
  5. Send each test result individually
  6. Update the run status when complete

For more details about debug files, see the Debug Pipe documentation.

Environment Variables

Section titled “Environment Variables”

Many commands rely on environment variables. You can set these in a command line, in a .env file, or use the --env-file option to specify a custom env file. Important variables include:

  • TESTOMATIO: Your Testomat.io API key.
  • TESTOMATIO_RUN: The current run ID (usually set automatically by the start command).
  • TESTOMATIO_TITLE: Title for the test run (optional).
  • TESTOMATIO_STACK_ARTIFACTS: Save large stack traces and steps as artifacts to avoid API size limits.
  • more..

Tips

Section titled “Tips”
  1. Always ensure your Testomat.io API key is properly set before running commands.
  2. Use the --env-file option when you have different configurations for various environments.
  3. The run command is versatile and can be used with various test runners like Jest, Mocha, or custom scripts.
  4. When using the xml command, ensure your XML reports are in a supported format.
  5. The upload-artifacts command is useful for adding additional files or logs to your test runs.

For more detailed information or troubleshooting, please refer to the Testomat.io documentation or contact support.