TEST-RUN(1) Test Manual TEST-RUN(1)
NAME
test-run - run the tests
SYNOPSIS
test run [OPTION]…
DESCRIPTION
Run all or only some of the tests. By default, the status of each test
is reported as they are executed. Here's the legend for test statuses:
• [PASS]: a successful test that was expected to succeed (good);
• [FAIL]: a failing test that was expected to succeed (needs fixing);
• [XFAIL]: a failing test that was expected to fail (tolerated failure);
• [XPASS]: a successful test that was expected to fail (progress?).
• [MISS]: a test that never ran;
• [SKIP]: a test that is always skipped but kept around for some reason;
• [xxxx*]: a new test for which there's no expected output yet.
In this case, you should review the test output and run the 'approve'
subcommand once you're satisfied with the output.
To review the status of the tests without rerunning them, use the
'status' subcommand.
OPTIONS
--autoclean
Remove test snapshots that don't match any test.
--debug
Log information that can be useful for debugging the Testo library.
-e KEY=VALUE, --env=KEY=VALUE
Pass a key/value pair to the function that creates the test suite.
KEY must be an alphanumeric identifier of the form
[A-Za-z_][A-Za-z_0-9]*. VALUE can be any string. This mechanism for
passing arbitrary runtime settings to the test suite is offered as
a safer alternative to environment variables. Multiple -e/--env
options are supported in the same command, each defining one
key/value pair.
--expert
Assume the user is familiar with Testo and don't show non-essential
messages or tips targeted at new users.
-j NUM, --jobs=NUM
Specify the number of jobs to run in parallel. By default, this
value is set to the number of CPUs detected on the machine. Both
'-j0' and '-j1' ensure sequential, non-overlapping execution of the
tests. Unlike '-j1', '-j0' will not create a separate worker
process to run the tests. The default can be changed by passing
'~default_workers' to the OCaml function 'Testo.interpret_argv'.
NOTE: Parallel executation of tests is not stable on Windows.
--lazy
Run only the tests that were not previously successful.
--max-inline-log-bytes=LIMIT (absent=1000000)
When displaying logs (unchecked stdout or stderr output), show at
most that many bytes for each test. The default limit is 1000000
bytes. To remove the default limit, specify 'unlimited'.
-s SUBSTRING, --filter-substring=SUBSTRING
Select tests whose description contains SUBSTRING. Multiple '-s'
search queries can be specified in which case only one of them
needs to match.
--slice=NUM/NUM_SLICES
Divide the test suite into NUM_SLICES and work on slice NUM only
(1-based). For example, '1/4' is the first of four slices and '4/4'
is the last one. If multiple '--slice' options are specified, they
are applied sequentially from left to right. If the original suite
defines 23 tests, '--slice 2/4' selects tests [7,8,9,10,11,12]
(1-based). If additionally '--slice 1/3' is specified after it on
the same command line, the remaining tests will be [7,8]. This
filter is meant for running tests in parallel, possibly across
multiple hosts. It is applied after any other filters to as to
divide the work more evenly.
--strict
Treat broken tests as ordinary tests. This disables the default
behavior of ignoring failing tests that were marked as broken by
the programmer when determining the overall success of the test
run.
-t QUERY, --filter-tag=QUERY
Select tests whose tags match QUERY. Filtering by tag is generally
more robust than selecting tests by text contained in their name
with '-s'. QUERY is a boolean query combining tags with 'and',
'or', 'not', and parentheses using the usual precedence rules.
Tag-like selectors 'all' and 'none' are also supported. For
example, '(foo or bar) and not e2e' will select any test with the
tag 'foo' or the tag 'bar' but not if it has the tag 'e2e'. Run
'./test show-tags' to see the list of tags defined for the current
test suite.
--test-list-checksum=STR
Internal use only. This is a checksum used to check that the list
of tests generated in a worker is the same as the list of tests
generated by the master.
-v, --verbose
Print more details than by default. Currently, this is equivalent
to '--show-output' but it may be extended in the future to bundle
up more options.
-w, --show-output
Show the output of all tests rather than only showing the output of
the failed tests. This excludes the output (stdout, stderr, or
both) that may be captured explicitly to be compared against
expectations.
--worker
Internal option used to launch a parallel worker.
COMMON OPTIONS
--help[=FMT] (default=auto)
Show this help in format FMT. The value FMT must be one of auto,
pager, groff or plain. With auto, the format is pager or plain
whenever the TERM env var is dumb or undefined.
EXIT STATUS
test run exits with:
0 on success.
123 on indiscriminate errors reported on standard error.
124 on command line parsing errors.
125 on unexpected internal errors (bugs).
SEE ALSO
test(1)
Test TEST-RUN(1)