Environment and CLI¶
Tests built with Criterion expose by default various command line switchs and environment variables to alter their runtime behaviour.
Command line arguments¶
-h or --help
: Show a help message with the available switches.-q or --quiet
: Disables all logging.-v or --version
: Prints the version of criterion that has been linked against.-l or --list
: Print all the tests in a list.-f or --fail-fast
: Exit after the first test failure.--ascii
: Don’t use fancy unicode symbols or colors in the output.-jN or --jobs N
: UseN
parallel jobs to run the tests.0
picks a number of jobs ideal for your hardware configuration.--pattern [PATTERN]
: Run tests whose string identifier matches the given shell wildcard pattern (see dedicated section below). (*nix only)--no-early-exit
: The test workers shall not prematurely exit when done and will properly return from the main, cleaning up their process space. This is useful when tracking memory leaks withvalgrind --tool=memcheck
.-S or --short-filename
: The filenames are displayed in their short form.--always-succeed
: The process shall exit with a status of0
.--tap[=FILE]
: Writes a TAP (Test Anything Protocol) report to FILE. No file or"-"
meansstderr
and implies--quiet
. This option is equivalent to--output=tap:FILE
.--xml[=FILE]
: Writes JUnit4 XML report to FILE. No file or"-"
meansstderr
and implies--quiet
. This option is equivalent to--output=tap:FILE
.--json[=FILE]
: Writes a JSON report to FILE. No file or"-"
meansstderr
and implies--quiet
. This option is equivalent to--output=tap:FILE
.--verbose[=level]
: Makes the output verbose. When provided with an integer, sets the verbosity level to that integer.-OPROVIDER:FILE or --output=PROVIDER:FILE
: Write a test report to FILE using the output provider named by PROVIDER. If FILE is"-"
, it implies--quiet
, and the report shall be written tostderr
.
Shell Wildcard Pattern¶
Extglob patterns in criterion are matched against a test’s string identifier.
This feature is only available on *nix systems where PCRE
is provided.
In the table below, a pattern-list
is a list of patterns separated by |
.
Any extglob pattern can be constructed by combining any of the following
sub-patterns:
Pattern | Meaning |
---|---|
* |
matches everything |
? |
matches any character |
[seq] |
matches any character in seq |
[!seq] |
matches any character not in seq |
?(pattern-list) |
Matches zero or one occurrence of the given patterns |
*(pattern-list) |
Matches zero or more occurrences of the given patterns |
+(pattern-list) |
Matches one or more occurrences of the given patterns |
@(pattern-list) |
Matches one of the given patterns |
!(pattern-list) |
Matches anything except one of the given patterns |
A test string identifier is of the form suite-name/test-name
, so a pattern
of simple/*
matches every tests in the simple
suite, */passing
matches all tests named passing
regardless of the suite, and *
matches
every possible test.
Environment Variables¶
Environment variables are alternatives to command line switches when set to 1.
CRITERION_ALWAYS_SUCCEED
: Same as--always-succeed
.CRITERION_NO_EARLY_EXIT
: Same as--no-early-exit
.CRITERION_FAIL_FAST
: Same as--fail-fast
.CRITERION_USE_ASCII
: Same as--ascii
.CRITERION_JOBS
: Same as--jobs
. Sets the number of jobs to its value.CRITERION_SHORT_FILENAME
: Same as--short-filename
.CRITERION_VERBOSITY_LEVEL
: Same as--verbose
. Sets the verbosity level to its value.CRITERION_TEST_PATTERN
: Same as--pattern
. Sets the test pattern to its value. (*nix only)CRITERION_DISABLE_TIME_MEASUREMENTS
: Disables any time measurements on the tests.CRITERION_OUTPUTS
: Can be set to a comma-separated list ofPROVIDER:FILE
entries. For instance, setting the variable totap:foo.tap,xml:bar.xml
has the same effect as specifying--tap=foo.tap
and--xml=bar.xml
at once.CRITERION_ENABLE_TAP
: (Deprecated, use CRITERION_OUTPUTS) Same as--tap
.