Skip to content

Tests Functions

🛠️ Tools · ⬅ Home

Test Subsystem

testTools

Load test tools

Usage

testTools [ --help ] [ binary ] [ ... ]

Load test tools

Arguments

  • --help - Flag. Optional. Display this help.
  • binary - Callable. Optional. Run this program after loading test tools.
  • ... - Optional. Arguments. Arguments for binary.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Tag filters

Prefix a tag with + for --tag or --skip-tag queries to add the meaning "previous AND". - --tag foo --tag bar means tests must have foo tag OR must have bar tag - --tag foo --tag +bar means tests must have foo tag AND must have bar tag (must have both) - --skip-tag foo --skip-tag bar means skip any test with foo tag OR with any test with bar tag (either) - --skip-tag foo --skip-tag +bar means skip any test with foo tag AND with the bar tag (must have both) - --tag a --tag +b --tag c --tag +d --tag +e is (a and b) or (c and d and e) Your test functions can contain tags as follows: # Tag: tagA tagB # Tag: tagC # Test-Platform: !alpine alpine !linux linux !darwin darwin # Test-Skip: true # Test-Housekeeper: false # Test-Plumber: true # Test-TAP-Directive: Something # Test-After: testWhichShouldGoBefore # Test-Before: testWhichShouldGoAfter # Test-Fail: true testThingy() { ... } Environment variables: - BUILD_TEST_FLAGS - SemicolonDelimitedList. Add flags like 'Plumber:false' to disable settings across tests. - BUILD_DEBUG - Many settings to debug different systems, comma-delimited. Filters (--tag and --skip-tag) are applied in order after the function pattern or suite filter.

Arguments

  • --help - Flag. Optional. Display this help.
  • --clean - Flag. Optional. Delete test artifact files and exit. (No tests run) -l |- --list - Flag. Optional. List all test names (which match if applicable).
  • --env-file environmentFile - EnvironmentFile. Optional. Load one ore more environment files prior to running tests
  • --index-file indexFile - RealDirectoryFile. Optional. If supplied and exists, uses the index file for tests.
  • --cache-path cachePath - Directory. Optional.
  • --make-index - Flag. Optional. Generate the index file if supplied and quit.
  • --continue - Flag. Optional. Continue from last successful test.
  • -c - Flag. Optional. Continue from last successful test.
  • --delete directoryOrFile - FileDirectory. Optional. A file or directory to delete when the test suite terminates.
  • --delete-common - Flag. Delete ./vendor and ./node_modules (and other temporary build directories) by default.
  • --debug - Flag. Optional. Enable debugging for --junit (saves caches).
  • --verbose - Flag. Optional. Be verbose.
  • --stop - Flag. Optional. Stop after a failure instead of attempting to continue.
  • --coverage - Flag. Optional. Feature in progress - generate a coverage file for tests.
  • --no-stats - Flag. Optional. Do not generate a test.stats file showing test timings when completed.
  • --messy - Flag. Optional. Do not delete test artifact files afterwards.
  • --fail executor - Callable. Optional. One or more programs to run on the failed test files. Takes arguments: testName testFile testLine
  • --cd-away - Flag. Optional. Change directories to a temporary directory before each test.
  • --tap tapFile - FileDirectory. Optional. Output test results in TAP format to tapFile.
  • --junit junitFile - FileDirectory. Optional. Output test results in junit format to junitFile. If a directory is specified the output is to junit.xml. --show |- --suites - Flag. Optional. List all test suites. -1 | --suite |- --one testSuite - String. Optional. Add one test suite to run.
  • --tag tagName - String. Optional. Include tests (only) tagged with this name. --list-tags | --tags |- --show-tags - Flag. Optional. Of the matched tests, display the tags that they have, if any. Unique list.
  • --skip-tag tagName - String. Optional. Skip tests tagged with this name.
  • testFunctionPattern ... - String. Optional. Test function (or substring of function name) to run.

Debugging settings

Append to the value of BUILD_DEBUG (a comma-delimited (,) list) and add these tokens to enable debugging:

  • test-dump-environment - When set tests will dump the environment at the end.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Environment

  • BUILD_TEST_FLAGS - String. Test flags affect controls and how tests are run. BUILD_DEBUG - CommaDelimitedList. Constant for turning debugging on during build to find errors in the build scripts. Enable debugging globally in the build scripts. Set to a comma (,) delimited list string to enable specific debugging, or true for ALL debugging, false (or blank) for NO debugging.

Requires

head tee printf trap decorate loadAverage consoleConfigureColorMode buildEnvironmentLoad usageArgumentString catchEnvironment bashCoverage

testSuiteOrdering

undocumented

Usage

testSuiteOrdering [ --cache cacheDirectory ] finderFile

No documentation for testSuiteOrdering.

Writes to standard output

Reordered file.

Arguments

  • --cache cacheDirectory - Directory. Optional. Cache directory to use for ordering work.
  • finderFile - File. Required. File to reorder.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

testSuiteFunctionTested

When environment variable TEST_TRACK_ASSERTIONS is true – testSuite and assertion

Usage

testSuiteFunctionTested [ --help ] [ --help ] [ --verbose ] [ functionName ... ]

When environment variable TEST_TRACK_ASSERTIONS is true – testSuite and assertion functions track which functions take a function value (for example, assertExitCode) and track functions which are run, and stores them in the testing cache directory which accumulate after each test run unless the cache is cleared.

Arguments

  • --help - Flag. Optional. Display this help.
  • --help - Flag. Optional. Display this help.
  • --verbose - Flag. Optional. Show list of true results when all arguments pass.
  • functionName ... - String. Function to look up to see if it has been tested. One or more.

Return codes

  • 0 - All functions were tested by the test suite at least once.
  • 1 - At least one function was not tested by the test suite at least once.

Environment

  • TEST_TRACK_ASSERTIONS - Boolean. Turn on or off tracking of function assertions within the testing core If blank, the default behavior is to track; disable it with setting the value to false

assertStatistics

Output assertion counts

Usage

assertStatistics [ --reset ] [ --total ] [ --help ]

Output the total number of assertion failures and assertion successes, separated by a space and terminated with a newline

Writes to standard output

UnsignedInteger. 2 lines.

Arguments

  • --reset - Flag. Optional. Reset statistics to zero.
  • --total - Flag. Optional. Just output the total.
  • --help - Flag. Optional. Display this help.

Examples

read -r failures successes < <(assertStatistics) || return $?

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Code quality tools

bashFindUncaughtAssertions

Search bash files for assertions which do not terminate a

Usage

bashFindUncaughtAssertions [ --help ] [ --handler handler ] [ --exclude path ] [ --exec binary ] [ directory ] [ --list ]

Search bash files for assertions which do not terminate a function and are likely an error

Arguments

  • --help - Flag. Optional. Display this help.
  • --handler handler - Function. Optional. Use this error handler instead of the default error handler.
  • --exclude path - String. Optional. Exclude paths which contain this string
  • --exec binary - Executable. Optional. For each failed file run this command.
  • directory - Directory. Optional. Where to search for files to check.
  • --list - Flag. Optional. List files which fail. (Default is simply to exit silently.)

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Usage

evalCheck

Check files to ensure evals in code have been checked

Arguments

  • none

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error