unittest2

Search:
Group by:
Source   Edit  

Authors:Zahary Karadjov, Ștefan Talpalaru, Status Research and Development

Types

ConsoleOutputFormatter = ref object of OutputFormatter
  when collect:
A formatter that prints test results to the console with optional color and various verbosity levels. Source   Edit  
JUnitOutputFormatter = ref object of OutputFormatter
A formatter that writes test results in JUnit-compatible XML format. Source   Edit  
OutputFormatter = ref object of RootObj
Base class for all test output formatters. Custom formatters should inherit from this type and override its methods. Source   Edit  
OutputLevel = enum
  VERBOSE,                  ## Print as much as possible.
  COMPACT,                  ## Print failures and compact success information
  FAILURES,                 ## Print only failures
  NONE                       ## Print nothing.
The output verbosity of the tests. Source   Edit  
TestResult = object
  suiteName*: string         ## Name of the test suite that contains this test case.
  testName*: string          ## Name of the test case.
  status*: TestStatus        ## Final status of the test (OK, FAILED, or SKIPPED).
  duration*: Duration        ## How long the test took to execute.
  output*: string            ## Combined stdout/stderr output captured during the test (if supported by the execution mode).
  errors*: string            ## Failure diagnostics and checkpoints collected during the test.
Represents the result of a single test case execution. Source   Edit  
TestStatus = enum
  OK, FAILED, SKIPPED
The status of a test when it is done. Source   Edit  

Vars

abortOnError {.threadvar.}: bool
Set to true in order to quit immediately on fail. Default is false, or override with -d:nimUnittestAbortOnError:on|off. Source   Edit  

Consts

unittest2ListTests {.booldefine.} = false
List tests at runtime without actually running them (useful for test runners). Set with -d:unittest2ListTests=true. Source   Edit  
unittest2Static {.booldefine.} = false
Run tests at compile time as well - only a subset of functionality is enabled at compile-time meaning that tests must be written conservatively. suite features (setup etc) in particular are not supported. Set with -d:unittest2Static=true. Source   Edit  

Procs

proc addOutputFormatter(formatter: OutputFormatter) {....raises: [], tags: [],
    forbids: [].}
Add an output formatter used for all subsequent reporting events. Source   Edit  
proc defaultConsoleFormatter(): ConsoleOutputFormatter {....raises: [],
    tags: [ReadEnvEffect], forbids: [].}
Create the default console formatter honoring current runtime settings. Source   Edit  
proc disableParamFiltering() {....deprecated: "Compile with -d:unittest2DisableParamFiltering instead",
                               raises: [], tags: [], forbids: [].}
Deprecated: Compile with -d:unittest2DisableParamFiltering instead
Deprecated runtime switch kept for source compatibility. Source   Edit  
proc newConsoleOutputFormatter(outputLevel: OutputLevel = outputLevelDefault;
                               colorOutput = true): ConsoleOutputFormatter {.
    ...raises: [], tags: [], forbids: [].}
Create a console formatter with explicit verbosity and color settings. Source   Edit  
proc newJUnitOutputFormatter(stream: Stream): JUnitOutputFormatter {....raises: [],
    tags: [WriteIOEffect], forbids: [].}
Creates a formatter that writes report to the specified stream in JUnit format. The stream is NOT closed automatically when the test are finished, because the formatter has no way to know when all tests are finished. You should invoke formatter.close() to finalize the report. Source   Edit  
proc parseParameters(args: openArray[string]) {....raises: [],
    tags: [ReadEnvEffect, WriteIOEffect], forbids: [].}

Parse unittest2 CLI options and test filters from args.

Recognized options include --help, --xml, --console, --output-level, and --verbose/-v.

Source   Edit  
proc resetOutputFormatters() {....raises: [], tags: [], forbids: [].}
Remove all currently registered output formatters. Source   Edit  

Methods

method failureOccurred(formatter: ConsoleOutputFormatter;
                       checkpoints: seq[string]; stackTrace: string) {.
    ...raises: [], tags: [], forbids: [].}
Source   Edit  
method failureOccurred(formatter: JUnitOutputFormatter;
                       checkpoints: seq[string]; stackTrace: string) {.
    ...raises: [], tags: [], forbids: [].}
stackTrace is provided only if the failure occurred due to an exception. checkpoints is never nil. Source   Edit  
method failureOccurred(formatter: OutputFormatter; checkpoints: seq[string];
                       stackTrace: string) {.base, ...gcsafe, raises: [], tags: [],
    forbids: [].}
Called when a check or expectation fails, or an unhandled exception occurs. stackTrace is provided only if the failure occurred due to an exception. checkpoints contains the list of checkpoints encountered before the failure. Source   Edit  
method suiteEnded(formatter: ConsoleOutputFormatter) {....raises: [],
    tags: [WriteIOEffect, ReadIOEffect], forbids: [].}
Source   Edit  
method suiteEnded(formatter: JUnitOutputFormatter) {....raises: [], tags: [],
    forbids: [].}
Source   Edit  
method suiteEnded(formatter: OutputFormatter) {.base, ...gcsafe, raises: [],
    tags: [], forbids: [].}
Called after all tests in the current suite have finished. Source   Edit  
method suiteStarted(formatter: ConsoleOutputFormatter; suiteName: string) {.
    ...raises: [], tags: [WriteIOEffect], forbids: [].}
Source   Edit  
method suiteStarted(formatter: JUnitOutputFormatter; suiteName: string) {.
    ...raises: [], tags: [], forbids: [].}
Source   Edit  
method suiteStarted(formatter: OutputFormatter; suiteName: string) {.base,
    ...gcsafe, raises: [], tags: [], forbids: [].}
Called before tests from suiteName start running. Source   Edit  
method testEnded(formatter: ConsoleOutputFormatter; testResult: TestResult) {.
    ...raises: [], tags: [ReadIOEffect, WriteIOEffect], forbids: [].}
Source   Edit  
method testEnded(formatter: JUnitOutputFormatter; testResult: TestResult) {.
    ...raises: [], tags: [], forbids: [].}
Source   Edit  
method testEnded(formatter: OutputFormatter; testResult: TestResult) {.base,
    ...gcsafe, raises: [], tags: [], forbids: [].}
Called when a test has finished and the final testResult is known. Source   Edit  
method testRunEnded(formatter: ConsoleOutputFormatter) {....raises: [],
    tags: [WriteIOEffect], forbids: [].}
Source   Edit  
method testRunEnded(formatter: JUnitOutputFormatter) {....raises: [],
    tags: [WriteIOEffect], forbids: [].}
Completes the report and closes the underlying stream. Source   Edit  
method testRunEnded(formatter: OutputFormatter) {.base, ...gcsafe, raises: [],
    tags: [], forbids: [].}
Called once when the full test run is about to terminate (usually via an exit proc). Source   Edit  
method testStarted(formatter: ConsoleOutputFormatter; testName: string) {.
    ...raises: [], tags: [WriteIOEffect], forbids: [].}
Source   Edit  
method testStarted(formatter: JUnitOutputFormatter; testName: string) {.
    ...raises: [], tags: [], forbids: [].}
Source   Edit  
method testStarted(formatter: OutputFormatter; testName: string) {.base, ...gcsafe,
    raises: [], tags: [], forbids: [].}
Called before testName starts running. Source   Edit  

Macros

macro check(conditions: untyped): untyped
Verify if a statement or a list of statements is true. A helpful error message and set checkpoints are printed out on failure (if outputLevel is not NONE).

Example:

import std/strutils

check("AKB48".toLowerAscii() == "akb48")

let teams = {'A', 'K', 'B', '4', '8'}

check:
  "AKB48".toLowerAscii() == "akb48"
  'C' notin teams
Source   Edit  
macro expect(exceptions: varargs[typed]; body: untyped): untyped
Test if body raises an exception found in the passed exceptions. The test passes if the raised exception is part of the acceptable exceptions. Otherwise, it fails.

Example:

import std/[math, random, strutils]
proc defectiveRobot() =
  randomize()
  case rand(1..4)
  of 1: raise newException(OSError, "CANNOT COMPUTE!")
  of 2: discard parseInt("Hello World!")
  of 3: raise newException(IOError, "I can't do that Dave.")
  else: assert 2 + 2 == 5

expect IOError, OSError, ValueError, AssertionDefect:
  defectiveRobot()
Source   Edit  

Templates

template checkpoint(msg: string) {..}
Set a checkpoint identified by msg. Upon test failure all checkpoints encountered so far are printed out. Example:
checkpoint("Checkpoint A")
check((42, "the Answer to life and everything") == (1, "a"))
checkpoint("Checkpoint B")

outputs "Checkpoint A" once it fails.

Source   Edit  
template dualTest(nameParam: string; body: untyped) {..}
Define a test case that runs both at compile-time and runtime. This ensures that your code behaves consistently in both environments. Source   Edit  
template fail() {..}
Print out the checkpoints encountered so far and quit if abortOnError is true. Otherwise, erase the checkpoints and indicate the test has failed (change exit code and test status). This template is useful for debugging, but is otherwise mostly used internally. Example:
checkpoint("Checkpoint A")
complicatedProcInThread()
fail()

outputs "Checkpoint A" before quitting.

Source   Edit  
template require(conditions: untyped)
Same as check except any failed test causes the program to quit immediately. Any teardown statements are not executed and the failed test output is not generated. Source   Edit  
template runtimeTest(nameParam: string; body: untyped) {..}
Define a test case that runs only at runtime. This is useful for tests that require OS features, I/O, or other functionality not available in the Nim VM. Source   Edit  
template skip() {..}
Mark the test as skipped. Should be used directly in case when it is not possible to perform test for reasons depending on outer environment, or certain application logic conditions or configurations. The test code is still executed.
if not isGLContextCreated():
  skip()
Source   Edit  
template staticTest(nameParam: string; body: untyped) {..}
Define a test case that runs only at compile-time in the Nim VM. This is useful for verifying logic, constant folding, and type-level transformations during compilation. Source   Edit  
template suite(nameParam: string; body: untyped) {.dirty.}

Declare a test suite identified by name with optional setup and/or teardown section.

A test suite is a series of one or more related tests sharing a common fixture (setup, teardown). The fixture is executed for EACH test.

suite "test suite for addition":
  setup:
    let result = 4
  
  test "2 + 2 = 4":
    check(2+2 == result)
  
  test "(2 + -2) != 4":
    check(2 + -2 != result)
  
  # No teardown needed

The suite will run the individual test cases in the order in which they were listed. With default global settings the above code prints:

[Suite] test suite for addition
  [OK] 2 + 2 = 4
  [OK] (2 + -2) != 4
Source   Edit  
template test(nameParam: string; body: untyped) {..}
Define a single test case identified by name.
test "roses are red":
  let roses = "red"
  check(roses == "red")

The above code outputs:

[OK] roses are red
Source   Edit