std.test.Tests
class pub Tests
A collection of tests to run.
Fields
concurrency
let pub @concurrency: Int
The number of tests to run concurrently.
This defaults to the number of CPU cores.
reporter
let pub @reporter: Reporter
The reporter to use for producing test output.
This defaults to the Plain
reporter that writes to STDOUT. The use of
colors is enabled if STDOUT is connected to a terminal, unless the
NO_COLOR
environment variable is set to a non-empty value.
filter
let pub @filter: Filter
The filter to apply to decide which tests to run.
seed
let pub @seed: Option[Int]
The seed to use for ordering the tests.
Tests are sorted in random order before running them, in an attempt to prevent them from depending on a specific execution order. When debugging test failures it may be useful to set the seed to a fixed value, ensuring tests are sorted in the same order.
While this value affects the order in which tests are sorted and scheduled,
tests may finish in a different order. For example, given a seed S and tests
[A, B, C]
, the tests might be ordered as [C, B, A]
but finish in the
order [B, C, A]
, due to tests being run concurrently. For a truly
deterministic execution order you'll also need to set the concurrency
field to 1
.
Static methods
new
Show source codeHide source code
fn pub static new -> Tests {
let out = Stdout.new
let colors = match env.opt('NO_COLOR') {
case Some(v) if v.size > 0 -> false
case _ -> out.terminal?
}
Tests(
tests: [],
children: [],
concurrency: cpu_cores,
reporter: Plain.new(out, colors) as Reporter,
filter: Filter.None,
seed: Option.None,
)
}
fn pub static new -> Tests
Returns a new test tests with its default settings.
Instance methods
fork
Show source codeHide source code
fn pub mut fork(name: String, child: fn, test: uni fn (mut Test, Process)) {
let id = @children.size
@children.push(child)
test(name, fn move (t) { test.call(t, Process.new(id)) })
}
fn pub mut fork(name: String, child: fn, test: uni fn (mut Test, Process))
Registers a new test using a fork/subprocess.
This doesn't use the actual fork()
system call. Instead, a new instance of
the test executable is spawned such that it only runs the closure specified
in the child
argument.
no_panic
Show source codeHide source code
fn pub mut no_panic(name: String, code: uni fn) {
fork(name, code, fn (test, process) {
let output = process.spawn
let code = output.status.to_int
if code != 101 { return }
let got = output.stderr.split('\n').last.or('the process panicked')
test.failures.push(Failure.new(got, 'the process not to panic'))
})
}
fn pub mut no_panic(name: String, code: uni fn)
Registers a new test that asserts the given closure doesn't panic.
ok
Show source codeHide source code
fn pub mut ok[T, E: Format](
name: String,
code: uni fn (mut Test) -> Result[T, E],
) {
test(name, fn move (t) {
match code.call(t) {
case Error(e) -> {
t.failures.push(
Failure(
got: 'Result.Error(${fmt(e)})',
expected: 'a value matching the pattern Result.Ok(_)',
path: t.path.clone,
line: t.line,
),
)
}
case _ -> {}
}
})
}
fn pub mut ok[T, E: Format](name: String, code: uni fn (mut Test) -> Result[T, E])
Registers a new test that returns a Result
.
If the test closure returns an Error
, an additional test failure is added.
This is useful when writing tests that perform operations that might fail
but in general shouldn't, such as connecting to a socket that's expected to
be available. Such tests could sporadically fail, resulting in any
getping terminating the entire test suite. Using this method, you can
instead use the try
operator. So instead of this:
t.test('Foo', fn (t) {
let a = foo.get
let b = bar(a).get
...
})
You'd write this:
t.ok('Foo', fn (t) {
let a = try foo
let b = try bar(a)
...
})
Using this method comes with the following trade-offs:
- All error values must be of the same type
2. The test failure location points to the location at which the test is
defined, not the location at which the error is produced
3. If the last expression doesn't return a Result
, you'll need to use
Result.Ok(nil)
as the last expression
panic
Show source codeHide source code
fn pub mut panic(name: String, code: uni fn) {
fork(name, code, fn (test, process) {
let output = process.spawn
let code = output.status.to_int
if code == 101 { return }
# These tests run in a separate OS process, and `debug.stacktrace` won't
# include any frames pointing to the unit tests's source file. To work
# around that, we reuse the test's location.
let failure = Failure(
got: 'the process exited with status ${code}',
expected: 'the process to panic with exit status 101',
path: test.path.clone,
line: test.line,
)
test.failures.push(failure)
})
}
fn pub mut panic(name: String, code: uni fn)
Registers a new test that asserts the given closure triggers a panic.
run
Show source codeHide source code
fn pub move run {
match env.opt(CHILD_VAR) {
case Some(id) -> return run_child(id)
case _ -> {}
}
let seed = match @seed {
case Some(seed) -> seed
case _ -> Random.new.int
}
# We shuffle tests in a random order to ensure they don't end up
# (implicitly) depending on a specific execution order. We do this first so
# we can build a unique list of filtered tests (which retain the order).
Shuffle.from_int(seed).sort(@tests)
let filter = @filter
let tests = @tests
.into_iter
.select_map(fn (test) {
if test.matches?(filter) { Option.Some(test) } else { Option.None }
})
.reduce(recover [], fn (tests, test) {
tests.push(test)
tests
})
let size = tests.size
let rep = @reporter
let start = Instant.new
let jobs = Jobs(tests)
let output = Channel.new
@concurrency.times(fn (_) { Runner(jobs, recover output.clone).schedule })
size.times(fn (_) {
let test = recover output.receive
if test.failures.empty? { rep.passed(test) } else { rep.failed(test) }
})
if rep.finished(start.elapsed, seed) {
exit(status: 0)
} else {
exit(status: 1)
}
}
fn pub move run
Runs all the tests.
run_child
Show source codeHide source code
fn pub move run_child(id: String) {
match
Int.parse(id, IntFormat.Decimal).then(fn (v) { @children.opt_mut(v) })
{
case Some(block) -> block.call
case _ -> process.panic("The child ID '${id}' is invalid")
}
}
fn pub move run_child(id: String)
test
Show source codeHide source code
fn pub mut test(name: String, code: uni fn (mut Test)) {
let id = @tests.size
let test = recover {
match unit_test_stack_frame {
case { @path = path, @line = line } -> {
Test.new(id, name, path, line, code)
}
}
}
@tests.push(test)
}
fn pub mut test(name: String, code: uni fn (mut Test))
Registers a new test with the test tests.