Search results

There are no results.

std.test.Tests

class pub Tests

A collection of tests to run.

Fields

concurrency

let pub @concurrency: Int

The number of tests to run concurrently.

This defaults to the number of CPU cores.

reporter

let pub @reporter: Reporter

The reporter to use for producing test output.

This defaults to the Plain reporter that writes to STDOUT. The use of colors is enabled if STDOUT is connected to a terminal, unless the NO_COLOR environment variable is set to a non-empty value.

filter

let pub @filter: Filter

The filter to apply to decide which tests to run.

seed

let pub @seed: Option[Int]

The seed to use for ordering the tests.

Tests are sorted in random order before running them, in an attempt to prevent them from depending on a specific execution order. When debugging test failures it may be useful to set the seed to a fixed value, ensuring tests are sorted in the same order.

While this value affects the order in which tests are sorted and scheduled, tests may finish in a different order. For example, given a seed S and tests [A, B, C], the tests might be ordered as [C, B, A] but finish in the order [B, C, A], due to tests being run concurrently. For a truly deterministic execution order you'll also need to set the concurrency field to 1.

Static methods

new

Show source code
Hide source code
fn pub static new -> Tests {
  let out = Stdout.new
  let colors = match env.opt('NO_COLOR') {
    case Some(v) if v.size > 0 -> false
    case _ -> out.terminal?
  }

  Tests(
    tests: [],
    children: [],
    concurrency: cpu_cores,
    reporter: Plain.new(out, colors) as Reporter,
    filter: Filter.None,
    seed: Option.None,
  )
}
fn pub static new -> Tests

Returns a new test tests with its default settings.

Instance methods

fork

Show source code
Hide source code
fn pub mut fork(name: String, child: fn, test: uni fn (mut Test, Process)) {
  let id = @children.size

  @children.push(child)
  test(name, fn move (t) { test.call(t, Process.new(id)) })
}
fn pub mut fork(name: String, child: fn, test: uni fn (mut Test, Process))

Registers a new test using a fork/subprocess.

This doesn't use the actual fork() system call. Instead, a new instance of the test executable is spawned such that it only runs the closure specified in the child argument.

no_panic

Show source code
Hide source code
fn pub mut no_panic(name: String, code: uni fn) {
  fork(name, code, fn (test, process) {
    let output = process.spawn
    let code = output.status.to_int

    if code != 101 { return }

    let got = output.stderr.split('\n').last.or('the process panicked')

    test.failures.push(Failure.new(got, 'the process not to panic'))
  })
}
fn pub mut no_panic(name: String, code: uni fn)

Registers a new test that asserts the given closure doesn't panic.

ok

Show source code
Hide source code
fn pub mut ok[T, E: Format](
  name: String,
  code: uni fn (mut Test) -> Result[T, E],
) {
  test(name, fn move (t) {
    match code.call(t) {
      case Error(e) -> {
        t.failures.push(
          Failure(
            got: 'Result.Error(${fmt(e)})',
            expected: 'a value matching the pattern Result.Ok(_)',
            path: t.path.clone,
            line: t.line,
          ),
        )
      }
      case _ -> {}
    }
  })
}
fn pub mut ok[T, E: Format](name: String, code: uni fn (mut Test) -> Result[T, E])

Registers a new test that returns a Result.

If the test closure returns an Error, an additional test failure is added.

This is useful when writing tests that perform operations that might fail but in general shouldn't, such as connecting to a socket that's expected to be available. Such tests could sporadically fail, resulting in any getping terminating the entire test suite. Using this method, you can instead use the try operator. So instead of this:

t.test('Foo', fn (t) {
  let a = foo.get
  let b = bar(a).get
  ...
})

You'd write this:

t.ok('Foo', fn (t) {
  let a = try foo
  let b = try bar(a)
  ...
})

Using this method comes with the following trade-offs:

  1. All error values must be of the same type

2. The test failure location points to the location at which the test is defined, not the location at which the error is produced 3. If the last expression doesn't return a Result, you'll need to use Result.Ok(nil) as the last expression

panic

Show source code
Hide source code
fn pub mut panic(name: String, code: uni fn) {
  fork(name, code, fn (test, process) {
    let output = process.spawn
    let code = output.status.to_int

    if code == 101 { return }

    # These tests run in a separate OS process, and `debug.stacktrace` won't
    # include any frames pointing to the unit tests's source file. To work
    # around that, we reuse the test's location.
    let failure = Failure(
      got: 'the process exited with status ${code}',
      expected: 'the process to panic with exit status 101',
      path: test.path.clone,
      line: test.line,
    )

    test.failures.push(failure)
  })
}
fn pub mut panic(name: String, code: uni fn)

Registers a new test that asserts the given closure triggers a panic.

run

Show source code
Hide source code
fn pub move run {
  match env.opt(CHILD_VAR) {
    case Some(id) -> return run_child(id)
    case _ -> {}
  }

  let seed = match @seed {
    case Some(seed) -> seed
    case _ -> Random.new.int
  }

  # We shuffle tests in a random order to ensure they don't end up
  # (implicitly) depending on a specific execution order. We do this first so
  # we can build a unique list of filtered tests (which retain the order).
  Shuffle.from_int(seed).sort(@tests)

  let filter = @filter
  let tests = @tests
    .into_iter
    .select_map(fn (test) {
      if test.matches?(filter) { Option.Some(test) } else { Option.None }
    })
    .reduce(recover [], fn (tests, test) {
      tests.push(test)
      tests
    })

  let size = tests.size
  let rep = @reporter
  let start = Instant.new
  let jobs = Jobs(tests)
  let output = Channel.new

  @concurrency.times(fn (_) { Runner(jobs, recover output.clone).schedule })

  size.times(fn (_) {
    let test = recover output.receive

    if test.failures.empty? { rep.passed(test) } else { rep.failed(test) }
  })

  if rep.finished(start.elapsed, seed) {
    exit(status: 0)
  } else {
    exit(status: 1)
  }
}
fn pub move run

Runs all the tests.

run_child

Show source code
Hide source code
fn pub move run_child(id: String) {
  match
    Int.parse(id, IntFormat.Decimal).then(fn (v) { @children.opt_mut(v) })
  {
    case Some(block) -> block.call
    case _ -> process.panic("The child ID '${id}' is invalid")
  }
}
fn pub move run_child(id: String)

test

Show source code
Hide source code
fn pub mut test(name: String, code: uni fn (mut Test)) {
  let id = @tests.size
  let test = recover {
    match unit_test_stack_frame {
      case { @path = path, @line = line } -> {
        Test.new(id, name, path, line, code)
      }
    }
  }

  @tests.push(test)
}
fn pub mut test(name: String, code: uni fn (mut Test))

Registers a new test with the test tests.