# Test scenarios

This directory contains both the scenarios used by C++ unit tests and those which are
WML unit tests.

## C++ unit tests

For the C++ unit tests, it is recommended to reuse the same scenario file as much as possible
and just inject WML into it.

Injection can be done by adding a config object containing event code and then registering that
manually for `game_events`.

## Manual tests

The `manual_tests` subdirectory contains scenarios that expect to be run interactively, either by
binding a hotkey for the main menu's "Choose Test Scenario" option, or with the command line
argument `-t <testname>`.

Many of these are closer to workbenches than tests, allowing developers to do some action that isn't
automated, and then to find out whether the result matched the expectation.

## Automated WML unit tests

WML unit tests are self-contained scenario files to test a specific area of WML.

The test result is a status code from the `unit_test_result` enum found in `game_launcher.hpp`, or
in rare cases tests expect to be timed out by the test runner. They can be run individually with
Wesnoth's `-u <testname>` command line argument, but are usually run by the `run_wml_tests script`
based on the list in `wml_test_schedule`.

They are unlikely to return the same status if run with `-t <testname>`. Running them with `-t` can
still be helpful for debugging.

The tests aren't meant to prevent intentional changes to behavior; they're for checking that
bugfixes or unrelated changes don't accidentally change the behavior. They make intentional changes
easier to notice and review, and you should pay attention to any comments in tests that you change,
but the mere existence of a test shouldn't be taken as saying that the current behavior is good.

### Guidelines for writing automated new tests

Tests are generally implemented with the `GENERIC_UNIT_TEST` macro, with two leaders called Alice
and Bob on separate keeps. If your test needs them to be adjacent to each other, consider using
`COMMON_KEEP_A_B_UNIT_TEST` instead, which puts their starting locations next to each other instead
of needing to move them.

Most will expect the result `PASS`, and new tests should generally be written to result in a `PASS`.
The testing mechanism supports other expectations too, however the optimisation to run a batch of
tests in a single instance of Wesnoth currently only supports batching for tests that return `PASS`.

Tests that shouldn't `PASS` should have a name that makes that expectation obvious. However, the
existing tests don't conform to this goal yet.

Names containing `_fail_` or ending `_fail` are reserved for tests that should not `PASS`. However,
they may expect a status that is neither `PASS` nor `FAIL`.

Names containing `_bug_` or ending `_bug` are reserved for tests where consensus is that the
behavior should change in the development branch, however that it's better to accept the current
behavior in the stable branch than to risk Out Of Sync errors.

The words `break` and `error` aren't reserved, and can be used even for tests expected to `PASS`.
Some of these are testing loops, or testing error-handling that is expected to handle the error.