The test_server module provides some useful functions to support the test suite author. This includes:

Please turn to the reference manual for the test_server module for details about these functions.

A test suite is an ordinary Erlang module that contains test cases. It's recommended that the module has a name on the form *_SUITE.erl. Otherwise, the directory function will not find the modules (by default).

For some of the test server support, the test server include file test_server.hrl must be included. Never include it with the full path, for portability reasons. Use the compiler include directive instead.

The special function all(suite) in each module is called to get the test specification for that module. The function typically returns a list of test cases in that module, but any test specification could be returned. Please see the chapter about test specifications for details about this.

In each test suite module, the functions init_per_testcase/2 and end_per_testcase/2 must be implemented.

init_per_testcase is called before each test case in the test suite, giving a (limited) possibility for initialization.

end_per_testcase/2 is called after each test case is completed, giving a possibility to clean up.

The first argument to these functions is the name of the test case. This can be used to do individual initialization and cleanup for each test cases.

The second argument is a list of tuples called Config. The first element in a Config tuple should be an atom - a key value to be used for searching. init_per_testcase/2 may modify the Config parameter or just return it as is. Whatever is retuned by init_per_testcase/2 is given as Config parameter to the test case itself.

The return value of end_per_testcase/2 is ignored by the test server.

The smallest unit that the test server is concerned with is a test case. Each test case can in turn test many things, for example make several calls to the same interface function with different parameters.

It is possible to put many or few tests into each test case. How many things each test case tests is up to the author, but here are some things to keep in mind.

Very small test cases often leads to more code, since initialization has to be duplicated. Larger code, especially with a lot of duplication, increases maintenance and reduces readability.

Larger test cases make it harder to tell what went wrong if it fails, and force us to skip larger portions of test code if a specific part fails. These effects are accentuated when running on multiple platforms because test cases often have to be skipped.

A test case generally consists of three parts, the documentation part, the specification part and the execution part. These are implemented as three clauses of the same function.

The documentation clause matches the argument 'doc' and returns a list for strings describing what the test case tests.

The specification clause matches the argument 'suite' and returns the test specification for this particular test case. If the test specification is an empty list, this indicates that the test case is a leaf test case, i.e. one to be executed.

The execution clause implements the actual test case. It takes one argument, Config, which contain configuration information like data_dir and priv_dir. See Data and Private Directories for more information about these.

The Config variable can also contain the nodenames key, if requested by the require_nodenames command in the test suite specification file. All Config items should be extracted using the ?config macro. This is to ensure future compatibility if the Config format changes. See the reference manual for test_server for details about this macro.

If the execution clause crashes or exits, it is considered a failure. If it returns {skip,Reason}, the test case is considered skipped. If it returns {comment,String}, the string will be added in the 'Comment' field on the HTML result page. If the execution clause returns anything else, it is considered a success, unless it is {'EXIT',Reason} or {'EXIT',Pid,Reason} which can't be distinguished from a crash, and thus will be considered a failure.

The data directory (data_dir) is the directory where the test module has its own files needed for the testing. A compiler test case may have source files to feed into the compiler, a release upgrade test case may have some old and new release of something. A graphics test case may have some icons and a test case doing a lot of math with bignums might store the correct answers there. The name of the data_dir is the the name of the test suite and then "_data". For example, "some_path/foo_SUITE.beam" has the data directory "some_path/foo_SUITE_data/".

The priv_dir is the test suite's private directory. This directory should be used when a test case needs to write to files. The name of the private directory is generated by the test server, which also creates the directory.

Warning: Do not depend on current directory to be writable, or to point to anything in particular. All scratch files are to be written in the priv_dir, and all data files found in data_dir. If the current directory has to be something specific, it must be set with file:set_cwd/1.

Each time a test case is about to be executed, a new process is created with spawn_link. This is so that the test case will have no dependencies to earlier tests, with respect to process flags, process links, messages in the queue, other processes having registered the process, etc. As little as possible is done to change the initial context of the process (what is created by plain spawn). Here is a list of differences:

There is no time limit for a test case, unless the test case itself imposes such a limit, by calling test_server:timetrap/1 for example. The call can be made in each test case, or in the init_per_testcase/2 function. Make sure to call the corresponding test_server:timetrap_cancel/1 function as well, e.g in the end_per_testcase/2 function, or else the test cases will always fail.