20022009 Ericsson AB. All Rights Reserved. The contents of this file are subject to the Erlang Public License, Version 1.1, (the "License"); you may not use this file except in compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. Write you own test server framework Siri Hansen write_framework_chapter.xml
Introduction

The test server controller can be interfaced from the operating system or from within Erlang. The nature of your new framework will decide which interface to use. If you want your framework to start a new node for each test, the operating system interface is very convenient. If your node is already started, going from within Erlang might be a more flexible solution.

The two methods are described below.

Interfacing the test server controller from Erlang

Using the test server from Erlang means that you have to start the test server and then add test jobs. Use test_server_ctrl:start/0 to start a local target or test_server_ctrl:start/1 to start a remote target. The test server is stopped by test_server_ctrl:stop/0.

The argument to test_server_ctrl:start/1 is the name of a parameter file. The parameter file specifies what type of target to start and where to start it, as well as some additional parameters needed for different target types. See the reference manual for a detailed description of all valid parameters.

Adding test jobs

There are many commands available for adding test cases to the test server's job queue:

Single test case

test_server_ctrl:add_case/2/3 Multiple test cases from same suite

test_server_ctrl:add_cases/2/3 Test suite module or modules

test_server_ctrl:add_module/1/2 Some or all test suite modules in a directory

test_server_ctrl:add_dir/2/3 Test cases specified in a test specification file

test_server_ctrl:add_spec/1

All test suites are given a unique name, which is usually given when the test suite is added to the job queue. In some cases, a default name is used, as in the case when a module is added without a specified name. The test job name is used to store logfiles, which are stored in the `name.logs' directory under the current directory.

See the reference manual for details about the functions for adding test jobs.

Interfacing the test server controller from the operating system.

The function run_test/1 is your interface in the test server controller if you wish to use it from the operating system. You simply start an erlang shell and invoke this function with the -s option. run_test/1 starts the test server, runs the test specified by the command line and stops the test server. The argument to run_test/1 is a list of command line flags, typically ['KEY1', Value1, 'KEY2', Value2, ...]. The valid command line flags are listed in the reference manual for test_server_ctrl.

A typical command line may look like this

erl -noshell -s test_server_ctrl run_test KEY1 Value1 KEY2 Value2 ... -s erlang halt

Or make an alias (this is for unix/tcsh)

alias erl_test 'erl -noshell -s test_server_ctrl run_test \!* -s erlang halt'

And then use it like this

erl_test KEY1 Value1 KEY2 Value2 ...

An Example

An example of starting a test run from the command line

erl -name test_srv -noshell -rsh /home/super/otp/bin/ctrsh

-pa /clearcase/otp/erts/lib/kernel/test

-boot start_sasl -sasl errlog_type error

-s test_server_ctrl run_test SPEC kernel.spec -s erlang halt

Framework callback functions

By defining the environment variable TEST_SERVER_FRAMEWORK to a module name, the framework callback functions can be used. The framework callback functions are called by the test server in order let the framework interact with the execution of the tests and to keep the framework upto date with information about the test progress.

The framework callback functions are described in the reference manual for test_server_ctrl.

Note that this topic is in an early stage of development, and changes might occur.

Other concerns

Some things to think about when writing you own test server framework:

emulator version - Make sure that the intended version of the emulator is started. operating system path - If test cases use port programs, make sure the paths are correct. recompilation - Make sure all test suites are fresh compiled. test_server.hrl - Make sure the test_server.hrl file is in the include path when compiling test suites. running applications - Some test suites require some applications to be running (e.g. sasl). Make sure they are started.