diff options
author | Erlang/OTP <[email protected]> | 2010-06-11 09:08:08 +0000 |
---|---|---|
committer | Erlang/OTP <[email protected]> | 2010-06-11 09:08:08 +0000 |
commit | c0f23851df50d6215981805102754b590c05acec (patch) | |
tree | 5a202fe2abc7cd77e9808930758b81d05957fab6 /lib/common_test/doc | |
parent | a28aa321007f62ae4dce415cd190778c3ac184e7 (diff) | |
parent | 2d7ba88ebbb59a473dbcefd0f9dee1f1b816935e (diff) | |
download | otp-c0f23851df50d6215981805102754b590c05acec.tar.gz otp-c0f23851df50d6215981805102754b590c05acec.tar.bz2 otp-c0f23851df50d6215981805102754b590c05acec.zip |
Merge branch 'peppe/common_test_r14_dev_2' into dev
* peppe/common_test_r14_dev_2:
Misc documentation updates
Add documentation for run_test program
Step vsn for test_server to 3.4
Make {repeat*,N} property in group execute the group N times exactly
Fix so that ct_run converts relative diretories in the code path to absolute
Fix bug in handling framework:end_tc timeouts
Fix bug that prevents the interactive shell mode to start properly
Fix failing multiply timetrap test case
Minor fixes in code and test suites
Add support for executing pre-loaded suites (e.g. modules loaded as binaries)
Add test suite for remote loading of binary suites
Fix error with {repeat,0} property in groups causing double iterations
Add support for config info functions (e.g. init_per_suite/0)
Add support for dynamic timetrap handling
Have end_per_testcase run even after timetrap_timeout and abort_testcase
Flush old DOWN messages in demonitor
Add groups in test specifications
Add new tests for test case groups and test specifications
Improve and fix various test suites
Add event_handler_init start flag that can pass init arguments to event handlers
...
OTP-8703 peppe/common_test_r14_dev_2
Diffstat (limited to 'lib/common_test/doc')
-rw-r--r-- | lib/common_test/doc/src/Makefile | 3 | ||||
-rw-r--r-- | lib/common_test/doc/src/common_test_app.xml | 10 | ||||
-rw-r--r-- | lib/common_test/doc/src/config_file_chapter.xml | 252 | ||||
-rw-r--r-- | lib/common_test/doc/src/ct_master_chapter.xml | 62 | ||||
-rw-r--r-- | lib/common_test/doc/src/event_handler_chapter.xml | 22 | ||||
-rw-r--r-- | lib/common_test/doc/src/install_chapter.xml | 139 | ||||
-rw-r--r-- | lib/common_test/doc/src/ref_man.xml | 7 | ||||
-rw-r--r-- | lib/common_test/doc/src/run_test.xml | 108 | ||||
-rw-r--r-- | lib/common_test/doc/src/run_test_chapter.xml | 78 | ||||
-rw-r--r-- | lib/common_test/doc/src/test_structure_chapter.xml | 10 | ||||
-rw-r--r-- | lib/common_test/doc/src/write_test_chapter.xml | 44 |
11 files changed, 550 insertions, 185 deletions
diff --git a/lib/common_test/doc/src/Makefile b/lib/common_test/doc/src/Makefile index a2c014418d..6322860088 100644 --- a/lib/common_test/doc/src/Makefile +++ b/lib/common_test/doc/src/Makefile @@ -45,7 +45,8 @@ CT_MODULES = \ ct_ssh \ ct_rpc \ ct_snmp \ - unix_telnet + unix_telnet \ + ct_slave CT_XML_FILES = $(CT_MODULES:=.xml) diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml index 7b52883f8a..e30eef2488 100644 --- a/lib/common_test/doc/src/common_test_app.xml +++ b/lib/common_test/doc/src/common_test_app.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Common Test</title> @@ -337,7 +337,7 @@ </func> <func> - <name>Module:testcase() -> [Info] </name> + <name>Module:Testcase() -> [Info] </name> <fsummary>Test case info function. </fsummary> <type> <v> Info = {timetrap,Time} | {require,Required} | @@ -396,7 +396,7 @@ <func> - <name>Module:testcase(Config) -> void() | {skip,Reason} | {comment,Comment} | {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit() </name> + <name>Module:Testcase(Config) -> void() | {skip,Reason} | {comment,Comment} | {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit() </name> <fsummary>A test case</fsummary> <type> <v> Config = SaveConfig = [{Key,Value}]</v> diff --git a/lib/common_test/doc/src/config_file_chapter.xml b/lib/common_test/doc/src/config_file_chapter.xml index a22a5270c1..850bc74e19 100644 --- a/lib/common_test/doc/src/config_file_chapter.xml +++ b/lib/common_test/doc/src/config_file_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2009</year> + <year>2004</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Config Files</title> @@ -180,7 +180,126 @@ </section> <section> - <title>Examples</title> + <title>User specific configuration data formats</title> + + <p>It is possible for the user to specify configuration data on a + different format than key-value tuples in a text file, as described + so far. The data can e.g. be read from arbitrary files, fetched from + the web over http, or requested from a user specific process. + To support this, Common Test provides a callback module plugin + mechanism to handle configuration data.</p> + + <section> + <title>Default callback modules for handling configuration data</title> + <p>The Common Test application includes default callback modules + for handling configuration data specified in standard config files + (see above) and in xml files:</p> + <list> + <item> + <c>ct_config_plain</c> - for reading configuration files with + key-value tuples (standard format). This handler will be used to + parse configuration files if no user callback is specified. + </item> + <item> + <c>ct_config_xml</c> - for reading configuration data from XML + files. + </item> + </list> + </section> + + <section> + <title>Using XML configuration files</title> + <p>This is an example of an XML configuration file:</p> + <pre><![CDATA[ +<config> + <ftp_host> + <ftp>"targethost"</ftp> + <username>"tester"</username> + <password>"letmein"</password> + </ftp_host> + <lm_directory>"/test/loadmodules"</lm_directory> +</config>]]></pre> + + <p>This configuration file, once read, will produce the same configuration + variables as the following text file:</p> + <pre> +{ftp_host, [{ftp,"targethost"}, + {username,"tester"}, + {password,"letmein"}]}. + +{lm_directory, "/test/loadmodules"}.</pre> + </section> + + <section> + <title>How to implement a user specific handler</title> + + <p>The user specific handler can be written to handle special + configuration file formats. The parameter can be either file + name(s) or configuration string(s) (the empty list is valid).</p> + + <p>The callback module implementing the handler is responsible for + checking correctness of configuration strings.</p> + + <p>To perform validation of the configuration strings, the callback module + should have the following function exported:</p> + + <p><c>Callback:check_parameter/1</c></p> + <p>The input argument will be passed from Common Test, as defined in the test + specification or given as an option to <c>run_test</c>.</p> + + <p>The return value should be any of the following values indicating if given + configuration parameter is valid:</p> + <list> + <item> + <c>{ok, {file, FileName}}</c> - parameter is a file name and + the file exists, + </item> + <item> + <c>{ok, {config, ConfigString}}</c> - parameter is a config string + and it is correct, + </item> + <item> + <c>{error, {nofile, FileName}}</c> - there is no file with the given + name in the current directory, + </item> + <item> + <c>{error, {wrong_config, ConfigString}}</c> - the configuration string + is wrong. + </item> + </list> + + <p>To perform reading of configuration data - initially before the tests + start, or as a result of data being reloaded during test execution - + the following function should be exported from the callback module:</p> + + <p><c>Callback:read_config/1</c></p> + + <p>The input argument is the same as for the <c>check_parameter/1</c> function.</p> + <p>The return value should be either:</p> + + <list> + <item> + <c>{ok, Config}</c> - if the configuration variables are read successfully, + </item> + <item> + <c>{error, Error, ErrorDetails}</c> - if the callback module fails to + proceed with the given configuration parameters. + </item> + </list> + <p><c>Config</c> is the proper Erlang key-value list, with possible + key-value sublists as values, like for the configuration file + example above:</p> + + <pre> + [{ftp_host, [{ftp, "targethost"}, {username, "tester"}, {password, "letmein"}]}, + {lm_directory, "/test/loadmodules"}]</pre> + + </section> + + </section> + + <section> + <title>Examples of configuration data handling</title> <p>A config file for using the FTP client to access files on a remote host could look like this:</p> @@ -188,28 +307,33 @@ <pre> {ftp_host, [{ftp,"targethost"}, {username,"tester"}, - {password,"letmein"}]}. + {password,"letmein"}]}. {lm_directory, "/test/loadmodules"}.</pre> - <p>Example of how to assert that the configuration data is available and + + <p>The XML version shown in the chapter above can also be used, but it should be + explicitly specified that the <c>ct_config_xml</c> callback module is to be + used by Common Test.</p> + + <p>Example of how to assert that the configuration data is available and use it for an FTP session:</p> <pre> init_per_testcase(ftptest, Config) -> {ok,_} = ct_ftp:open(ftp), - Config. + Config. end_per_testcase(ftptest, _Config) -> ct_ftp:close(ftp). ftptest() -> [{require,ftp,ftp_host}, - {require,lm_directory}]. + {require,lm_directory}]. ftptest(Config) -> - Remote = filename:join(ct:get_config(lm_directory), "loadmodX"), + Remote = filename:join(ct:get_config(lm_directory), "loadmodX"), Local = filename:join(?config(priv_dir,Config), "loadmodule"), ok = ct_ftp:recv(ftp, Remote, Local), - ...</pre> + ...</pre> <p>An example of how the above functions could be rewritten if necessary to open multiple connections to the FTP server:</p> @@ -217,7 +341,7 @@ init_per_testcase(ftptest, Config) -> {ok,Handle1} = ct_ftp:open(ftp_host), {ok,Handle2} = ct_ftp:open(ftp_host), - [{ftp_handles,[Handle1,Handle2]} | Config]. + [{ftp_handles,[Handle1,Handle2]} | Config]. end_per_testcase(ftptest, Config) -> lists:foreach(fun(Handle) -> ct_ftp:close(Handle) end, @@ -225,17 +349,117 @@ ftptest() -> [{require,ftp_host}, - {require,lm_directory}]. + {require,lm_directory}]. ftptest(Config) -> - Remote = filename:join(ct:get_config(lm_directory), "loadmodX"), + Remote = filename:join(ct:get_config(lm_directory), "loadmodX"), Local = filename:join(?config(priv_dir,Config), "loadmodule"), [Handle | MoreHandles] = ?config(ftp_handles,Config), ok = ct_ftp:recv(Handle, Remote, Local), - ...</pre> + ...</pre> </section> + <section> + <title>Example of user specific configuration handler</title> + <p>A simple configuration handling driver which will ask an external server for + configuration data can be implemented this way:</p> + <pre> +-module(config_driver). +-export([read_config/1, check_parameter/1]). + +read_config(ServerName)-> + ServerModule = list_to_atom(ServerName), + ServerModule:start(), + ServerModule:get_config(). + +check_parameter(ServerName)-> + ServerModule = list_to_atom(ServerName), + case code:is_loaded(ServerModule) of + {file, _}-> + {ok, {config, ServerName}}; + false-> + case code:load_file(ServerModule) of + {module, ServerModule}-> + {ok, {config, ServerName}}; + {error, nofile}-> + {error, {wrong_config, "File not found: " ++ ServerName ++ ".beam"}} + end + end.</pre> + + <p>The configuration string for this driver may be "config_server", if the + config_server.erl module below is compiled and exists in the code path + during test execution:</p> + <pre> +-module(config_server). +-export([start/0, stop/0, init/1, get_config/0, loop/0]). + +-define(REGISTERED_NAME, ct_test_config_server). + +start()-> + case whereis(?REGISTERED_NAME) of + undefined-> + spawn(?MODULE, init, [?REGISTERED_NAME]), + wait(); + _Pid-> + ok + end, + ?REGISTERED_NAME. + +init(Name)-> + register(Name, self()), + loop(). + +get_config()-> + call(self(), get_config). + +stop()-> + call(self(), stop). + +call(Client, Request)-> + case whereis(?REGISTERED_NAME) of + undefined-> + {error, not_started, Request}; + Pid-> + Pid ! {Client, Request}, + receive + Reply-> + {ok, Reply} + after 4000-> + {error, timeout, Request} + end + end. + +loop()-> + receive + {Pid, stop}-> + Pid ! ok; + {Pid, get_config}-> + {D,T} = erlang:localtime(), + Pid ! + [{localtime, [{date, D}, {time, T}]}, + {node, erlang:node()}, + {now, erlang:now()}, + {config_server_pid, self()}, + {config_server_vsn, ?vsn}], + ?MODULE:loop() + end. + +wait()-> + case whereis(?REGISTERED_NAME) of + undefined-> + wait(); + _Pid-> + ok + end.</pre> + + <p>In this example, the handler also provides the ability to dynamically reload + configuration variables. If <c>ct:reload_config(localtime)</c> is called from + the test case function, all variables loaded with <c>config_driver:read_config/1</c> + will be updated with their latest values, and the new value for variable + <c>localtime</c> will be returned.</p> + </section> + </chapter> diff --git a/lib/common_test/doc/src/ct_master_chapter.xml b/lib/common_test/doc/src/ct_master_chapter.xml index 79288cfe4c..01f8e61d36 100644 --- a/lib/common_test/doc/src/ct_master_chapter.xml +++ b/lib/common_test/doc/src/ct_master_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Using Common Test for Large Scale Testing</title> @@ -30,6 +30,7 @@ </header> <section> + <marker id="general"></marker> <title>General</title> <p>Large scale automated testing requires running multiple independent test sessions in parallel. This is accomplished by running @@ -102,9 +103,10 @@ <p><c>ct_master:abort()</c> (stop all) or <c>ct_master:abort(Nodes)</c></p> <p>For detailed information about the <c>ct_master</c> API, please see the - manual page for this module.</p> + <seealso marker="ct_master">manual page</seealso> for this module.</p> </section> <section> + <marker id="test_specifications"></marker> <title>Test Specifications</title> <p>The test specifications used as input to CT Master are fully compatible with the specifications used as input to the regular CT server. The syntax is described in the @@ -186,7 +188,7 @@ <seealso marker="run_test_chapter#test_specifications">Running Test Suites</seealso> chapter). The result is that any test specified to run on a node with the same name as the Common Test node in question (typically <c>ct@somehost</c> if started - with the <c>run_test</c> script), will be performed. Tests without explicit + with the <c>run_test</c> program), will be performed. Tests without explicit node association will always be performed too of course!</p> <note><p>It is recommended that absolute paths are used for log directories, @@ -194,6 +196,56 @@ current working directory settings are not important.</p></note> </section> + <section> + <title>Automatic startup of test target nodes</title> + <marker id="ct_slave"></marker> + <p>Is is possible to automatically start, and perform initial actions, on + test target nodes by using the test specification term <c>init</c>.</p> + <p>Currently, two sub-terms are supported, <c>node_start</c> and <c>eval</c>.</p> + <p>Example:</p> + <pre> + {node, node1, node1@host1}. + {node, node2, node1@host2}. + {node, node3, node2@host2}. + {node, node4, node1@host3}. + {init, node1, [{node_start, [{callback_module, my_slave_callback}]}]}. + {init, [node2, node3], {node_start, [{username, "ct_user"}, {password, "ct_password"}]}}. + {init, node4, {eval, {module, function, []}}}.</pre> + + <p>This test specification declares that <c>node1@host1</c> is to be started using + the user callback function <c>callback_module:my_slave_callback/0</c>, and nodes + <c>node1@host2</c> and <c>node2@host2</c> will be started with the default callback + module <c>ct_slave</c>. The given user name and password is used to log into remote + host <c>host2</c>. Also, the function <c>module:function/0</c> will be evaluated on + <c>node1@host3</c>, and the result of this call will be printed to the log.</p> + + <p>The default <seealso marker="ct_slave">ct_slave</seealso> callback module, + which is part of the Common Test application, has the following features: + <list> + <item>Starting Erlang target nodes on local or remote hosts + (ssh is used for communication). + </item> + <item>Ability to start an Erlang emulator with additional flags + (any flags supported by <c>erl</c> are supported). + </item> + <item>Supervision of a node being started by means of internal callback + functions. Used to prevent hanging nodes. (Configurable). + </item> + <item>Monitoring of the master node by the slaves. A slave node may be + stopped in case the master node terminates. (Configurable). + </item> + <item>Execution of user functions after a slave node is started. + Functions can be given as a list of {Module, Function, Arguments} tuples. + </item> + </list> + </p> + <p>Note that it is possible to specify an <c>eval</c> term for the node as well + as <c>startup_functions</c> in the <c>node_start</c> options list. In this + case first the node will be started, then the <c>startup_functions</c> are + executed, and finally functions specified with <c>eval</c> are called. + </p> + </section> + </chapter> diff --git a/lib/common_test/doc/src/event_handler_chapter.xml b/lib/common_test/doc/src/event_handler_chapter.xml index a550810850..7f5144b760 100644 --- a/lib/common_test/doc/src/event_handler_chapter.xml +++ b/lib/common_test/doc/src/event_handler_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Event Handling</title> @@ -68,29 +68,27 @@ Example:</p> <p><c>$ run_test -suite test/my_SUITE -event_handler handlers/my_evh1 handlers/my_evh2 -pa $PWD/handlers</c></p> + <p>Use the <c><![CDATA[run_test -event_handler_init]]></c> option instead of + <c><![CDATA[-event_handler]]></c> to pass start arguments to the event handler + init function.</p> <p>All event handler modules must have gen_event behaviour. Note also that these modules must be precompiled, and that their locations must be added explicitly to the Erlang code server search path (like in the example).</p> - <p>It is not possible to specify start arguments to the event handlers when - using the <c>run_test</c> script. You may however pass along start arguments - if you use the <c>ct:run_test/1</c> function. An event_handler tuple in the argument - <c>Opts</c> has the following definition (see also <c>ct:run_test/1</c> in the - reference manual):</p> + <p>An event_handler tuple in the argument <c>Opts</c> has the following + definition (see also <c>ct:run_test/1</c> in the reference manual):</p> <pre> {event_handler,EventHandlers} EventHandlers = EH | [EH] EH = atom() | {atom(),InitArgs} | {[atom()],InitArgs} - InitArgs = [term()] - </pre> + InitArgs = [term()]</pre> <p>Example:</p> <pre> - 1> ct:run_test([{suite,"test/my_SUITE"},{event_handler,[my_evh1,{my_evh2,[node()]}]}]). - </pre> + 1> ct:run_test([{suite,"test/my_SUITE"},{event_handler,[my_evh1,{my_evh2,[node()]}]}]).</pre> <p>This will install two event handlers for the <c>my_SUITE</c> test. Event handler <c>my_evh1</c> is started with <c>[]</c> as argument to the init function. Event handler <c>my_evh2</c> is started with the name of the current node in the init argument list.</p> diff --git a/lib/common_test/doc/src/install_chapter.xml b/lib/common_test/doc/src/install_chapter.xml index e1ff5abf6a..828588a673 100644 --- a/lib/common_test/doc/src/install_chapter.xml +++ b/lib/common_test/doc/src/install_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2007</year><year>2009</year> + <year>2007</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Installation</title> @@ -33,82 +33,77 @@ <marker id="general"></marker> <title>General information</title> - <p>The two main interfaces for running tests with Common Test - are an executable Bourne shell script named <c>run_test</c> and an - erlang module named <c>ct</c>. The shell script will work on Unix/Linux - (and Linux-like environments such as Cygwin on Windows) and the - <c>ct</c> interface functions can be called from the Erlang shell - (or from any Erlang function) on any supported platform.</p> - - <p>The Common Test application is installed with the Erlang/OTP - system and no explicit installation is required to start using - Common Test by means of the interface functions in the <c>ct</c> - module. If you wish to use <c>run_test</c>, however, this script - needs to be generated first, according to the instructions below.</p> - </section> + <p>The two main interfaces for running tests with Common Test + are an executable program named run_test and an + erlang module named <c>ct</c>. The run_test program + is compiled for the underlying operating system (e.g. Unix/Linux + or Windows) during the build of the Erlang/OTP system, and is + installed automatically with other executable programs in + the top level <c>bin</c> directory of Erlang/OTP. + The <c>ct</c> interface functions can be called from the Erlang shell, + or from any Erlang function, on any supported platform.</p> + + <p>A legacy Bourne shell script - also named run_test - exists, + which may be manually generated and installed. This script may be used + instead of the run_test program mentioned above, e.g. if the user + wishes to modify or customize the Common Test start flags in a simpler + way than making changes to the run_test C program.</p> - <section> - <title>Unix/Linux</title> - - <p>Go to the <c><![CDATA[common_test-<vsn>]]></c> directory, located - among the other OTP applications (under the OTP lib directory). Here you - execute the <c>install.sh</c> script with argument <c>local</c>:</p> - - <p><c> - $ ./install.sh local - </c></p> + <p>The Common Test application is installed with the Erlang/OTP + system and no additional installation step is required to start using + Common Test by means of the run_test executable program, and/or the interface + functions in the <c>ct</c> module. If you wish to use the legacy Bourne + shell script version of run_test, however, this script needs to be + generated first, according to the instructions below.</p> + + <p><note>Before reading on, please note that since Common Test version + 1.5, the run_test shell script is no longer required for starting + tests with Common Test from the OS command line. The run_test + program (descibed above) is the new recommended command line interface + for Common Test. The shell script exists mainly for legacy reasons and + may not be updated in future releases of Common Test. It may even be removed. + </note></p> + + <p>Optional step to generate a shell script for starting Common Test:</p> + <p>To generate the run_test shell script, navigate to the + <c><![CDATA[common_test-<vsn>]]></c> directory, located among the other + OTP applications (under the OTP lib directory). Here execute the + <c>install.sh</c> script with argument <c>local</c>:</p> + + <p><c> + $ ./install.sh local + </c></p> - <p>This generates the executable <c>run_test</c> script in the - <c><![CDATA[common_test-<vsn>/priv/bin]]></c> directory. The script - will include absolute paths to the Common Test and Test Server - application directories, so it's possible to copy or move the script to - a different location on the file system, if desired, without having to - update it. It's of course possible to leave the script under the - <c>priv/bin</c> directory and update the PATH variable accordingly (or - create a link or alias to it).</p> - - <p>If you, for any reason, have copied Common Test and Test Server - to a different location than the default OTP lib directory, you can - generate a <c>run_test</c> script with a different top level directory, - simply by specifying the directory, instead of <c>local</c>, when running - <c>install.sh</c>. Example:</p> - - <p><c> - $ install.sh /usr/local/test_tools + <p>This generates the executable run_test script in the + <c><![CDATA[common_test-<vsn>/priv/bin]]></c> directory. The script + will include absolute paths to the Common Test and Test Server + application directories, so it's possible to copy or move the script to + a different location on the file system, if desired, without having to + update it. It's of course possible to leave the script under the + <c>priv/bin</c> directory and update the PATH variable accordingly (or + create a link or alias to it).</p> + + <p>If you, for any reason, have copied Common Test and Test Server + to a different location than the default OTP lib directory, you can + generate a run_test script with a different top level directory, + simply by specifying the directory, instead of <c>local</c>, when running + <c>install.sh</c>. Example:</p> + + <p><c> + $ install.sh /usr/local/test_tools </c></p> <p>Note that the <c><![CDATA[common_test-<vsn>]]></c> and - <c><![CDATA[test_server-<vsn>]]></c> directories must be located under the - same top directory. Note also that the install script does not copy files - or update environment variables. It only generates the <c>run_test</c> - script.</p> + <c><![CDATA[test_server-<vsn>]]></c> directories must be located under the + same top directory. Note also that the install script does not copy files + or update environment variables. It only generates the run_test + script.</p> - <p>Whenever you install a new version of Erlang/OTP, the <c>run_test</c> - script needs to be regenerated, or updated manually with new directory names - (new version numbers), for it to "see" the latest Common Test and Test Server - versions.</p> + <p>Whenever you install a new version of Erlang/OTP, the run_test + script needs to be regenerated, or updated manually with new directory names + (new version numbers), for it to "see" the latest Common Test and Test Server + versions.</p> - <p>For more information on the <c>run_test</c> script and the <c>ct</c> - module, please see the reference manual.</p> - </section> - - <section> - <title>Windows</title> - - <p>On Windows it is very convenient to use Cygwin (<c>www.cygwin.com</c>) - for running Common Test and Erlang, since it enables you to use the - <c>run_test</c> script for starting Common Test. If you are a Cygwin - user, simply follow the instructions above for generating the <c>run_test</c> - script.</p> - - <p>If you do not use Cygwin, you have to rely on the API functions - in the <c>ct</c> module (instead of <c>run_test</c>) for running - Common Test as described initially in this chapter.</p> - - <p>If you, for any reason, have chosen to store Common Test and Test Server - in a different location than the default OTP lib directory, make - sure the <c>ebin</c> directories of these applications are included - in the Erlang code server path (so the application modules can be loaded).</p> </section> </chapter> diff --git a/lib/common_test/doc/src/ref_man.xml b/lib/common_test/doc/src/ref_man.xml index beb3ed3247..8be234d979 100644 --- a/lib/common_test/doc/src/ref_man.xml +++ b/lib/common_test/doc/src/ref_man.xml @@ -4,7 +4,7 @@ <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Common Test Reference Manual</title> @@ -75,6 +75,7 @@ <xi:include href="ct_snmp.xml"/> <xi:include href="ct_telnet.xml"/> <xi:include href="unix_telnet.xml"/> + <xi:include href="ct_slave.xml"/> </application> diff --git a/lib/common_test/doc/src/run_test.xml b/lib/common_test/doc/src/run_test.xml index d9dd22d411..d609c4287f 100644 --- a/lib/common_test/doc/src/run_test.xml +++ b/lib/common_test/doc/src/run_test.xml @@ -4,7 +4,7 @@ <comref> <header> <copyright> - <year>2007</year><year>2009</year> + <year>2007</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,62 +13,87 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> - <title>The run_test shell script</title> + <title>The run_test program</title> <prepared>Peter Andersson</prepared> <responsible>Peter Andersson</responsible> <docno></docno> <approved></approved> <checked></checked> - <date>2007-07-04</date> - <rev>PA1</rev> + <date>2010-04-01</date> + <rev>PA2</rev> <file>run_test.xml</file> </header> - <com>run_test</com> - <comsummary>Shell script used for starting - Common Test from the Unix command line. + <com>run_test</com> + <comsummary>Program used for starting Common Test from the + OS command line. </comsummary> <description> - <p>The <c>run_test</c> script is automatically generated as Common - Test is installed (please see the Installation chapter in the Common - Test User's Guide for more information). The script accepts a number - of different start flags. Some flags trigger <c>run_test</c> - to start the Common Test application and pass on data to it. Some - flags start an Erlang node prepared for running Common Test in a - particular mode.</p> - <p><c>run_test</c> also accepts Erlang emulator - flags. These are used when <c>run_test</c> calls <c>erl</c> to - start the Erlang node (making it possible to e.g. add directories to - the code server path, change the cookie on the node, start - additional applications, etc).</p> - <p>If <c>run_test</c> is called without parameters, it prints all valid - start flags to stdout.</p> + <p>The <c>run_test</c> program is automatically installed with Erlang/OTP + and Common Test (please see the Installation chapter in the Common + Test User's Guide for more information). The program accepts a number + of different start flags. Some flags trigger <c>run_test</c> + to start the Common Test application and pass on data to it. Some + flags start an Erlang node prepared for running Common Test in a + particular mode.</p> + + <p><c>run_test</c> also accepts Erlang emulator flags. These are used + when <c>run_test</c> calls <c>erl</c> to start the Erlang node + (making it possible to e.g. add directories to the code server path, + change the cookie on the node, start additional applications, etc).</p> + + <p>With the optional flag:</p> + <pre>-erl_args</pre> + <p>it's possible to divide the options on the <c>run_test</c> command line into + two groups, one that Common Test should process (those preceding <c>-erl_args</c>), + and one it should completely ignore and pass on directly to the emulator + (those following <c>-erl_args</c>). Options preceding <c>-erl_args</c> that Common Test + doesn't recognize, also get passed on to the emulator untouched. + By means of <c>-erl_args</c> the user may specify flags with the same name, but + with different destinations, on the <c>run_test</c> command line.</p> + <p>If <c>-pa</c> or <c>-pz</c> flags are specified in the Common Test group of options + (preceding <c>-erl_args</c>), relative directories will be converted to + absolute and re-inserted into the code path by Common Test (to avoid + problems loading user modules when Common Test changes working directory + during test runs). Common Test will however ignore <c>-pa</c> and <c>-pz</c> flags + following <c>-erl_args</c> on the command line. These directories are added + to the code path normally (i.e. on specified form)</p> + + <p>If <c>run_test</c> is called with option:</p> + <pre>-help</pre> + <p>it prints all valid start flags to stdout.</p> </description> <section> <title>Run tests from command line</title> <pre> - run_test [-dir TestDir1 TestDir2 .. TestDirN] | - [-suite Suite1 Suite2 .. SuiteN + run_test [-dir TestDir1 TestDir2 .. TestDirN] | + [-suite Suite1 Suite2 .. SuiteN [[-group Group1 Group2 .. GroupN] [-case Case1 Case2 .. CaseN]]] [-step [config | keep_inactive]] [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] [-decrypt_key Key] | [-decrypt_file KeyFile] [-logdir LogDir] [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] [-stylesheet CSSFile] [-cover CoverCfgFile] - [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] + [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] | + [-event_handler_init EvHandler1 InitArg1 and + EvHandler2 InitArg2 and .. EvHandlerN InitArgN] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-muliply_timetraps Multiplier] + [-scale_timetraps] [-repeat N [-force_stop]] | [-duration HHMMSS [-force_stop]] | [-until [YYMoMoDD]HHMMSS [-force_stop]] @@ -79,15 +104,21 @@ <pre> run_test -spec TestSpec1 TestSpec2 .. TestSpecN [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] [-decrypt_key Key] | [-decrypt_file KeyFile] [-logdir LogDir] [-allow_user_terms] [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] [-stylesheet CSSFile] [-cover CoverCfgFile] - [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] + [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] | + [-event_handler_init EvHandler1 InitArg1 and + EvHandler2 InitArg2 and .. EvHandlerN InitArgN] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-muliply_timetraps Multiplier] + [-scale_timetraps] [-repeat N [-force_stop]] | [-duration HHMMSS [-force_stop]] | [-until [YYMoMoDD]HHMMSS [-force_stop]] @@ -98,11 +129,15 @@ <pre> run_test -vts [-browser Browser] [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] [-decrypt_key Key] | [-decrypt_file KeyFile] [-dir TestDir1 TestDir2 .. TestDirN] | [-suite Suite [[-group Group] [-case Case]]] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-muliply_timetraps Multiplier] + [-scale_timetraps] [-basic_html]</pre> </section> <section> @@ -113,28 +148,23 @@ <section> <title>Run CT in interactive mode</title> <pre> - run_test -shell + run_test -shell [-config ConfigFile1 ConfigFile2 ... ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] [-decrypt_key Key] | [-decrypt_file KeyFile]</pre> </section> <section> - <title>Start an Erlang node with a given name</title> - <pre> - run_test -ctname NodeName</pre> - </section> - <section> <title>Start a Common Test Master node</title> <pre> run_test -ctmaster</pre> </section> <section> - <title>See also</title> - <p>Please read the <seealso marker="run_test_chapter">Running Test Suites</seealso> - chapter in the Common Test User's Guide for information about the meaning of the - different start flags.</p> + <title>See also</title> + <p>Please read the <seealso marker="run_test_chapter">Running Test Suites</seealso> + chapter in the Common Test User's Guide for information about the meaning of the + different start flags.</p> </section> </comref> - - diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index d731d18783..207df7f5b5 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Running Test Suites</title> @@ -97,25 +97,32 @@ the <c><![CDATA[{auto_compile,false}]]></c> option with <c><![CDATA[ct:run_test/1]]></c>. With automatic compilation disabled, the user is responsible for compiling the test suite modules - (and any help modules) before the test run. Common Test will only verify - that the specified test suites exist before starting the tests.</p> + (and any help modules) before the test run. If the modules can not be loaded + from the local file system during startup of Common Test, the user needs to + pre-load the modules before starting the test. Common Test will only verify + that the specified test suites exist (i.e. that they are, or can be, loaded). + This is useful e.g. if the test suites are transferred and loaded as binaries via + RPC from a remote node.</p> </section> <section> - <title>Running tests from the UNIX command line</title> + <title>Running tests from the OS command line</title> - <p>The script <c>run_test</c> can be used for running tests from - the Unix/Linux command line, e.g. + <p>The <c>run_test</c> program can be used for running tests from + the OS command line, e.g. </p> <list> <item><c><![CDATA[run_test -config <configfilenames> -dir <dirs>]]></c></item> <item><c><![CDATA[run_test -config <configfilenames> -suite <suiteswithfullpath>]]></c> </item> + <item><c><![CDATA[run_test -userconfig <callbackmodulename> <configfilenames> -suite <suiteswithfullpath>]]></c> + </item> <item><c><![CDATA[run_test -config <configfilenames> -suite <suitewithfullpath> -group <groupnames> -case <casenames>]]></c></item> </list> <p>Examples:</p> <p><c>$ run_test -config $CFGS/sys1.cfg $CFGS/sys2.cfg -dir $SYS1_TEST $SYS2_TEST</c></p> + <p><c>$ run_test -userconfig ct_config_xml $CFGS/sys1.xml $CFGS/sys2.xml -dir $SYS1_TEST $SYS2_TEST</c></p> <p><c>$ run_test -suite $SYS1_TEST/setup_SUITE $SYS2_TEST/config_SUITE</c></p> <p><c>$ run_test -suite $SYS1_TEST/setup_SUITE -case start stop</c></p> <p><c>$ run_test -suite $SYS1_TEST/setup_SUITE -group installation -case start stop</c></p> @@ -136,8 +143,14 @@ <seealso marker="cover_chapter#cover">Code Coverage Analysis</seealso>).</item> <item><c><![CDATA[-event_handler <event_handlers>]]></c>, to install <seealso marker="event_handler_chapter#event_handling">event handlers</seealso>.</item> + <item><c><![CDATA[-event_handler_init <event_handlers>]]></c>, to install + <seealso marker="event_handler_chapter#event_handling">event handlers</seealso> including start arguments.</item> <item><c><![CDATA[-include]]></c>, specifies include directories (see above).</item> <item><c><![CDATA[-no_auto_compile]]></c>, disables the automatic test suite compilation feature (see above).</item> + <item><c><![CDATA[-multiply_timetraps <n>]]></c>, extends <seealso marker="write_test_chapter#timetraps">timetrap + timeout</seealso> values.</item> + <item><c><![CDATA[-scale_timetraps <bool>]]></c>, enables automatic <seealso marker="write_test_chapter#timetraps">timetrap + timeout</seealso> scaling.</item> <item><c><![CDATA[-repeat <n>]]></c>, tells Common Test to repeat the tests n times (see below).</item> <item><c><![CDATA[-duration <time>]]></c>, tells Common Test to repeat the tests for duration of time (see below).</item> <item><c><![CDATA[-until <stop_time>]]></c>, tells Common Test to repeat the tests until stop_time (see below).</item> @@ -165,7 +178,7 @@ the current working directory of the Erlang Runtime System during the test run!</p> </note> - <p>For details on how to generate the <c>run_test</c> script, see the + <p>For more information about the <c>run_test</c> program, see the <seealso marker="install_chapter#general">Installation</seealso> chapter. </p> </section> @@ -174,7 +187,7 @@ <title>Running tests from the Web based GUI</title> <p>The web based GUI, VTS, is started with the <c>run_test</c> - script. From the GUI you can load config files, and select + program. From the GUI you can load config files, and select directories, suites and cases to run. You can also state the config files, directories, suites and cases on the command line when starting the web based GUI. @@ -198,10 +211,11 @@ <p>Example:</p> <p><c><![CDATA[$ run_test -vts -browser 'firefox&']]></c></p> <p>Note that the browser must run as a separate OS process or VTS will hang!</p> - <p>If no specific browser start command is specified, netscape will + <p>If no specific browser start command is specified, Firefox will be the default browser on Unix platforms and Internet Explorer on Windows. - If Common Test fails to start a browser automatically, start your - favourite browser manually instead and type in the URL that Common Test + If Common Test fails to start a browser automatically, or <c>'none'</c> is + specified as the value for -browser (i.e. <c>-browser none</c>), start your + favourite browser manually and type in the URL that Common Test displays in the shell.</p> </section> @@ -211,7 +225,7 @@ <p>Common Test provides an Erlang API for running tests. The main (and most flexible) function for specifying and executing tests is called <c>ct:run_test/1</c>. This function takes the same start parameters as - the <c>run_test</c> script described above, only the flags are instead + the <c>run_test</c> program described above, only the flags are instead given as options in a list of key-value tuples. E.g. a test specified with <c>run_test</c> like:</p> <p><c>$ run_test -suite ./my_SUITE -logdir ./results</c></p> @@ -237,13 +251,14 @@ manually and call <c>ct:install/1</c> to install any configuration data you might need (use <c>[]</c> as argument otherwise), then call <c>ct:start_interactive/0</c> to start Common Test. If you use - the <c>run_test</c> script, you may start the Erlang shell and Common Test + the <c>run_test</c> program, you may start the Erlang shell and Common Test in the same go by using the <c>-shell</c> and, optionally, the <c>-config</c> - flag: + and/or <c>-userconfig</c> flag. Examples: </p> <list> <item><c>run_test -shell</c></item> - <item><c><![CDATA[run_test -shell -config <configfilename>]]></c></item> + <item><c><![CDATA[run_test -shell -config cfg/db.cfg]]></c></item> + <item><c><![CDATA[run_test -shell -userconfig db_login testuser x523qZ]]></c></item> </list> <p>If no config file is given with the <c>run_test</c> command, @@ -268,7 +283,8 @@ 2> ct_telnet:open(unix_telnet). {ok,<0.105.0>} 4> ct_telnet:cmd(unix_telnet, "ls ."). - {ok,["ls .","file1 ...",...]}</pre> + {ok,["ls .","file1 ...",...]} + </pre> <p>Everything that Common Test normally prints in the test case logs, will in the interactive mode be written to a log named @@ -350,14 +366,21 @@ <p>Below is the test specification syntax. Test specifications can be used to run tests both in a single test host environment and in - a distributed Common Test environment. Node parameters are only relevant in the - latter (see the chapter about running Common Test in distributed mode for information). - For details on the event_handler term, see the + a distributed Common Test environment (Large Scale Testing). The init term, + as well as node parameters, are only relevant in the latter (see the + <seealso marker="ct_master_chapter#test_specifications">Large Scale Testing</seealso> + chapter for information). For details on the event_handler term, see the <seealso marker="event_handler_chapter#event_handling">Event Handling</seealso> chapter.</p> <p>Config terms:</p> <pre> {node, NodeAlias, Node}. + + {init, InitOptions}. + {init, [NodeAlias], InitOptions}. + + {multiply_timetraps, N}. + {scale_timetraps, Bool}. {cover, CoverSpecFile}. {cover, NodeRef, CoverSpecFile}. @@ -367,6 +390,9 @@ {config, ConfigFiles}. {config, NodeRefs, ConfigFiles}. + + {userconfig, {CallbackModule, ConfigStrings}}. + {userconfig, NodeRefs, {CallbackModule, ConfigStrings}}. {alias, DirAlias, Dir}. @@ -395,9 +421,12 @@ <p>Types:</p> <pre> NodeAlias = atom() + InitOptions = term() Node = node() NodeRef = NodeAlias | Node | master NodeRefs = all_nodes | [NodeRef] | NodeRef + N = integer() + Bool = true | false CoverSpecFile = string() IncludeDirs = string() | [string()] ConfigFiles = string() | [string()] @@ -449,9 +478,14 @@ <item>Secondly, the test for system t2 should run. The included suites are t2B and t2C. Included are also test cases test4, test1 and test7 in suite t2A. Note that the test cases will be executed in the specified order.</item> - <item>Lastly, all suites for systems t3 are to be completely skipped and this + <item>Lastly, all suites for systems t3 are to be completely skipped and this should be explicitly noted in the log files.</item> </list> + <p>It is possible to specify initialization options for nodes defined in the + test specification. Currently, there are options to start the node and/or to + evaluate any function on the node. + See the <seealso marker="ct_master_chapter#ct_slave">Automatic startup of + the test target nodes</seealso> chapter for details.</p> <p>It is possible for the user to provide a test specification that includes (for Common Test) unrecognizable terms. If this is desired, the <c>-allow_user_terms</c> flag should be used when starting tests with diff --git a/lib/common_test/doc/src/test_structure_chapter.xml b/lib/common_test/doc/src/test_structure_chapter.xml index c8628b3a7a..cd38ae0c7c 100644 --- a/lib/common_test/doc/src/test_structure_chapter.xml +++ b/lib/common_test/doc/src/test_structure_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Test Structure</title> @@ -146,8 +146,8 @@ <tag><em>run_test</em></tag> <item> - The name of an executable Bourne shell script that may be - used on Linux/Unix as an interface for specifying and running + The name of an executable program that may be + used as an interface for specifying and running tests with Common Test. </item> diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml index 212e3d85be..5afec6de6a 100644 --- a/lib/common_test/doc/src/write_test_chapter.xml +++ b/lib/common_test/doc/src/write_test_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Writing Test Suites</title> @@ -157,6 +157,15 @@ <c>{skipped,Reason}</c> (where Reason is a user specific term). </p> + <p>The <c>end_per_testcase/2</c> function is called even after a + test case terminates due to a call to <c>ct:abort_current_testcase/1</c>, + or after a timetrap timeout. However, <c>end_per_testcase</c> + will then execute on a different process than the test case + function, and in this situation, <c>end_per_testcase</c> will + not be able to change the reason for test case termination by + returning <c>{fail,Reason}</c>, nor will it be able to save data with + <c>{save_config,Data}</c>.</p> + <p>If <c>init_per_testcase</c> crashes, the test case itself is skipped automatically (so called <em>auto skipped</em>). If <c>init_per_testcase</c> returns a <c>skip</c> tuple, also then will the test case be skipped (so @@ -682,12 +691,33 @@ <c>end_per_suite</c> execute, like test cases, on dedicated Erlang processes. </p> + </section> + <section> + <title>Timetrap timeouts</title> + <marker id="timetraps"></marker> <p>The default time limit for a test case is 30 minutes, unless a - <c>timetrap</c> is specified either by the test case info function - or the <c>suite/0</c> function. - </p> - + <c>timetrap</c> is specified either by the suite info function + or a test case info function. The timetrap timeout value defined + in <c>suite/0</c> is the value that will be used for each test case + in the suite (as well as for the configuration functions + <c>init_per_suite/1</c> and <c>end_per_suite</c>). A timetrap timeout + value set with the test case info function will override the value set + by <c>suite/0</c>, but only for that particular test case.</p> + <p>It is also possible to set/reset a timetrap during test case (or + configuration function) execution. This is done by calling + <c>ct:timetrap/1</c>. This function will cancel the current timetrap + and start a new one.</p> + <p>Timetrap values can be extended with a multiplier value specified at + startup with the <c>multiply_timetraps</c> option. It is also possible + to let Test Server decide to scale up timetrap timeout values + automatically, e.g. if tools such as cover or trace are running during + the test. This feature is disabled by default and can be enabled with + the <c>scale_timetraps</c> start option.</p> + <p>If a test case needs to suspend itself for a time that also gets + multipled by <c>multiply_timetraps</c>, and possibly scaled up if + <c>scale_timetraps</c> is enabled, the function <c>ct:sleep/1</c> + may be called.</p> </section> <section> |