From fc13629197e7c3698ff0739b51d8219069444dec Mon Sep 17 00:00:00 2001
From: Kenneth Lundin
The
With
The
This function is called as the last test case in the
suite. It is meant to be used for cleaning up after
-
The
With
The
OPTIONAL
This function is called after the execution of a test case group is finished.
- It is meant to be used for cleaning up after
OPTIONAL
This function is called after each test case, and can be used
- to clean up after
This is the test case info function. It is supposed to
return a list of tagged tuples that specify various properties
related to the execution of this particular test case.
- Properties set by
The
If
With
Other tuples than the ones defined will simply be ignored.
@@ -554,7 +554,7 @@This is the implementation of a test case. Here you must
call the functions you want to test, and do whatever you
need to check the result. If something fails, make sure the
- function causes a runtime error, or call
Elements from the
To read the value of a config variable, use the function
-
Example:
@@ -121,7 +121,7 @@If a configuration variable is defined in multiple files and you
- want to access all possible values, you may use the
It is possible to encrypt configuration files containing sensitive data if these files must be stored in open and shared directories.
-Call
Call
Common Test also provides decryption functions,
-
Please see the There are two different methods for opening a connection
- by means of the support functions in e.g.
You may also pass the cover specification file name in a
- call to
By default the Cover tool is automatically stopped when the tests are completed. This causes the original (non cover @@ -120,8 +120,8 @@
The option can be set by using the
The cross cover mechanism allows cover analysis of modules
across multiple tests. It is useful if some code, e.g. a library
diff --git a/lib/common_test/doc/src/ct_hooks_chapter.xml b/lib/common_test/doc/src/ct_hooks_chapter.xml
index fe871eb516..a706bbf9e6 100644
--- a/lib/common_test/doc/src/ct_hooks_chapter.xml
+++ b/lib/common_test/doc/src/ct_hooks_chapter.xml
@@ -192,12 +192,12 @@
It's possible in the CTH to read configuration data values
- by calling The CT hook functions may call any of the logging functions available
in the If using multiple CTHs, the first part of the return tuple will be
used as input for the next CTH. So in the case above the next CTH might
get Recovering from a testcase failure using CTHs should only be done as
a last resort. If used wrongly it could become very difficult to
- determine which tests pass or fail in a test run
The default
Note that it is possible to specify an The There is an interface function that corresponds to this program,
- called it prints all valid start flags to stdout. Event handlers may be installed by means of an An event_handler tuple in the argument
ct_run [-dir TestDir1 TestDir2 .. TestDirN] |
diff --git a/lib/common_test/doc/src/event_handler_chapter.xml b/lib/common_test/doc/src/event_handler_chapter.xml
index 10a9b52d39..3cc21f28de 100644
--- a/lib/common_test/doc/src/event_handler_chapter.xml
+++ b/lib/common_test/doc/src/event_handler_chapter.xml
@@ -64,7 +64,7 @@
{event_handler,EventHandlers} @@ -224,8 +224,9 @@end_per_testcase for the case failed. +- - +
#event{name = tc_auto_skip, data = {Suite,Func,Reason}} +#event{name = tc_auto_skip, data = {Suite,Func,Reason}}
Suite = atom() , the name of the suite.
Func = atom() , the name of the test case or configuration function.
Reason = {failed,FailReason} | @@ -251,8 +252,9 @@ the tc_done event.- #event{name = tc_user_skip, data = {Suite,TestCase,Comment}} +#event{name = tc_user_skip, data = {Suite,TestCase,Comment}}
Suite = atom() , name of the suite.
TestCase = atom() , name of the test case.@@ -308,7 +310,7 @@ manager can look like.
Comment = string() , reason for skipping the test case.To ensure that printouts to standard out (or printouts made with -
or ct:log/2/3 ) get written to the test case log + ct:pal/2/3 or ct:log/2/3 ) get written to the test case log file, and not to the Common Test framework log, you can syncronize with the Common Test server by matching on the ct:pal/2/3 tc_start andtc_done events. In the period between these events, all IO gets directed to the diff --git a/lib/common_test/doc/src/getting_started_chapter.xml b/lib/common_test/doc/src/getting_started_chapter.xml index 3cf04bb1a2..0b42445540 100644 --- a/lib/common_test/doc/src/getting_started_chapter.xml +++ b/lib/common_test/doc/src/getting_started_chapter.xml @@ -61,13 +61,11 @@Test case execution Execution of test cases is handled this way:
-
- Successful vs unsuccessful test case execution. For each test case that Common Test is told to execute, it spawns a dedicated process on which the test case function in question starts @@ -90,7 +88,7 @@
As you can understand from the illustration above, Common Test requires that a test case generates a runtime error to indicate failure (e.g. by causing a bad match error or by calling
@@ -100,13 +98,15 @@exit/1 , preferrably - through thehelp function). A succesful execution is + through the ct:fail/1,2 help function). A succesful execution is indicated by means of a normal return from the test case function. ct:fail/1,2 As you've seen in the basics chapter, the test suite module implements
callback functions (mandatory or optional) for various purposes, e.g: +- The configuration functions are optional and if you don't need them for +
- Init/end configuration function for the test suite
- Init/end configuration function for a test case
- Init/end configuration function for a test case group
- Test cases
+ The configuration functions are optional and if you don't need them for your test, a test suite with one simple test case could look like this:
@@ -136,13 +136,11 @@ "lower level"). The data flow looks like this: -
- Config data flow in the suite. Here's an example of a test suite which uses configuration functions @@ -203,13 +201,11 @@ shows the log file structure:
-
- HTML log file structure. diff --git a/lib/common_test/doc/src/install_chapter.xml b/lib/common_test/doc/src/install_chapter.xml index 89c497962d..4ef4e6de94 100644 --- a/lib/common_test/doc/src/install_chapter.xml +++ b/lib/common_test/doc/src/install_chapter.xml @@ -56,13 +56,13 @@ shell script version run_test, however, this script needs to be generated first, according to the instructions below. - +
Before reading on, please note that since Common Test version + Before reading on, please note that since Common Test version 1.5, the run_test shell script is no longer required for starting tests with Common Test from the OS command line. The ct_run program (descibed above) is the new recommended command line interface for Common Test. The shell script exists mainly for legacy reasons and may not be updated in future releases of Common Test. It may even be removed. -
Optional step to generate a shell script for starting Common Test:
To generate the run_test shell script, navigate to the diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index afaed29626..657a72ef8c 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -105,8 +105,8 @@ RPC from a remote node.
-+ -Running tests from the OS command line The
ct_run program can be used for running tests from @@ -225,15 +225,15 @@Common Test provides an Erlang API for running tests. The main (and most flexible) function for specifying and executing tests is called -
. + ct:run_test/1 . This function takes the same start parameters as - the ct:run_test/1 + the ct_run program described above, only the flags are instead given as options in a list of key-value tuples. E.g. a test specified with ct_run ct_run like:-
$ ct_run -suite ./my_SUITE -logdir ./results is with
+specified as: ct:run_test/1 is with
specified as: ct:run_test/1
1> ct:run_test([{suite,"./my_SUITE"},{logdir,"./results"}]). The function returns the test result, represented by the tuple: @@ -245,7 +245,7 @@
Releasing the Erlang shell During execution of tests, started with -
, + ct:run_test/1 , the Erlang shell process, controlling stdin, will remain the top level process of the Common Test system of processes. The result is that the Erlang shell is not available for interaction during @@ -260,19 +260,19 @@ ct:run_test/1 ct:run_test/1 returns the pid of this process rather than the test result - which instead is printed to tty at the end of the test run.Note that in order to use the -
and - ct:break/1/2 functions, + ct:continue/0/1 and + ct:break/1/2 functions, ct:continue/0/1 release_shell must be set totrue .For detailed documentation about -
+, + ct:run_test/1 , please see the - ct:run_test/1 manual page. ct manual page. ct + @@ -503,7 +503,7 @@Test case group execution With the
ct_run flag, orct:run_test/1 optiongroup , @@ -442,9 +442,9 @@ for trying out various operations during test suite development.To invoke the interactive shell mode, you can start an Erlang shell - manually and call
to install any configuration + manually and call ct:install/1 to install any configuration data you might need (use ct:install/1 [] as argument otherwise), then - callto start Common Test. If you use + call ct:start_interactive/0 to start Common Test. If you use the ct:start_interactive/0 ct_run program, you may start the Erlang shell and Common Test in the same go by using the-shell and, optionally, the-config and/or-userconfig flag. Examples: @@ -463,8 +463,8 @@If any functions using "required config data" (e.g. ct_telnet or ct_ftp functions) are to be called from the erlang shell, config - data must first be required with
. This is + data must first be required with - ct:require/1/2 . This is equivalent to a + ct:require/1/2 require statement in theTest Suite Info Function or in theIf you wish to exit the interactive mode (e.g. to start an - automated test run with
), call the function - ct:run_test/1 . This shuts down the + automated test run with ct:stop_interactive/0 ), call the function + ct:run_test/1 . This shuts down the running ct:stop_interactive/0 ct application. Associations between configuration names and data created withrequire are - consequently deleted.will get you + consequently deleted. ct:start_interactive/0 will get you back into interactive mode, but the previous state is not restored. ct:start_interactive/0 Step by step execution of test cases with the Erlang Debugger By means of
@@ -527,17 +527,17 @@ withct_run -step [opts] , or by passing the -{step,Opts} option to, it is possible + ct:run_test/1 {step,Opts} option to, it is possible to get the Erlang Debugger started automatically and use its graphical interface to investigate the state of the current test case and to execute it step by step and/or set execution breakpoints. ct:run_test/1 dir . -+ -Test Specifications @@ -999,7 +999,7 @@ General description The most flexible way to specify what to test, is to use a so called test specification. A test specification is a sequence of Erlang terms. The terms are normally declared in one or more text files - (see
), but + (see ct:run_test/1 ), but may also be passed to Common Test on the form of a list (see - ct:run_test/1 ). + ct:run_testspec/1 ). There are two general types of terms: configuration terms and test specification terms. ct:run_testspec/1 With configuration terms it is possible to e.g. label the test @@ -989,7 +989,7 @@
ct_run . This forces Common Test to ignore unrecognizable terms. Note that in this mode, Common Test is not able to check the specification for errors as efficiently as if the scanner runs in default mode. - Ifis used + If ct:run_test/1 is used for starting the tests, the relaxed scanner mode is enabled by means of the tuple: ct:run_test/1 {allow_user_terms,true} Running tests from the Web based GUI The web based GUI, VTS, is started with the -
+ ct_run program. From the GUI you can load config files, and select directories, suites and cases to run. You can also state the config files, directories, suites and cases on the command line diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml index cc8d913994..a33b22ac39 100644 --- a/lib/common_test/doc/src/write_test_chapter.xml +++ b/lib/common_test/doc/src/write_test_chapter.xml @@ -129,8 +129,8 @@ ct_run + Init and end per test case Each test suite module can contain the optional configuration functions @@ -173,7 +173,7 @@
The
end_per_testcase/2 function is called even after a - test case terminates due to a call to, + test case terminates due to a call to ct:abort_current_testcase/1 , or after a timetrap timeout. However, ct:abort_current_testcase/1 end_per_testcase will then execute on a different process than the test case function, and in this situation,end_per_testcase will @@ -243,8 +243,8 @@The test case function argument
Config should not be confused with the information that can be retrieved from - configuration files (using). The Config argument + configuration files (using - ct:get_config/1/2 ). The Config argument should be used for runtime configuration of the test suite and the test cases, while configuration files should typically contain data related to the SUT. These two types of configuration data are handled @@ -303,7 +303,7 @@ + ct:get_config/1/2 - @@ -348,8 +348,8 @@
Use this to specify arbitrary data related to the testcase. This - data can be retrieved at any time using the
+ data can be retrieved at any time using the ct:userdata/3 utility function. ct:userdata/3 See the
@@ -826,14 +826,16 @@ Common Test to create one dedicated private directory per test case and execution instead. This is accomplished by means of the flag/option:Config files - chapter and thefunction in the + chapter and the - ct:require/1/2 function in the + ct:require/1/2 ct reference manual for more information aboutrequire .create_priv_dir (to be used with the -ct_run program, thefunction, or + ct:run_test/1 ct_run program, thefunction, or as test specification term). There are three possible values for this option: + ct:run_test/1 +
auto_per_run auto_per_tc manual_per_tc The first value indicates the default priv_dir behaviour, i.e. one private directory created per test run. The two latter values tell Common Test to generate a unique test directory name @@ -842,7 +844,7 @@ become very inefficient for test runs with many test cases and/or repetitions. Therefore, in case the manual version is instead used, the test case must tell Common Test to create priv_dir when it needs it. - It does this by calling the function
. + It does this by calling the function ct:make_priv_dir/0 . ct:make_priv_dir/0 You should not depend on current working directory for @@ -890,7 +892,7 @@
It is also possible to dynamically set/reset a timetrap during the excution of a test case, or configuration function. This is done by calling -
@@ -903,12 +905,12 @@. This function cancels the current timetrap + ct:timetrap/1 . This function cancels the current timetrap and starts a new one (that stays active until timeout, or end of the current function). ct:timetrap/1 If a test case needs to suspend itself for a time that also gets multipled by
multiply_timetraps (and possibly also scaled up if -scale_timetraps is enabled), the function+ ct:sleep/1 scale_timetraps is enabled), the functionmay be used (instead of e.g. ct:sleep/1 timer:sleep/1 ).A function (
+ well as argument to thefun/0 orMFA ) may be specified as timetrap value in the suite-, group- and test case info function, as - well as argument to thefunction. Examples: ct:timetrap/1 function. Examples: ct:timetrap/1
{timetrap,{my_test_utils,timetrap,[?MODULE,system_start]}} -- cgit v1.2.3
ct:timetrap(fun() -> my_timetrap(TestCaseName, Config) end)