From fc13629197e7c3698ff0739b51d8219069444dec Mon Sep 17 00:00:00 2001
From: Kenneth Lundin
The
With
The
This function is called as the last test case in the
suite. It is meant to be used for cleaning up after
-
The
With
The
OPTIONAL
This function is called after the execution of a test case group is finished.
- It is meant to be used for cleaning up after
OPTIONAL
This function is called after each test case, and can be used
- to clean up after
This is the test case info function. It is supposed to
return a list of tagged tuples that specify various properties
related to the execution of this particular test case.
- Properties set by
The
If
With
Other tuples than the ones defined will simply be ignored.
@@ -554,7 +554,7 @@This is the implementation of a test case. Here you must
call the functions you want to test, and do whatever you
need to check the result. If something fails, make sure the
- function causes a runtime error, or call
Elements from the
To read the value of a config variable, use the function
-
Example:
@@ -121,7 +121,7 @@If a configuration variable is defined in multiple files and you
- want to access all possible values, you may use the
It is possible to encrypt configuration files containing sensitive data if these files must be stored in open and shared directories.
-Call
Call
Common Test also provides decryption functions,
-
Please see the There are two different methods for opening a connection
- by means of the support functions in e.g.
You may also pass the cover specification file name in a
- call to
By default the Cover tool is automatically stopped when the tests are completed. This causes the original (non cover @@ -120,8 +120,8 @@
The option can be set by using the
The cross cover mechanism allows cover analysis of modules
across multiple tests. It is useful if some code, e.g. a library
diff --git a/lib/common_test/doc/src/ct_hooks_chapter.xml b/lib/common_test/doc/src/ct_hooks_chapter.xml
index fe871eb516..a706bbf9e6 100644
--- a/lib/common_test/doc/src/ct_hooks_chapter.xml
+++ b/lib/common_test/doc/src/ct_hooks_chapter.xml
@@ -192,12 +192,12 @@
It's possible in the CTH to read configuration data values
- by calling The CT hook functions may call any of the logging functions available
in the If using multiple CTHs, the first part of the return tuple will be
used as input for the next CTH. So in the case above the next CTH might
get Recovering from a testcase failure using CTHs should only be done as
a last resort. If used wrongly it could become very difficult to
- determine which tests pass or fail in a test run
The default
Note that it is possible to specify an The There is an interface function that corresponds to this program,
- called it prints all valid start flags to stdout. Event handlers may be installed by means of an An event_handler tuple in the argument
ct_run [-dir TestDir1 TestDir2 .. TestDirN] |
diff --git a/lib/common_test/doc/src/event_handler_chapter.xml b/lib/common_test/doc/src/event_handler_chapter.xml
index 10a9b52d39..3cc21f28de 100644
--- a/lib/common_test/doc/src/event_handler_chapter.xml
+++ b/lib/common_test/doc/src/event_handler_chapter.xml
@@ -64,7 +64,7 @@
{event_handler,EventHandlers}
@@ -224,8 +224,9 @@
end_per_testcase for the case failed.
+ -
- #event{name = tc_auto_skip, data = {Suite,Func,Reason}}
+ #event{name = tc_auto_skip, data = {Suite,Func,Reason}}
Suite = atom() , the name of the suite.
Func = atom() , the name of the test case or configuration function.
Reason = {failed,FailReason} |
@@ -251,8 +252,9 @@
the tc_done event.
+ -
- #event{name = tc_user_skip, data = {Suite,TestCase,Comment}}
+ #event{name = tc_user_skip, data = {Suite,TestCase,Comment}}
Suite = atom() , name of the suite.
TestCase = atom() , name of the test case.
Comment = string() , reason for skipping the test case.
@@ -308,7 +310,7 @@
manager can look like.
To ensure that printouts to standard out (or printouts made with
- ct:log/2/3 or ct:pal/2/3 ) get written to the test case log
+ ct:log/2/3 or ct:pal/2/3 ) get written to the test case log
file, and not to the Common Test framework log, you can syncronize
with the Common Test server by matching on the tc_start and tc_done
events. In the period between these events, all IO gets directed to the
diff --git a/lib/common_test/doc/src/getting_started_chapter.xml b/lib/common_test/doc/src/getting_started_chapter.xml
index 3cf04bb1a2..0b42445540 100644
--- a/lib/common_test/doc/src/getting_started_chapter.xml
+++ b/lib/common_test/doc/src/getting_started_chapter.xml
@@ -61,13 +61,11 @@
Test case execution
Execution of test cases is handled this way:
-
Successful vs unsuccessful test case execution.
-
For each test case that Common Test is told to execute, it spawns a
dedicated process on which the test case function in question starts
@@ -90,7 +88,7 @@
As you can understand from the illustration above, Common Test requires
that a test case generates a runtime error to indicate failure (e.g.
by causing a bad match error or by calling exit/1 , preferrably
- through the ct:fail/1,2 help function). A succesful execution is
+ through the ct:fail/1,2 help function). A succesful execution is
indicated by means of a normal return from the test case function.
@@ -100,13 +98,15 @@
As you've seen in the basics chapter, the test suite module implements
callback functions
(mandatory or optional) for various purposes, e.g:
+
- Init/end configuration function for the test suite
- Init/end configuration function for a test case
- Init/end configuration function for a test case group
- Test cases
- The configuration functions are optional and if you don't need them for
+
+ The configuration functions are optional and if you don't need them for
your test, a test suite with one simple test case could look like this:
@@ -136,13 +136,11 @@
"lower level"). The data flow looks like this:
-
Config data flow in the suite.
-
Here's an example of a test suite which uses configuration functions
@@ -203,13 +201,11 @@
shows the log file structure:
-
HTML log file structure.
-
diff --git a/lib/common_test/doc/src/install_chapter.xml b/lib/common_test/doc/src/install_chapter.xml
index 89c497962d..4ef4e6de94 100644
--- a/lib/common_test/doc/src/install_chapter.xml
+++ b/lib/common_test/doc/src/install_chapter.xml
@@ -56,13 +56,13 @@
shell script version run_test, however, this script needs to be
generated first, according to the instructions below.
- Before reading on, please note that since Common Test version
+ Before reading on, please note that since Common Test version
1.5, the run_test shell script is no longer required for starting
tests with Common Test from the OS command line. The ct_run
program (descibed above) is the new recommended command line interface
for Common Test. The shell script exists mainly for legacy reasons and
may not be updated in future releases of Common Test. It may even be removed.
-
+
Optional step to generate a shell script for starting Common Test:
To generate the run_test shell script, navigate to the
diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml
index afaed29626..657a72ef8c 100644
--- a/lib/common_test/doc/src/run_test_chapter.xml
+++ b/lib/common_test/doc/src/run_test_chapter.xml
@@ -105,8 +105,8 @@
RPC from a remote node.
-
+
Running tests from the OS command line
The ct_run program can be used for running tests from
@@ -225,15 +225,15 @@
Common Test provides an Erlang API for running tests. The main (and most
flexible) function for specifying and executing tests is called
- ct:run_test/1 .
+ ct:run_test/1 .
This function takes the same start parameters as
- the ct_run
+ the ct_run
program described above, only the flags are instead
given as options in a list of key-value tuples. E.g. a test specified
with ct_run like:
$ ct_run -suite ./my_SUITE -logdir ./results
- is with ct:run_test/1 specified as:
+ is with ct:run_test/1 specified as:
1> ct:run_test([{suite,"./my_SUITE"},{logdir,"./results"}]).
The function returns the test result, represented by the tuple:
@@ -245,7 +245,7 @@
Releasing the Erlang shell
During execution of tests, started with
- ct:run_test/1 ,
+ ct:run_test/1 ,
the Erlang shell process, controlling stdin, will remain the top
level process of the Common Test system of processes. The result
is that the Erlang shell is not available for interaction during
@@ -260,19 +260,19 @@
ct:run_test/1 returns the pid of this process rather than the
test result - which instead is printed to tty at the end of the test run.
Note that in order to use the
- ct:break/1/2 and
- ct:continue/0/1 functions,
+ ct:break/1/2 and
+ ct:continue/0/1 functions,
release_shell must be set to true .
For detailed documentation about
- ct:run_test/1 ,
+ ct:run_test/1 ,
please see the
- ct manual page.
+ ct manual page.
-
+
Test case group execution
With the ct_run flag, or ct:run_test/1 option group ,
@@ -442,9 +442,9 @@
for trying out various operations during test suite development.
To invoke the interactive shell mode, you can start an Erlang shell
- manually and call ct:install/1 to install any configuration
+ manually and call ct:install/1 to install any configuration
data you might need (use [] as argument otherwise), then
- call ct:start_interactive/0 to start Common Test. If you use
+ call ct:start_interactive/0 to start Common Test. If you use
the ct_run program, you may start the Erlang shell and Common Test
in the same go by using the -shell and, optionally, the -config
and/or -userconfig flag. Examples:
@@ -463,8 +463,8 @@
If any functions using "required config data" (e.g. ct_telnet or
ct_ftp functions) are to be called from the erlang shell, config
- data must first be required with
- ct:require/1/2 . This is
+ data must first be required with
+ ct:require/1/2 . This is
equivalent to a require statement in the Test Suite Info
Function or in the
If you wish to exit the interactive mode (e.g. to start an
- automated test run with ct:run_test/1 ), call the function
- ct:stop_interactive/0 . This shuts down the
+ automated test run with ct:run_test/1 ), call the function
+ ct:stop_interactive/0 . This shuts down the
running ct application. Associations between
configuration names and data created with require are
- consequently deleted. ct:start_interactive/0 will get you
+ consequently deleted. ct:start_interactive/0 will get you
back into interactive mode, but the previous state is not restored.
@@ -503,7 +503,7 @@
Step by step execution of test cases with the Erlang Debugger
By means of ct_run -step [opts] , or by passing the
- {step,Opts} option to ct:run_test/1 , it is possible
+ {step,Opts} option to ct:run_test/1 , it is possible
to get the Erlang Debugger started automatically and use its
graphical interface to investigate the state of the current test
case and to execute it step by step and/or set execution breakpoints.
@@ -527,17 +527,17 @@
with dir .
-
+
Test Specifications
General description
The most flexible way to specify what to test, is to use a so
called test specification. A test specification is a sequence of
Erlang terms. The terms are normally declared in one or more text files
- (see ct:run_test/1 ), but
+ (see ct:run_test/1 ), but
may also be passed to Common Test on the form of a list (see
- ct:run_testspec/1 ).
+ ct:run_testspec/1 ).
There are two general types of terms: configuration terms and test
specification terms.
With configuration terms it is possible to e.g. label the test
@@ -989,7 +989,7 @@
ct_run . This forces Common Test to ignore unrecognizable terms.
Note that in this mode, Common Test is not able to check the specification
for errors as efficiently as if the scanner runs in default mode.
- If ct:run_test/1 is used
+ If ct:run_test/1 is used
for starting the tests, the relaxed scanner
mode is enabled by means of the tuple: {allow_user_terms,true}
@@ -999,7 +999,7 @@
Running tests from the Web based GUI
The web based GUI, VTS, is started with the
- ct_run
+ ct_run
program. From the GUI you can load config files, and select
directories, suites and cases to run. You can also state the
config files, directories, suites and cases on the command line
diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml
index cc8d913994..a33b22ac39 100644
--- a/lib/common_test/doc/src/write_test_chapter.xml
+++ b/lib/common_test/doc/src/write_test_chapter.xml
@@ -129,8 +129,8 @@
-
+
Init and end per test case
Each test suite module can contain the optional configuration functions
@@ -173,7 +173,7 @@
The end_per_testcase/2 function is called even after a
- test case terminates due to a call to ct:abort_current_testcase/1 ,
+ test case terminates due to a call to ct:abort_current_testcase/1 ,
or after a timetrap timeout. However, end_per_testcase
will then execute on a different process than the test case
function, and in this situation, end_per_testcase will
@@ -243,8 +243,8 @@
The test case function argument Config should not be
confused with the information that can be retrieved from
- configuration files (using
- ct:get_config/1/2 ). The Config argument
+ configuration files (using
+ ct:get_config/1/2 ). The Config argument
should be used for runtime configuration of the test suite and the
test cases, while configuration files should typically contain data
related to the SUT. These two types of configuration data are handled
@@ -303,7 +303,7 @@
-
Use this to specify arbitrary data related to the testcase. This
- data can be retrieved at any time using the ct:userdata/3
+ data can be retrieved at any time using the ct:userdata/3
utility function.
@@ -348,8 +348,8 @@
See the Config files
- chapter and the
- ct:require/1/2 function in the
+ chapter and the
+ ct:require/1/2 function in the
ct reference manual for more information about
require .
@@ -826,14 +826,16 @@
Common Test to create one dedicated private directory per
test case and execution instead. This is accomplished by means of
the flag/option: create_priv_dir (to be used with the
- ct_run program, the ct:run_test/1 function, or
+ ct_run program, the ct:run_test/1 function, or
as test specification term). There are three possible values
for this option:
+
auto_per_run
auto_per_tc
manual_per_tc
+
The first value indicates the default priv_dir behaviour, i.e.
one private directory created per test run. The two latter
values tell Common Test to generate a unique test directory name
@@ -842,7 +844,7 @@
become very inefficient for test runs with many test cases and/or
repetitions. Therefore, in case the manual version is instead used, the
test case must tell Common Test to create priv_dir when it needs it.
- It does this by calling the function ct:make_priv_dir/0 .
+ It does this by calling the function ct:make_priv_dir/0 .
You should not depend on current working directory for
@@ -890,7 +892,7 @@
It is also possible to dynamically set/reset a timetrap during the
excution of a test case, or configuration function. This is done by calling
- ct:timetrap/1 . This function cancels the current timetrap
+ ct:timetrap/1 . This function cancels the current timetrap
and starts a new one (that stays active until timeout, or end of the
current function).
@@ -903,12 +905,12 @@
If a test case needs to suspend itself for a time that also gets
multipled by multiply_timetraps (and possibly also scaled up if
- scale_timetraps is enabled), the function ct:sleep/1
+ scale_timetraps is enabled), the function ct:sleep/1
may be used (instead of e.g. timer:sleep/1 ).
A function (fun/0 or MFA ) may be specified as
timetrap value in the suite-, group- and test case info function, as
- well as argument to the ct:timetrap/1 function. Examples:
+ well as argument to the ct:timetrap/1 function. Examples:
{timetrap,{my_test_utils,timetrap,[?MODULE,system_start]}}
ct:timetrap(fun() -> my_timetrap(TestCaseName, Config) end)
--
cgit v1.2.3