From 96b3f1cdf435954b826be3da8de714b13910d396 Mon Sep 17 00:00:00 2001 From: Peter Andersson Date: Wed, 28 Mar 2012 14:06:59 +0200 Subject: Update documentation OTP-9972 OTP-9659 OTP-9855 OTP-9880 OTP-9973 OTP-9986 OTP-9992 OTP-9935 --- lib/common_test/doc/src/basics_chapter.xml | 5 +- lib/common_test/doc/src/ct_hooks_chapter.xml | 28 +++++-- lib/common_test/doc/src/ct_run.xml | 3 + .../doc/src/getting_started_chapter.xml | 3 +- lib/common_test/doc/src/run_test_chapter.xml | 12 ++- lib/common_test/doc/src/write_test_chapter.xml | 87 +++++++++++++++++++--- 6 files changed, 118 insertions(+), 20 deletions(-) diff --git a/lib/common_test/doc/src/basics_chapter.xml b/lib/common_test/doc/src/basics_chapter.xml index 20141d2561..d43ac44ac5 100644 --- a/lib/common_test/doc/src/basics_chapter.xml +++ b/lib/common_test/doc/src/basics_chapter.xml @@ -141,8 +141,9 @@ individual test case.

- The test suite module must conform to a callback interface specified - by the CT test server. See the + The test suite module must conform to a + callback interface + specified by the CT test server. See the Writing Test Suites chapter for more information.

diff --git a/lib/common_test/doc/src/ct_hooks_chapter.xml b/lib/common_test/doc/src/ct_hooks_chapter.xml index c5b4fd0073..cb06e570bd 100644 --- a/lib/common_test/doc/src/ct_hooks_chapter.xml +++ b/lib/common_test/doc/src/ct_hooks_chapter.xml @@ -189,6 +189,22 @@ it.

+
+ External configuration data and Logging +

It's possible in the CTH to read configuration data values + by calling ct:get_config/1/2/3 (as explained in the + + External configuration data + chapter). The config variables in question must, as always, first have been + required by means of a suite-, group-, or test case info function, + or the ct:require/1/2 function. Note that the latter can also be used + in CT hook functions.

+

The CT hook functions may call any of the logging functions available + in the ct interface to print information to the log files, or to + add comments in the suite overview page. +

+
+ @@ -201,11 +217,13 @@ functions for a CTH follow a common interface, this interface is described below.

-

It is only possible to hook into test function which exists in the test - suite. So in order for a CTH to hook in before - init_per_suite, - the init_per_suite - function must exist in the test suite.

+

Common Test will always call all available hook functions, even pre- and post + hooks for configuration functions that are not implemented in the suite. + For example, pre_init_per_suite(x_SUITE, ...) and + post_init_per_suite(x_SUITE, ...) will be called for test suite + x_SUITE, even if it doesn't export init_per_suite/1. This feature + makes it possible to use hooks as configuration fallbacks, or even + completely replace all configuration functions with hook functions.

diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index b01ab3667d..4d4521957c 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -103,6 +103,7 @@ [-no_auto_compile] [-muliply_timetraps Multiplier] [-scale_timetraps] + [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] [-repeat N [-force_stop]] | [-duration HHMMSS [-force_stop]] | [-until [YYMoMoDD]HHMMSS [-force_stop]] @@ -130,6 +131,7 @@ [-no_auto_compile] [-muliply_timetraps Multiplier] [-scale_timetraps] + [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] [-repeat N [-force_stop]] | [-duration HHMMSS [-force_stop]] | [-until [YYMoMoDD]HHMMSS [-force_stop]] @@ -150,6 +152,7 @@ [-no_auto_compile] [-muliply_timetraps Multiplier] [-scale_timetraps] + [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] [-basic_html]
diff --git a/lib/common_test/doc/src/getting_started_chapter.xml b/lib/common_test/doc/src/getting_started_chapter.xml index 7de0912036..05e21241cb 100644 --- a/lib/common_test/doc/src/getting_started_chapter.xml +++ b/lib/common_test/doc/src/getting_started_chapter.xml @@ -98,7 +98,8 @@
A simple test suite

As you've seen in the basics chapter, the test suite module implements - callback functions (mandatory or optional) for various purposes, e.g: + callback functions + (mandatory or optional) for various purposes, e.g: Init/end configuration function for the test suite Init/end configuration function for a test case diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index 848f278fa6..b3ec524b21 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -466,6 +466,9 @@ {logdir, LogDir}. {logdir, NodeRefs, LogDir}. + + {create_priv_dir, PrivDirOption}. + {create_priv_dir, NodeRefs, PrivDirOption}. {event_handler, EventHandlers}. {event_handler, NodeRefs, EventHandlers}. @@ -493,6 +496,9 @@ {skip_suites, DirRef, Suites, Comment}. {skip_suites, NodeRefs, DirRef, Suites, Comment}. + + {skip_groups, DirRef, Suite, GroupNames, Comment}. + {skip_groups, NodeRefs, DirRef, Suite, GroupNames, Comment}. {skip_cases, DirRef, Suite, Cases, Comment}. {skip_cases, NodeRefs, DirRef, Suite, Cases, Comment}. @@ -512,6 +518,7 @@ DirAlias = atom() Dir = string() LogDir = string() + PrivDirOption = auto_per_run | auto_per_tc | manual_per_tc EventHandlers = atom() | [atom()] InitArgs = [term()] CTHModules = [CTHModule | {CTHModule, CTHInitArgs} | {CTHModule, CTHInitArgs, CTHPriority}] @@ -521,8 +528,9 @@ Suites = atom() | [atom()] | all Suite = atom() Groups = GroupSpec | [GroupSpec] | all - GroupSpec = Group | {Group,Properties} | {Group,Properties,GroupSpec} - Group = atom() + GroupSpec = GroupName | {GroupName,Properties} | {GroupName,Properties,GroupSpec} + GroupName = atom() + GroupNames = GroupName | [GroupName] Cases = atom() | [atom()] | all Comment = string() | "" diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml index c0ec26ddcc..09f4f8ed3b 100644 --- a/lib/common_test/doc/src/write_test_chapter.xml +++ b/lib/common_test/doc/src/write_test_chapter.xml @@ -29,7 +29,6 @@ write_test_chapter.xml -

Support for test suite authors @@ -63,7 +62,8 @@ function in CT will not be able to locate it (at least not per default).

-

The ct.hrl header file must be included in all test suite files. +

It is also recommended that the ct.hrl header file is included + in all test suite modules.

Each test suite module must export the function all/0 @@ -71,6 +71,12 @@ to be executed in that module.

+

The callback functions that the test suite should implement, and + which will be described in more detail below, are + all listed in the common_test + reference manual page. +

+
@@ -113,6 +119,14 @@ suite will be skipped automatically (so called auto skipped), including end_per_suite.

+ +

Note that if init_per_suite and end_per_suite do not exist + in the suite, Common Test calls dummy functions (with the same names) + instead, so that output generated by hook functions may be saved to the log + files for these dummies + (see the Common Test Hooks + chapter for more information). +

@@ -513,8 +527,7 @@ execution is immediately stopped and the rest of the cases skipped.

Before execution of a group begins, the configuration function - init_per_group(GroupName, Config) is called (the function is - mandatory if one or more test case groups are defined). The list of tuples + init_per_group(GroupName, Config) is called. The list of tuples returned from this function is passed to the test cases in the usual manner by means of the Config argument. init_per_group/2 is meant to be used for initializations common for the test cases in the @@ -522,6 +535,14 @@ end_per_group(GroupName, Config function is called. This function is meant to be used for cleaning up after init_per_group/2.

+

Whenever a group is executed, if init_per_group and + end_per_group do not exist in the suite, Common Test calls + dummy functions (with the same names) instead. Output generated by + hook functions will be saved to the log files for these dummies + (see the Common Test + Hooks chapter for more information). +

+

init_per_testcase/2 and end_per_testcase/2 are always called for each individual test case, no matter if the case belongs to a group or not.

 @@ -610,6 +631,25 @@ in the previous group get spawned.

+
+ Parallel test cases and IO +

A parallel test case has a private IO server as its group leader. + (Please see the Erlang Run-Time System Application documentation for + a description of the group leader concept). The + central IO server process that handles the output from regular test + cases and configuration functions, does not respond to IO messages + during execution of parallel groups. This is important to understand + in order to avoid certain traps, like this one:

+

If a process, P, is spawned during execution of e.g. + init_per_suite/1, it will inherit the group leader of the + init_per_suite process. This group leader is the central IO server + process mentioned above. If, at a later time, during parallel test case + execution, some event triggers process P to call + io:format/1/2, that call will never return (since the group leader + is in a non-responsive state) and cause P to hang. +

+
+
Repeated groups @@ -745,7 +785,7 @@ Data and Private Directories -

The data directory (data_dir) is the directory where the +

The data directory, data_dir, is the directory where the test module has its own files needed for the testing. The name of the data_dir is the the name of the test suite followed by "_data". For example, @@ -769,12 +809,39 @@

-->

- The priv_dir is the test suite's private directory. This - directory should be used when a test case needs to write to - files. The name of the private directory is generated by the test - server, which also creates the directory. + priv_dir is the private directory for the test cases. + This directory may be used whenever a test case (or configuration function) + needs to write something to file. The name of the private directory is + generated by Common Test, which also creates the directory.

- +

By default, Common Test creates one central private directory + per test run that all test cases share. This may not always be suitable, + especially if the same test cases are executed multiple times during + a test run (e.g. if they belong to a test case group with repeat + property), and there's a risk that files in the private directory get + overwritten. Under these circumstances, it's possible to configure + Common Test to create one dedicated private directory per + test case and execution instead. This is accomplished by means of + the flag/option: create_priv_dir (to be used with the + ct_run program, the ct:run_test/1 function, or + as test specification term). There are three possible values + for this option: + + auto_per_run + auto_per_tc + manual_per_tc + + The first value indicates the default priv_dir behaviour, i.e. + one private directory created per test run. The two latter + values tell Common Test to generate a unique test directory name + per test case and execution. If the auto version is used, all + private directories will be created automatically. This can obviously + become very inefficient for test runs with many test cases and/or + repetitions. Therefore, in case the manual version is instead used, the + test case must tell Common Test to create priv_dir when it needs it. + It does this by calling the function ct:make_priv_dir/0. +

+

You should not depend on current working directory for reading and writing data files since this is not portable. All scratch files are to be written in the priv_dir and all -- cgit v1.2.3