diff options
Diffstat (limited to 'lib/common_test/doc')
| -rw-r--r-- | lib/common_test/doc/src/common_test_app.xml | 8 | ||||
| -rw-r--r-- | lib/common_test/doc/src/ct_master_chapter.xml | 2 | ||||
| -rw-r--r-- | lib/common_test/doc/src/ct_run.xml | 169 | ||||
| -rw-r--r-- | lib/common_test/doc/src/example_chapter.xml | 16 | ||||
| -rw-r--r-- | lib/common_test/doc/src/install_chapter.xml | 63 | ||||
| -rw-r--r-- | lib/common_test/doc/src/run_test_chapter.xml | 33 |
6 files changed, 134 insertions, 157 deletions
diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml index ea4522c40b..f63f9581a6 100644 --- a/lib/common_test/doc/src/common_test_app.xml +++ b/lib/common_test/doc/src/common_test_app.xml @@ -252,7 +252,7 @@ </func> <func> - <name>Module:end_per_suite(Config) -> void() | + <name>Module:end_per_suite(Config) -> term() | {save_config,SaveConfig}</name> <fsummary>Test suite finalization. </fsummary> <type> @@ -385,7 +385,7 @@ </func> <func> - <name>Module:end_per_group(GroupName, Config) -> void() | + <name>Module:end_per_group(GroupName, Config) -> term() | {return_group_result,Status}</name> <fsummary>Test case group finalization.</fsummary> <type> @@ -440,7 +440,7 @@ </func> <func> - <name>Module:end_per_testcase(TestCase, Config) -> void() | {fail,Reason} | {save_config,SaveConfig}</name> + <name>Module:end_per_testcase(TestCase, Config) -> term() | {fail,Reason} | {save_config,SaveConfig}</name> <fsummary>Test case finalization.</fsummary> <type> <v> TestCase = atom()</v> @@ -538,7 +538,7 @@ <func> - <name>Module:Testcase(Config) -> void() | {skip,Reason} | {comment,Comment} | {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit() </name> + <name>Module:Testcase(Config) -> term() | {skip,Reason} | {comment,Comment} | {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit() </name> <fsummary>A test case</fsummary> <type> <v> Config = SaveConfig = [{Key,Value}]</v> diff --git a/lib/common_test/doc/src/ct_master_chapter.xml b/lib/common_test/doc/src/ct_master_chapter.xml index 37a0805055..adfe79e41a 100644 --- a/lib/common_test/doc/src/ct_master_chapter.xml +++ b/lib/common_test/doc/src/ct_master_chapter.xml @@ -198,7 +198,7 @@ <section> <title>Automatic startup of test target nodes</title> <marker id="ct_slave"></marker> - <p>Is is possible to automatically start, and perform initial actions, on + <p>It is possible to automatically start, and perform initial actions, on test target nodes by using the test specification term <c>init</c>.</p> <p>Currently, two sub-terms are supported, <c>node_start</c> and <c>eval</c>.</p> <p>Example:</p> diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index d8e79ca80e..3ac8691fb5 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -86,98 +86,99 @@ <marker id="ct_run"></marker> <title>Run tests from command line</title> <pre> - ct_run [-dir TestDir1 TestDir2 .. TestDirN] | - [[-dir TestDir] -suite Suite1 Suite2 .. SuiteN - [[-group Groups1 Groups2 .. GroupsN] [-case Case1 Case2 .. CaseN]]] - [-step [config | keep_inactive]] - [-config ConfigFile1 ConfigFile2 .. ConfigFileN] - [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 - ConfigString2 and .. CallbackModuleN ConfigStringN] - [-decrypt_key Key] | [-decrypt_file KeyFile] - [-label Label] - [-logdir LogDir] - [-logopts LogOpts] - [-verbosity GenVLevel | [Category1 VLevel1 and - Category2 VLevel2 and .. CategoryN VLevelN]] - [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] - [-stylesheet CSSFile] - [-cover CoverCfgFile] - [-cover_stop Bool] - [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] | - [-event_handler_init EvHandler1 InitArg1 and - EvHandler2 InitArg2 and .. EvHandlerN InitArgN] - [-include InclDir1 InclDir2 .. InclDirN] - [-no_auto_compile] - [-abort_if_missing_suites] - [-muliply_timetraps Multiplier] - [-scale_timetraps] - [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] - [-repeat N] | - [-duration HHMMSS [-force_stop [skip_rest]]] | - [-until [YYMoMoDD]HHMMSS [-force_stop [skip_rest]]] - [-basic_html] - [-ct_hooks CTHModule1 CTHOpts1 and CTHModule2 CTHOpts2 and .. - CTHModuleN CTHOptsN] - [-exit_status ignore_config] + ct_run -dir TestDir1 TestDir2 .. TestDirN | + [-dir TestDir] -suite Suite1 Suite2 .. SuiteN + [-group Groups1 Groups2 .. GroupsN] [-case Case1 Case2 .. CaseN] + [-step [config | keep_inactive]] + [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. CallbackModuleN ConfigStringN] + [-decrypt_key Key] | [-decrypt_file KeyFile] + [-label Label] + [-logdir LogDir] + [-logopts LogOpts] + [-verbosity GenVLevel | [Category1 VLevel1 and + Category2 VLevel2 and .. CategoryN VLevelN]] + [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] + [-stylesheet CSSFile] + [-cover CoverCfgFile] + [-cover_stop Bool] + [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] | + [-event_handler_init EvHandler1 InitArg1 and + EvHandler2 InitArg2 and .. EvHandlerN InitArgN] + [-include InclDir1 InclDir2 .. InclDirN] + [-no_auto_compile] + [-abort_if_missing_suites] + [-muliply_timetraps Multiplier] + [-scale_timetraps] + [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] + [-repeat N] | + [-duration HHMMSS [-force_stop [skip_rest]]] | + [-until [YYMoMoDD]HHMMSS [-force_stop [skip_rest]]] + [-basic_html] + [-ct_hooks CTHModule1 CTHOpts1 and CTHModule2 CTHOpts2 and .. + CTHModuleN CTHOptsN] + [-exit_status ignore_config] + [-help] </pre> </section> <section> <title>Run tests using test specification</title> <pre> ct_run -spec TestSpec1 TestSpec2 .. TestSpecN - [-join_specs] - [-config ConfigFile1 ConfigFile2 .. ConfigFileN] - [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 - ConfigString2 and .. and CallbackModuleN ConfigStringN] - [-decrypt_key Key] | [-decrypt_file KeyFile] - [-label Label] - [-logdir LogDir] - [-logopts LogOpts] - [-verbosity GenVLevel | [Category1 VLevel1 and - Category2 VLevel2 and .. CategoryN VLevelN]] - [-allow_user_terms] - [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] - [-stylesheet CSSFile] - [-cover CoverCfgFile] - [-cover_stop Bool] - [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] | - [-event_handler_init EvHandler1 InitArg1 and - EvHandler2 InitArg2 and .. EvHandlerN InitArgN] - [-include InclDir1 InclDir2 .. InclDirN] - [-no_auto_compile] - [-abort_if_missing_suites] - [-muliply_timetraps Multiplier] - [-scale_timetraps] - [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] - [-repeat N] | - [-duration HHMMSS [-force_stop [skip_rest]]] | - [-until [YYMoMoDD]HHMMSS [-force_stop [skip_rest]]] - [-basic_html] - [-ct_hooks CTHModule1 CTHOpts1 and CTHModule2 CTHOpts2 and .. - CTHModuleN CTHOptsN] - [-exit_status ignore_config] + [-join_specs] + [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] + [-decrypt_key Key] | [-decrypt_file KeyFile] + [-label Label] + [-logdir LogDir] + [-logopts LogOpts] + [-verbosity GenVLevel | [Category1 VLevel1 and + Category2 VLevel2 and .. CategoryN VLevelN]] + [-allow_user_terms] + [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] + [-stylesheet CSSFile] + [-cover CoverCfgFile] + [-cover_stop Bool] + [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] | + [-event_handler_init EvHandler1 InitArg1 and + EvHandler2 InitArg2 and .. EvHandlerN InitArgN] + [-include InclDir1 InclDir2 .. InclDirN] + [-no_auto_compile] + [-abort_if_missing_suites] + [-muliply_timetraps Multiplier] + [-scale_timetraps] + [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] + [-repeat N] | + [-duration HHMMSS [-force_stop [skip_rest]]] | + [-until [YYMoMoDD]HHMMSS [-force_stop [skip_rest]]] + [-basic_html] + [-ct_hooks CTHModule1 CTHOpts1 and CTHModule2 CTHOpts2 and .. + CTHModuleN CTHOptsN] + [-exit_status ignore_config] </pre> </section> <section> <title>Run tests in web based GUI</title> <pre> ct_run -vts [-browser Browser] - [-dir TestDir1 TestDir2 .. TestDirN] | - [[dir TestDir] -suite Suite [[-group Group] [-case Case]]] - [-config ConfigFile1 ConfigFile2 .. ConfigFileN] - [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 - ConfigString2 and .. and CallbackModuleN ConfigStringN] - [-logopts LogOpts] - [-verbosity GenVLevel | [Category1 VLevel1 and - Category2 VLevel2 and .. CategoryN VLevelN]] - [-decrypt_key Key] | [-decrypt_file KeyFile] - [-include InclDir1 InclDir2 .. InclDirN] - [-no_auto_compile] - [-abort_if_missing_suites] - [-muliply_timetraps Multiplier] - [-scale_timetraps] - [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] - [-basic_html]</pre> + [-dir TestDir1 TestDir2 .. TestDirN] | + [[dir TestDir] -suite Suite [[-group Group] [-case Case]]] + [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] + [-logopts LogOpts] + [-verbosity GenVLevel | [Category1 VLevel1 and + Category2 VLevel2 and .. CategoryN VLevelN]] + [-decrypt_key Key] | [-decrypt_file KeyFile] + [-include InclDir1 InclDir2 .. InclDirN] + [-no_auto_compile] + [-abort_if_missing_suites] + [-muliply_timetraps Multiplier] + [-scale_timetraps] + [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] + [-basic_html]</pre> </section> <section> <title>Refresh the HTML index files</title> @@ -188,10 +189,10 @@ <title>Run CT in interactive mode</title> <pre> ct_run -shell - [-config ConfigFile1 ConfigFile2 ... ConfigFileN] - [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 - ConfigString2 and .. and CallbackModuleN ConfigStringN] - [-decrypt_key Key] | [-decrypt_file KeyFile]</pre> + [-config ConfigFile1 ConfigFile2 ... ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] + [-decrypt_key Key] | [-decrypt_file KeyFile]</pre> </section> <section> <title>Start a Common Test Master node</title> diff --git a/lib/common_test/doc/src/example_chapter.xml b/lib/common_test/doc/src/example_chapter.xml index 36781d152c..2bc8cfdbcc 100644 --- a/lib/common_test/doc/src/example_chapter.xml +++ b/lib/common_test/doc/src/example_chapter.xml @@ -81,7 +81,7 @@ init_per_suite(Config) -> [{con_ref, Ref },{table_name, TableName}| Config]. %%-------------------------------------------------------------------- -%% Function: end_per_suite(Config) -> void() +%% Function: end_per_suite(Config) -> term() %% %% Config = [tuple()] %% A list of key/value pairs, holding the test case configuration. @@ -110,7 +110,7 @@ init_per_testcase(Case, Config) -> Config. %%-------------------------------------------------------------------- -%% Function: end_per_testcase(TestCase, Config) -> void() +%% Function: end_per_testcase(TestCase, Config) -> term() %% %% TestCase = atom() %% Name of the test case that is finished. @@ -226,7 +226,7 @@ init_per_suite(Config) -> Config. %%-------------------------------------------------------------------- -%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% Function: end_per_suite(Config0) -> term() | {save_config,Config1} %% %% Config0 = Config1 = [tuple()] %% A list of key/value pairs, holding the test case configuration. @@ -254,7 +254,7 @@ init_per_group(_GroupName, Config) -> %%-------------------------------------------------------------------- %% Function: end_per_group(GroupName, Config0) -> -%% void() | {save_config,Config1} +%% term() | {save_config,Config1} %% %% GroupName = atom() %% Name of the test case group that is finished. @@ -287,7 +287,7 @@ init_per_testcase(_TestCase, Config) -> %%-------------------------------------------------------------------- %% Function: end_per_testcase(TestCase, Config0) -> -%% void() | {save_config,Config1} | {fail,Reason} +%% term() | {save_config,Config1} | {fail,Reason} %% %% TestCase = atom() %% Name of the test case that is finished. @@ -414,7 +414,7 @@ init_per_suite(Config) -> Config. %%-------------------------------------------------------------------- -%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% Function: end_per_suite(Config0) -> term() | {save_config,Config1} %% Config0 = Config1 = [tuple()] %%-------------------------------------------------------------------- end_per_suite(_Config) -> @@ -432,7 +432,7 @@ init_per_group(_GroupName, Config) -> %%-------------------------------------------------------------------- %% Function: end_per_group(GroupName, Config0) -> -%% void() | {save_config,Config1} +%% term() | {save_config,Config1} %% GroupName = atom() %% Config0 = Config1 = [tuple()] %%-------------------------------------------------------------------- @@ -451,7 +451,7 @@ init_per_testcase(_TestCase, Config) -> %%-------------------------------------------------------------------- %% Function: end_per_testcase(TestCase, Config0) -> -%% void() | {save_config,Config1} | {fail,Reason} +%% term() | {save_config,Config1} | {fail,Reason} %% TestCase = atom() %% Config0 = Config1 = [tuple()] %% Reason = term() diff --git a/lib/common_test/doc/src/install_chapter.xml b/lib/common_test/doc/src/install_chapter.xml index 7f8c606324..31125b945c 100644 --- a/lib/common_test/doc/src/install_chapter.xml +++ b/lib/common_test/doc/src/install_chapter.xml @@ -34,7 +34,7 @@ <title>General information</title> <p>The two main interfaces for running tests with Common Test - are an executable program named ct_run and an + are an executable program named <c>ct_run</c> and an erlang module named <c>ct</c>. The ct_run program is compiled for the underlying operating system (e.g. Unix/Linux or Windows) during the build of the Erlang/OTP system, and is @@ -43,67 +43,10 @@ The <c>ct</c> interface functions can be called from the Erlang shell, or from any Erlang function, on any supported platform.</p> - <p>A legacy Bourne shell script - named run_test - exists, - which may be manually generated and installed. This script may be used - instead of the ct_run program mentioned above, e.g. if the user - wishes to modify or customize the Common Test start flags in a simpler - way than making changes to the ct_run C program.</p> - <p>The Common Test application is installed with the Erlang/OTP system and no additional installation step is required to start using - Common Test by means of the ct_run executable program, and/or the interface - functions in the <c>ct</c> module. If you wish to use the legacy Bourne - shell script version run_test, however, this script needs to be - generated first, according to the instructions below.</p> - - <note><p>Before reading on, please note that since Common Test version - 1.5, the run_test shell script is no longer required for starting - tests with Common Test from the OS command line. The ct_run - program (descibed above) is the new recommended command line interface - for Common Test. The shell script exists mainly for legacy reasons and - may not be updated in future releases of Common Test. It may even be removed. - </p></note> - - <p>Optional step to generate a shell script for starting Common Test:</p> - <p>To generate the run_test shell script, navigate to the - <c><![CDATA[common_test-<vsn>]]></c> directory, located among the other - OTP applications (under the OTP lib directory). Here execute the - <c>install.sh</c> script with argument <c>local</c>:</p> - - <p><c> - $ ./install.sh local - </c></p> - - <p>This generates the executable run_test script in the - <c><![CDATA[common_test-<vsn>/priv/bin]]></c> directory. The script - will include absolute paths to the Common Test and Test Server - application directories, so it's possible to copy or move the script to - a different location on the file system, if desired, without having to - update it. It's of course possible to leave the script under the - <c>priv/bin</c> directory and update the PATH variable accordingly (or - create a link or alias to it).</p> - - <p>If you, for any reason, have copied Common Test and Test Server - to a different location than the default OTP lib directory, you can - generate a run_test script with a different top level directory, - simply by specifying the directory, instead of <c>local</c>, when running - <c>install.sh</c>. Example:</p> - - <p><c> - $ install.sh /usr/local/test_tools - </c></p> - - <p>Note that the <c><![CDATA[common_test-<vsn>]]></c> and - <c><![CDATA[test_server-<vsn>]]></c> directories must be located under the - same top directory. Note also that the install script does not copy files - or update environment variables. It only generates the run_test - script.</p> - - <p>Whenever you install a new version of Erlang/OTP, the run_test - script needs to be regenerated, or updated manually with new directory names - (new version numbers), for it to "see" the latest Common Test and Test Server - versions.</p> - + Common Test by means of the <c>ct_run</c> executable program, and/or + the interface functions in the <c>ct</c> module.</p> </section> </chapter> diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index 864f82cb63..fc8d82c2c3 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -149,6 +149,7 @@ <p>Other flags that may be used with <c>ct_run</c>:</p> <list> + <item><c><![CDATA[-help]]></c>, lists all available start flags.</item> <item><c><![CDATA[-logdir <dir>]]></c>, specifies where the HTML log files are to be written.</item> <item><c><![CDATA[-label <name_of_test_run>]]></c>, associates the test run with a name that gets printed in the overview HTML log files.</item> @@ -223,6 +224,9 @@ behaviour using start flag:</p> <pre>-exit_status ignore_config</pre> + <note><p>Executing <c>ct_run</c> without start flags, is equal to the command: + <c>ct_run -dir ./</c></p></note> + <p>For more information about the <c>ct_run</c> program, see the <seealso marker="ct_run">Reference Manual</seealso> and the <seealso marker="install_chapter#general">Installation</seealso> chapter. @@ -251,6 +255,10 @@ <c>{error,Reason}</c>, where the term <c>Reason</c> explains the failure.</p> + <p>The default start option <c>{dir,Cwd}</c> (run all suites in the current + working directory) is used if the function is called with an empty + list of options.</p> + <section> <title>Releasing the Erlang shell</title> <p>During execution of tests, started with @@ -1005,6 +1013,31 @@ for starting the tests, the relaxed scanner mode is enabled by means of the tuple: <c>{allow_user_terms,true}</c></p> </section> + <section> + <title>Reading test specification terms</title> + <p>It's possible to look up terms in the current test specification + (i.e. the spec that's been used to configure and run the current test). + The function <c>get_testspec_terms()</c> returns a list of all test spec + terms (both config- and test terms) and <c>get_testspec_terms(Tags)</c> + returns the term (or a list of terms) matching the tag (or tags) in + <c>Tags</c>.</p> + <p>For example, in the test specification:</p> + <pre> + ... + {label, my_server_smoke_test}. + {config, "../../my_server_setup.cfg"}. + {config, "../../my_server_interface.cfg"}. + ...</pre> + <p>And in e.g. a test suite or a CT hook function:</p> + <pre> + ... + [{label,[{_Node,TestType}]}, {config,CfgFiles}] = + ct:get_testspec_terms([label,config]), + + [verify_my_server_cfg(TestType, CfgFile) || {Node,CfgFile} <- CfgFiles, + Node == node()]; + ...</pre> + </section> </section> <section> |