diff options
author | Lukas Larsson <[email protected]> | 2011-10-04 10:16:09 +0200 |
---|---|---|
committer | Lukas Larsson <[email protected]> | 2011-10-04 10:16:09 +0200 |
commit | 2d25f96f904bfa33c4bba4f3026f9fd2117ed8ab (patch) | |
tree | 26acbda51fa2ac4dd5f809e400d05aa1021eba20 /lib/common_test/doc | |
parent | 622daf3b96666f8bbabec44a15b26a188a83b95e (diff) | |
parent | 1a42cde6d62a8ec0f9e22215d3c3ec655201c78f (diff) | |
download | otp-2d25f96f904bfa33c4bba4f3026f9fd2117ed8ab.tar.gz otp-2d25f96f904bfa33c4bba4f3026f9fd2117ed8ab.tar.bz2 otp-2d25f96f904bfa33c4bba4f3026f9fd2117ed8ab.zip |
Merge branch 'dev' into major
* dev: (38 commits)
Update documentation
Rid ct_telnet of doc build warnings
Create temporary fix for problem with parallel test cases
Update primary bootstrap
Correct "Missing Suites" link
Add documentation on timetraps and start flags
Add missing tests for timetrap handling and fix remaining errors
Solve problem with ct_init/end_per_group being counted as test cases
Fix errors in test suites
Fix invalid call to undefined function
Fix problem with test_server_ctrl creating invalid conf test
Improve info in CT framework log
Update vsn.mk for common_test and test_server
Enhance logging performance
Change order of include files
Add link to last executed test suite on index page
Fix problem with location value when init config func calls help func
Fix crash when CTHook init fails
Correct error in test suite
Fix error with incorrect notification after end_per_testcase craches
...
Conflicts:
bootstrap/bin/start.boot
bootstrap/bin/start_clean.boot
bootstrap/lib/compiler/ebin/beam_asm.beam
bootstrap/lib/compiler/ebin/beam_disasm.beam
bootstrap/lib/compiler/ebin/compile.beam
bootstrap/lib/compiler/ebin/sys_pre_expand.beam
bootstrap/lib/kernel/ebin/code.beam
bootstrap/lib/kernel/ebin/code_server.beam
bootstrap/lib/kernel/ebin/hipe_unified_loader.beam
bootstrap/lib/kernel/ebin/inet.beam
bootstrap/lib/kernel/ebin/inet_config.beam
bootstrap/lib/kernel/ebin/inet_dns.beam
bootstrap/lib/stdlib/ebin/beam_lib.beam
bootstrap/lib/stdlib/ebin/dets.beam
bootstrap/lib/stdlib/ebin/erl_compile.beam
bootstrap/lib/stdlib/ebin/erl_internal.beam
bootstrap/lib/stdlib/ebin/erl_scan.beam
bootstrap/lib/stdlib/ebin/erl_tar.beam
bootstrap/lib/stdlib/ebin/io_lib_fread.beam
bootstrap/lib/stdlib/ebin/otp_internal.beam
bootstrap/lib/stdlib/ebin/sofs.beam
bootstrap/lib/stdlib/ebin/supervisor.beam
bootstrap/lib/stdlib/ebin/zip.beam
lib/common_test/src/ct.erl
lib/common_test/src/ct_run.erl
lib/common_test/test/ct_error_SUITE.erl
lib/common_test/test/ct_repeat_1_SUITE.erl
lib/common_test/test/ct_skip_SUITE.erl
lib/test_server/src/test_server.erl
Diffstat (limited to 'lib/common_test/doc')
-rw-r--r-- | lib/common_test/doc/src/common_test_app.xml | 32 | ||||
-rw-r--r-- | lib/common_test/doc/src/config_file_chapter.xml | 6 | ||||
-rw-r--r-- | lib/common_test/doc/src/ct_run.xml | 7 | ||||
-rw-r--r-- | lib/common_test/doc/src/run_test_chapter.xml | 40 | ||||
-rw-r--r-- | lib/common_test/doc/src/write_test_chapter.xml | 14 |
5 files changed, 80 insertions, 19 deletions
diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml index 57b032b3fd..f58b2ab0a9 100644 --- a/lib/common_test/doc/src/common_test_app.xml +++ b/lib/common_test/doc/src/common_test_app.xml @@ -133,9 +133,15 @@ {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} | {ct_hooks, CTHs}</v> - <v> Time = MilliSec | {seconds,integer()} | {minutes,integer()} - | {hours,integer()}</v> + <v> Time = TimeVal | TimeFunc</v> + <v> TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | + {hours,integer()}</v> + <v> TimeFunc = {Mod,Func,Args} | Fun</v> <v> MilliSec = integer()</v> + <v> Mod = atom()</v> + <v> Func = atom()</v> + <v> Args = list()</v> + <v> Fun = fun()</v> <v> Required = Key | {Key,SubKeys}</v> <v> Key = atom()</v> <v> SubKeys = SubKey | [SubKey]</v> @@ -161,7 +167,9 @@ test case is allowed to take (including <c>init_per_testcase/2</c> and <c>end_per_testcase/2</c>). If the timetrap time is exceeded, the test case fails with reason - <c>timetrap_timeout</c>.</p> + <c>timetrap_timeout</c>. If a <c>TimeFunc</c> function is specified, + it will be called initially and must return a value on + <c>TimeVal</c> format.</p> <p>The <c>require</c> tag specifies configuration variables that are required by test cases in the suite. If the required @@ -248,7 +256,7 @@ </type> <desc> - <p> MANDATORY (only if one or more groups are defined) </p> + <p> OPTIONAL </p> <p>This function is called before execution of a test case group. It typically contains initialization which is common for @@ -279,7 +287,7 @@ </type> <desc> - <p> MANDATORY (only if one or more groups are defined) </p> + <p> OPTIONAL </p> <p>This function is called after the execution of a test case group is finished. It is meant to be used for cleaning up after <c>init_per_group/2</c>. @@ -353,9 +361,15 @@ <v> Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns}</v> - <v> Time = MilliSec | {seconds,integer()} | {minutes,integer()} - | {hours,integer()}</v> + <v> Time = TimeVal | TimeFunc</v> + <v> TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | + {hours,integer()}</v> + <v> TimeFunc = {Mod,Func,Args} | Fun</v> <v> MilliSec = integer()</v> + <v> Mod = atom()</v> + <v> Func = atom()</v> + <v> Args = list()</v> + <v> Fun = fun()</v> <v> Required = Key | {Key,SubKeys}</v> <v> Key = atom()</v> <v> SubKeys = SubKey | [SubKey]</v> @@ -378,7 +392,9 @@ exceeded, the test case fails with reason <c>timetrap_timeout</c>. <c>init_per_testcase/2</c> and <c>end_per_testcase/2</c> are included in the - timetrap time.</p> + timetrap time. If a <c>TimeFunc</c> function is specified, + it will be called before the test case (or <c>init_per_testcase/2</c>) + and must return a value on <c>TimeVal</c> format.</p> <p>The <c>require</c> tag specifies configuration variables that are required by the test case. If the required diff --git a/lib/common_test/doc/src/config_file_chapter.xml b/lib/common_test/doc/src/config_file_chapter.xml index 59151a73ec..6fc6638bf7 100644 --- a/lib/common_test/doc/src/config_file_chapter.xml +++ b/lib/common_test/doc/src/config_file_chapter.xml @@ -285,7 +285,7 @@ <c>{ok, Config}</c> - if the configuration variables are read successfully, </item> <item> - <c>{error, Error, ErrorDetails}</c> - if the callback module fails to + <c>{error, {Error, ErrorDetails}}</c> - if the callback module fails to proceed with the given configuration parameters. </item> </list> @@ -422,14 +422,14 @@ stop()-> call(Client, Request)-> case whereis(?REGISTERED_NAME) of undefined-> - {error, not_started, Request}; + {error, {not_started, Request}}; Pid-> Pid ! {Client, Request}, receive Reply-> {ok, Reply} after 4000-> - {error, timeout, Request} + {error, {timeout, Request}} end end. diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index 1ab563d74f..9045646733 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -83,7 +83,7 @@ <title>Run tests from command line</title> <pre> ct_run [-dir TestDir1 TestDir2 .. TestDirN] | - [-suite Suite1 Suite2 .. SuiteN + [[-dir TestDir] -suite Suite1 Suite2 .. SuiteN [[-group Group1 Group2 .. GroupN] [-case Case1 Case2 .. CaseN]]] [-step [config | keep_inactive]] [-config ConfigFile1 ConfigFile2 .. ConfigFileN] @@ -92,6 +92,7 @@ [-decrypt_key Key] | [-decrypt_file KeyFile] [-label Label] [-logdir LogDir] + [-logopts LogOpts] [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] [-stylesheet CSSFile] [-cover CoverCfgFile] @@ -117,6 +118,7 @@ [-decrypt_key Key] | [-decrypt_file KeyFile] [-label Label] [-logdir LogDir] + [-logopts LogOpts] [-allow_user_terms] [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] [-stylesheet CSSFile] @@ -138,10 +140,11 @@ <pre> ct_run -vts [-browser Browser] [-dir TestDir1 TestDir2 .. TestDirN] | - [-suite Suite [[-group Group] [-case Case]]] + [[dir TestDir] -suite Suite [[-group Group] [-case Case]]] [-config ConfigFile1 ConfigFile2 .. ConfigFileN] [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 ConfigString2 and .. and CallbackModuleN ConfigStringN] + [-logopts LogOpts] [-decrypt_key Key] | [-decrypt_file KeyFile] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index 816aa5b1eb..57059f0ba2 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -128,6 +128,15 @@ <p><c>$ ct_run -suite $SYS1_TEST/setup_SUITE -case start stop</c></p> <p><c>$ ct_run -suite $SYS1_TEST/setup_SUITE -group installation -case start stop</c></p> + <p>It is also possible to combine the <c>dir</c>, <c>suite</c> and <c>group/case</c> flags. E.g, to run + <c>x_SUITE</c> and <c>y_SUITE</c> in directory <c>testdir</c>:</p> + + <p><c>$ ct_run -dir ./testdir -suite x_SUITE y_SUITE</c></p> + + <p>This has the same effect as calling:</p> + + <p><c>$ ct_run -suite ./testdir/x_SUITE ./testdir/y_SUITE</c></p> + <p>Other flags that may be used with <c>ct_run</c>:</p> <list> <item><c><![CDATA[-logdir <dir>]]></c>, specifies where the HTML log files are to be written.</item> @@ -167,6 +176,8 @@ <item><c><![CDATA[-decrypt_file <key_file>]]></c>, points out a file containing a decryption key for <seealso marker="config_file_chapter#encrypted_config_files">encrypted configuration files</seealso>.</item> <item><c><![CDATA[-basic_html]]></c>, switches off html enhancements that might not be compatible with older browsers.</item> + <item><c><![CDATA[-logopts <opts>]]></c>, makes it possible to modify aspects of the logging behaviour, see + <seealso marker="run_test_chapter#logopts">Log options</seealso> below.</item> </list> <note><p>Directories passed to Common Test may have either relative or absolute paths.</p></note> @@ -324,8 +335,9 @@ are to be executed by Common Test, and those functions only. If the step option <c>config</c> is specified, breakpoints will also be initially set on the configuration functions in the suite, i.e. - <c>init_per_suite/1</c>, <c>end_per_suite/1</c>, <c>init_per_testcase/2</c> - and <c>end_per_testcase/2</c>.</p> + <c>init_per_suite/1</c>, <c>end_per_suite/1</c>, + <c>init_per_group/2</c>, <c>end_per_group/2</c>, + <c>init_per_testcase/2</c> and <c>end_per_testcase/2</c>.</p> <p>Common Test enables the Debugger auto attach feature, which means that for every new interpreted test case function that starts to execute, a new trace window will automatically pop up. (This is because each test @@ -652,6 +664,30 @@ to follow test progress simply by refreshing pages in the HTML browser. Statistics totals are not presented until a test is complete however.</p> + <section> + <marker id="logopts"></marker> + <title>Log options</title> + <p>With the <c>logopts</c> start flag, it's possible to specify + options that modify some aspects of the logging behaviour. + Currently, the following options are available:</p> + <list> + <item><c>no_src</c></item> + <item><c>no_nl</c></item> + </list> + <p>With <c>no_src</c>, the html version of the test suite source + code will not be generated during the test run (and consequently + not be available in the log file system).</p> + <p>With <c>no_nl</c>, Common Test will not add a newline character + (\n) to the end of an output string that it receives from a call to e.g. + <c>io:format/2</c>, and which it prints to the test case log.</p> + <p>For example, if a test is started with:</p> + <p><c>$ ct_run -suite my_SUITE -logopts no_src</c></p> + <p>then printouts during the test made by successive calls to <c>io:format("x")</c>, + will appear in the test case log as:</p> + <p><c>xxx</c></p> + <p>instead of each <c>x</c> printed on a new line, which is the default behaviour.</p> + </section> + </section> <section> diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml index 3f9fdb7121..e35888e68f 100644 --- a/lib/common_test/doc/src/write_test_chapter.xml +++ b/lib/common_test/doc/src/write_test_chapter.xml @@ -280,6 +280,8 @@ the timetrap time is exceeded, the test case fails with reason <c>timetrap_timeout</c>. Note that <c>init_per_testcase</c> and <c>end_per_testcase</c> are included in the timetrap time. + Please see the <seealso marker="write_test_chapter#timetraps">Timetrap</seealso> + section for more details. </p> </item> <tag><em><c>userdata</c></em></tag> @@ -699,8 +701,8 @@ </section> <section> - <title>Timetrap timeouts</title> <marker id="timetraps"></marker> + <title>Timetrap timeouts</title> <p>The default time limit for a test case is 30 minutes, unless a <c>timetrap</c> is specified either by the suite info function or a test case info function. The timetrap timeout value defined @@ -723,6 +725,13 @@ multipled by <c>multiply_timetraps</c>, and possibly scaled up if <c>scale_timetraps</c> is enabled, the function <c>ct:sleep/1</c> may be called.</p> + <p>A function (<c>fun</c> or <c>MFA</c>) may be specified as timetrap value + in the suite- and test case info function, e.g:</p> + <p><c>{timetrap,{test_utils,get_timetrap_value,[?MODULE,system_start]}}</c></p> + <p>The function will be called initially by Common Test (before execution + of the suite or the test case) and must return a time value such as an + integer (millisec), or a <c>{SecMinOrHourTag,Time}</c> tuple. More + information can be found in the <c>common_test</c> reference manual.</p> </section> <section> @@ -818,6 +827,3 @@ </list> </section> </chapter> - - - |