aboutsummaryrefslogtreecommitdiffstats
path: root/lib/common_test/doc
diff options
context:
space:
mode:
authorLukas Larsson <[email protected]>2011-10-04 10:16:09 +0200
committerLukas Larsson <[email protected]>2011-10-04 10:16:09 +0200
commit2d25f96f904bfa33c4bba4f3026f9fd2117ed8ab (patch)
tree26acbda51fa2ac4dd5f809e400d05aa1021eba20 /lib/common_test/doc
parent622daf3b96666f8bbabec44a15b26a188a83b95e (diff)
parent1a42cde6d62a8ec0f9e22215d3c3ec655201c78f (diff)
downloadotp-2d25f96f904bfa33c4bba4f3026f9fd2117ed8ab.tar.gz
otp-2d25f96f904bfa33c4bba4f3026f9fd2117ed8ab.tar.bz2
otp-2d25f96f904bfa33c4bba4f3026f9fd2117ed8ab.zip
Merge branch 'dev' into major
* dev: (38 commits) Update documentation Rid ct_telnet of doc build warnings Create temporary fix for problem with parallel test cases Update primary bootstrap Correct "Missing Suites" link Add documentation on timetraps and start flags Add missing tests for timetrap handling and fix remaining errors Solve problem with ct_init/end_per_group being counted as test cases Fix errors in test suites Fix invalid call to undefined function Fix problem with test_server_ctrl creating invalid conf test Improve info in CT framework log Update vsn.mk for common_test and test_server Enhance logging performance Change order of include files Add link to last executed test suite on index page Fix problem with location value when init config func calls help func Fix crash when CTHook init fails Correct error in test suite Fix error with incorrect notification after end_per_testcase craches ... Conflicts: bootstrap/bin/start.boot bootstrap/bin/start_clean.boot bootstrap/lib/compiler/ebin/beam_asm.beam bootstrap/lib/compiler/ebin/beam_disasm.beam bootstrap/lib/compiler/ebin/compile.beam bootstrap/lib/compiler/ebin/sys_pre_expand.beam bootstrap/lib/kernel/ebin/code.beam bootstrap/lib/kernel/ebin/code_server.beam bootstrap/lib/kernel/ebin/hipe_unified_loader.beam bootstrap/lib/kernel/ebin/inet.beam bootstrap/lib/kernel/ebin/inet_config.beam bootstrap/lib/kernel/ebin/inet_dns.beam bootstrap/lib/stdlib/ebin/beam_lib.beam bootstrap/lib/stdlib/ebin/dets.beam bootstrap/lib/stdlib/ebin/erl_compile.beam bootstrap/lib/stdlib/ebin/erl_internal.beam bootstrap/lib/stdlib/ebin/erl_scan.beam bootstrap/lib/stdlib/ebin/erl_tar.beam bootstrap/lib/stdlib/ebin/io_lib_fread.beam bootstrap/lib/stdlib/ebin/otp_internal.beam bootstrap/lib/stdlib/ebin/sofs.beam bootstrap/lib/stdlib/ebin/supervisor.beam bootstrap/lib/stdlib/ebin/zip.beam lib/common_test/src/ct.erl lib/common_test/src/ct_run.erl lib/common_test/test/ct_error_SUITE.erl lib/common_test/test/ct_repeat_1_SUITE.erl lib/common_test/test/ct_skip_SUITE.erl lib/test_server/src/test_server.erl
Diffstat (limited to 'lib/common_test/doc')
-rw-r--r--lib/common_test/doc/src/common_test_app.xml32
-rw-r--r--lib/common_test/doc/src/config_file_chapter.xml6
-rw-r--r--lib/common_test/doc/src/ct_run.xml7
-rw-r--r--lib/common_test/doc/src/run_test_chapter.xml40
-rw-r--r--lib/common_test/doc/src/write_test_chapter.xml14
5 files changed, 80 insertions, 19 deletions
diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml
index 57b032b3fd..f58b2ab0a9 100644
--- a/lib/common_test/doc/src/common_test_app.xml
+++ b/lib/common_test/doc/src/common_test_app.xml
@@ -133,9 +133,15 @@
{require,Name,Required} | {userdata,UserData} |
{silent_connections,Conns} | {stylesheet,CSSFile} |
{ct_hooks, CTHs}</v>
- <v> Time = MilliSec | {seconds,integer()} | {minutes,integer()}
- | {hours,integer()}</v>
+ <v> Time = TimeVal | TimeFunc</v>
+ <v> TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} |
+ {hours,integer()}</v>
+ <v> TimeFunc = {Mod,Func,Args} | Fun</v>
<v> MilliSec = integer()</v>
+ <v> Mod = atom()</v>
+ <v> Func = atom()</v>
+ <v> Args = list()</v>
+ <v> Fun = fun()</v>
<v> Required = Key | {Key,SubKeys}</v>
<v> Key = atom()</v>
<v> SubKeys = SubKey | [SubKey]</v>
@@ -161,7 +167,9 @@
test case is allowed to take (including <c>init_per_testcase/2</c>
and <c>end_per_testcase/2</c>). If the timetrap time is
exceeded, the test case fails with reason
- <c>timetrap_timeout</c>.</p>
+ <c>timetrap_timeout</c>. If a <c>TimeFunc</c> function is specified,
+ it will be called initially and must return a value on
+ <c>TimeVal</c> format.</p>
<p>The <c>require</c> tag specifies configuration variables
that are required by test cases in the suite. If the required
@@ -248,7 +256,7 @@
</type>
<desc>
- <p> MANDATORY (only if one or more groups are defined) </p>
+ <p> OPTIONAL </p>
<p>This function is called before execution of a test case group.
It typically contains initialization which is common for
@@ -279,7 +287,7 @@
</type>
<desc>
- <p> MANDATORY (only if one or more groups are defined) </p>
+ <p> OPTIONAL </p>
<p>This function is called after the execution of a test case group is finished.
It is meant to be used for cleaning up after <c>init_per_group/2</c>.
@@ -353,9 +361,15 @@
<v> Info = {timetrap,Time} | {require,Required} |
{require,Name,Required} | {userdata,UserData} |
{silent_connections,Conns}</v>
- <v> Time = MilliSec | {seconds,integer()} | {minutes,integer()}
- | {hours,integer()}</v>
+ <v> Time = TimeVal | TimeFunc</v>
+ <v> TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} |
+ {hours,integer()}</v>
+ <v> TimeFunc = {Mod,Func,Args} | Fun</v>
<v> MilliSec = integer()</v>
+ <v> Mod = atom()</v>
+ <v> Func = atom()</v>
+ <v> Args = list()</v>
+ <v> Fun = fun()</v>
<v> Required = Key | {Key,SubKeys}</v>
<v> Key = atom()</v>
<v> SubKeys = SubKey | [SubKey]</v>
@@ -378,7 +392,9 @@
exceeded, the test case fails with reason
<c>timetrap_timeout</c>. <c>init_per_testcase/2</c>
and <c>end_per_testcase/2</c> are included in the
- timetrap time.</p>
+ timetrap time. If a <c>TimeFunc</c> function is specified,
+ it will be called before the test case (or <c>init_per_testcase/2</c>)
+ and must return a value on <c>TimeVal</c> format.</p>
<p>The <c>require</c> tag specifies configuration variables
that are required by the test case. If the required
diff --git a/lib/common_test/doc/src/config_file_chapter.xml b/lib/common_test/doc/src/config_file_chapter.xml
index 59151a73ec..6fc6638bf7 100644
--- a/lib/common_test/doc/src/config_file_chapter.xml
+++ b/lib/common_test/doc/src/config_file_chapter.xml
@@ -285,7 +285,7 @@
<c>{ok, Config}</c> - if the configuration variables are read successfully,
</item>
<item>
- <c>{error, Error, ErrorDetails}</c> - if the callback module fails to
+ <c>{error, {Error, ErrorDetails}}</c> - if the callback module fails to
proceed with the given configuration parameters.
</item>
</list>
@@ -422,14 +422,14 @@ stop()->
call(Client, Request)->
case whereis(?REGISTERED_NAME) of
undefined->
- {error, not_started, Request};
+ {error, {not_started, Request}};
Pid->
Pid ! {Client, Request},
receive
Reply->
{ok, Reply}
after 4000->
- {error, timeout, Request}
+ {error, {timeout, Request}}
end
end.
diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml
index 1ab563d74f..9045646733 100644
--- a/lib/common_test/doc/src/ct_run.xml
+++ b/lib/common_test/doc/src/ct_run.xml
@@ -83,7 +83,7 @@
<title>Run tests from command line</title>
<pre>
ct_run [-dir TestDir1 TestDir2 .. TestDirN] |
- [-suite Suite1 Suite2 .. SuiteN
+ [[-dir TestDir] -suite Suite1 Suite2 .. SuiteN
[[-group Group1 Group2 .. GroupN] [-case Case1 Case2 .. CaseN]]]
[-step [config | keep_inactive]]
[-config ConfigFile1 ConfigFile2 .. ConfigFileN]
@@ -92,6 +92,7 @@
[-decrypt_key Key] | [-decrypt_file KeyFile]
[-label Label]
[-logdir LogDir]
+ [-logopts LogOpts]
[-silent_connections [ConnType1 ConnType2 .. ConnTypeN]]
[-stylesheet CSSFile]
[-cover CoverCfgFile]
@@ -117,6 +118,7 @@
[-decrypt_key Key] | [-decrypt_file KeyFile]
[-label Label]
[-logdir LogDir]
+ [-logopts LogOpts]
[-allow_user_terms]
[-silent_connections [ConnType1 ConnType2 .. ConnTypeN]]
[-stylesheet CSSFile]
@@ -138,10 +140,11 @@
<pre>
ct_run -vts [-browser Browser]
[-dir TestDir1 TestDir2 .. TestDirN] |
- [-suite Suite [[-group Group] [-case Case]]]
+ [[dir TestDir] -suite Suite [[-group Group] [-case Case]]]
[-config ConfigFile1 ConfigFile2 .. ConfigFileN]
[-userconfig CallbackModule1 ConfigString1 and CallbackModule2
ConfigString2 and .. and CallbackModuleN ConfigStringN]
+ [-logopts LogOpts]
[-decrypt_key Key] | [-decrypt_file KeyFile]
[-include InclDir1 InclDir2 .. InclDirN]
[-no_auto_compile]
diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml
index 816aa5b1eb..57059f0ba2 100644
--- a/lib/common_test/doc/src/run_test_chapter.xml
+++ b/lib/common_test/doc/src/run_test_chapter.xml
@@ -128,6 +128,15 @@
<p><c>$ ct_run -suite $SYS1_TEST/setup_SUITE -case start stop</c></p>
<p><c>$ ct_run -suite $SYS1_TEST/setup_SUITE -group installation -case start stop</c></p>
+ <p>It is also possible to combine the <c>dir</c>, <c>suite</c> and <c>group/case</c> flags. E.g, to run
+ <c>x_SUITE</c> and <c>y_SUITE</c> in directory <c>testdir</c>:</p>
+
+ <p><c>$ ct_run -dir ./testdir -suite x_SUITE y_SUITE</c></p>
+
+ <p>This has the same effect as calling:</p>
+
+ <p><c>$ ct_run -suite ./testdir/x_SUITE ./testdir/y_SUITE</c></p>
+
<p>Other flags that may be used with <c>ct_run</c>:</p>
<list>
<item><c><![CDATA[-logdir <dir>]]></c>, specifies where the HTML log files are to be written.</item>
@@ -167,6 +176,8 @@
<item><c><![CDATA[-decrypt_file <key_file>]]></c>, points out a file containing a decryption key for
<seealso marker="config_file_chapter#encrypted_config_files">encrypted configuration files</seealso>.</item>
<item><c><![CDATA[-basic_html]]></c>, switches off html enhancements that might not be compatible with older browsers.</item>
+ <item><c><![CDATA[-logopts <opts>]]></c>, makes it possible to modify aspects of the logging behaviour, see
+ <seealso marker="run_test_chapter#logopts">Log options</seealso> below.</item>
</list>
<note><p>Directories passed to Common Test may have either relative or absolute paths.</p></note>
@@ -324,8 +335,9 @@
are to be executed by Common Test, and those functions only. If
the step option <c>config</c> is specified, breakpoints will
also be initially set on the configuration functions in the suite, i.e.
- <c>init_per_suite/1</c>, <c>end_per_suite/1</c>, <c>init_per_testcase/2</c>
- and <c>end_per_testcase/2</c>.</p>
+ <c>init_per_suite/1</c>, <c>end_per_suite/1</c>,
+ <c>init_per_group/2</c>, <c>end_per_group/2</c>,
+ <c>init_per_testcase/2</c> and <c>end_per_testcase/2</c>.</p>
<p>Common Test enables the Debugger auto attach feature, which means
that for every new interpreted test case function that starts to execute,
a new trace window will automatically pop up. (This is because each test
@@ -652,6 +664,30 @@
to follow test progress simply by refreshing pages in the HTML browser.
Statistics totals are not presented until a test is complete however.</p>
+ <section>
+ <marker id="logopts"></marker>
+ <title>Log options</title>
+ <p>With the <c>logopts</c> start flag, it's possible to specify
+ options that modify some aspects of the logging behaviour.
+ Currently, the following options are available:</p>
+ <list>
+ <item><c>no_src</c></item>
+ <item><c>no_nl</c></item>
+ </list>
+ <p>With <c>no_src</c>, the html version of the test suite source
+ code will not be generated during the test run (and consequently
+ not be available in the log file system).</p>
+ <p>With <c>no_nl</c>, Common Test will not add a newline character
+ (\n) to the end of an output string that it receives from a call to e.g.
+ <c>io:format/2</c>, and which it prints to the test case log.</p>
+ <p>For example, if a test is started with:</p>
+ <p><c>$ ct_run -suite my_SUITE -logopts no_src</c></p>
+ <p>then printouts during the test made by successive calls to <c>io:format("x")</c>,
+ will appear in the test case log as:</p>
+ <p><c>xxx</c></p>
+ <p>instead of each <c>x</c> printed on a new line, which is the default behaviour.</p>
+ </section>
+
</section>
<section>
diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml
index 3f9fdb7121..e35888e68f 100644
--- a/lib/common_test/doc/src/write_test_chapter.xml
+++ b/lib/common_test/doc/src/write_test_chapter.xml
@@ -280,6 +280,8 @@
the timetrap time is exceeded, the test case fails with
reason <c>timetrap_timeout</c>. Note that <c>init_per_testcase</c>
and <c>end_per_testcase</c> are included in the timetrap time.
+ Please see the <seealso marker="write_test_chapter#timetraps">Timetrap</seealso>
+ section for more details.
</p>
</item>
<tag><em><c>userdata</c></em></tag>
@@ -699,8 +701,8 @@
</section>
<section>
- <title>Timetrap timeouts</title>
<marker id="timetraps"></marker>
+ <title>Timetrap timeouts</title>
<p>The default time limit for a test case is 30 minutes, unless a
<c>timetrap</c> is specified either by the suite info function
or a test case info function. The timetrap timeout value defined
@@ -723,6 +725,13 @@
multipled by <c>multiply_timetraps</c>, and possibly scaled up if
<c>scale_timetraps</c> is enabled, the function <c>ct:sleep/1</c>
may be called.</p>
+ <p>A function (<c>fun</c> or <c>MFA</c>) may be specified as timetrap value
+ in the suite- and test case info function, e.g:</p>
+ <p><c>{timetrap,{test_utils,get_timetrap_value,[?MODULE,system_start]}}</c></p>
+ <p>The function will be called initially by Common Test (before execution
+ of the suite or the test case) and must return a time value such as an
+ integer (millisec), or a <c>{SecMinOrHourTag,Time}</c> tuple. More
+ information can be found in the <c>common_test</c> reference manual.</p>
</section>
<section>
@@ -818,6 +827,3 @@
</list>
</section>
</chapter>
-
-
-