diff options
Diffstat (limited to 'lib/test_server/doc/src')
| -rw-r--r-- | lib/test_server/doc/src/notes.xml | 362 | ||||
| -rw-r--r-- | lib/test_server/doc/src/test_server.xml | 18 | ||||
| -rw-r--r-- | lib/test_server/doc/src/test_server_ctrl.xml | 32 | ||||
| -rw-r--r-- | lib/test_server/doc/src/ts.xml | 8 |
4 files changed, 409 insertions, 11 deletions
diff --git a/lib/test_server/doc/src/notes.xml b/lib/test_server/doc/src/notes.xml index b6e0a6cefa..beeff55ffe 100644 --- a/lib/test_server/doc/src/notes.xml +++ b/lib/test_server/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2010</year> + <year>2004</year><year>2011</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,366 @@ <file>notes.xml</file> </header> +<section><title>Test_Server 3.4.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + An error in how comments are colored in the test suite + overview html log file has been corrected. As result, a + new framework callback function, format_comment/1, has + been introduced.</p> + <p> + Own Id: OTP-9237</p> + </item> + <item> + <p> + Test Server did not release SASL TTY handlers + (sasl_report_tty_h and error_logger_tty_h) properly after + each test run. This error has been fixed.</p> + <p> + Own Id: OTP-9311</p> + </item> + <item> + <p> + Automatically generated init- and end-configuration + functions for test case groups caused incorrect execution + order of test cases. This has been corrected.</p> + <p> + Own Id: OTP-9369</p> + </item> + <item> + <p> + If ct:log/2 was called with bad arguments, this could + cause the Common Test IO handling process to crash. This + fault has been corrected.</p> + <p> + Own Id: OTP-9371 Aux Id: OTP-8933 </p> + </item> + <item> + <p> + A bug has been fixed that made Test Server call the + end_tc/3 framework function with an incorrect module name + as first argument.</p> + <p> + Own Id: OTP-9379 Aux Id: seq11863 </p> + </item> + <item> + <p> + If end_per_testcase caused a timetrap timeout, the actual + test case status was discarded and the test case logged + as successful (even if the case had actually failed + before the call to end_per_testcase). This fault has been + fixed.</p> + <p> + Own Id: OTP-9397</p> + </item> + <item> + <p> + If a timetrap timeout occured during execution of of a + function in a lib module (i.e. a function called directly + or indirectly from a test case), the Suite argument in + the end_tc/3 framework callback function would not + correctly contain the name of the test suite, but the lib + module. (This would only happen if the lib module was + compiled with ct.hrl included). This error has been + solved.</p> + <p> + Own Id: OTP-9398</p> + </item> + <item> + <p> + Add a proplist() type</p> + <p> + Recently I was adding specs to an API and found that + there is no canonical proplist() type defined. (Thanks to + Ryan Zezeski)</p> + <p> + Own Id: OTP-9499</p> + </item> + <item> + <p> XML files have been corrected. </p> + <p> + Own Id: OTP-9550 Aux Id: OTP-9541 </p> + </item> + <item> + <p> + If a test suite would start with a test case group + defined without the init_per_group/2 and end_per_group/2 + function, init_per_suite/1 would not execute initially + and logging of the test run would fail. This error has + been fixed.</p> + <p> + Own Id: OTP-9584</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + A new option, 'logopts', has been introduced, to make it + possible to modify some aspects of the logging behaviour + in Common Test (or Test Server). For example, whenever an + io printout is made, test_server adds newline (\n) to the + end of the output string. This may not always be a + preferred action and can therefore be disabled by means + of "ct_run ... -logopts no_nl" (or ct:run_test([..., + {logopts,[no_nl]}])). A new framework callback function, + get_logopts/0, has been introduced (see the ct_framework + module for details).</p> + <p> + Own Id: OTP-9372 Aux Id: OTP-9396 </p> + </item> + <item> + <p> + A new option, 'logopts', has been introduced, to make it + possible to modify some aspects of the logging behaviour + in Common Test (or Test Server). For example, if the html + version of the test suite source code should not be + generated during the test run (and consequently be + unavailable in the log file system), the feature may be + disabled by means of "ct_run ... -logopts no_src" (or + ct:run_test([..., {logopts,[no_src]}])). A new framework + callback function, get_logopts/0, has been introduced + (see the ct_framework module for details).</p> + <p> + Own Id: OTP-9396 Aux Id: seq11869, OTP-9372 </p> + </item> + <item> + <p> + It is now possible to use a tuple {M,F,A}, or a fun, as + timetrap specification in the suite info function or test + case info functions. The function must return a valid + timeout value, as documented in the common_test man page + and in the User's Guide.</p> + <p> + Own Id: OTP-9501 Aux Id: seq11894 </p> + </item> + </list> + </section> + +</section> + +<section><title>Test_Server 3.4.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + It was previously not possible to use timetrap value + 'infinity' with ct:timetrap/1. This has been fixed.</p> + <p> + Own Id: OTP-9159</p> + </item> + <item> + <p> + A bug that made it impossible to cancel the previous + timetrap when calling ct:timetrap/1 has been corrected.</p> + <p> + Own Id: OTP-9233 Aux Id: OTP-9159 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + When running tests with auto-compilation disabled, Common + Test could only display the test suite source code on + html format in the test case log if the source file was + located in the same directory as the pre-compiled suite. + This has been modified so that Common Test now tries to + locate the source file by means of the test suite module + info (Suite:module_info/1). As a result, a suite may now + be compiled to a different output directory (e.g. + $MYTEST/bin) than the source code directory (e.g. + $MYTEST/src), without the source-code-to-html generation + being affected.</p> + <p> + Own Id: OTP-9138</p> + </item> + <item> + <p> + It is now possible to return a tuple {fail,Reason} from + init_per_testcase/2. The result is that the associated + test case gets logged as failed without ever executing.</p> + <p> + Own Id: OTP-9160 Aux Id: seq11502 </p> + </item> + <item> + <p> + Added DragonflyBSD check in test_server configure.</p> + <p> + Own Id: OTP-9249</p> + </item> + </list> + </section> + +</section> + +<section><title>Test_Server 3.4.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Updated the ts*.config files to contain information + relevant to testing Erlang/OTP in an open source + environment.</p> + <p> + Own Id: OTP-9017</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Alpha release of Common Test Hooks (CTH). CTHs allow the + users of common test to abtract out common behaviours + from test suites in a much more elegant and flexible way + than was possible before. Note that the addition of this + feature may introduce minor changes in the undocumented + behaviour of the interface inbetween common_test and + test_server.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8851</p> + </item> + </list> + </section> + +</section> + +<section><title>Test_Server 3.4.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Miscellaneous updates</p> + <p> + Own Id: OTP-8976</p> + </item> + </list> + </section> + +</section> + +<section><title>Test_Server 3.4.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Returning {return_group_result,failed} from end_per_group + in a group that is part of a sequence, did not cause the + proceeding cases (or groups) to get skipped. This has + been fixed.</p> + <p> + Own Id: OTP-8753 Aux Id: seq11644 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Common Test has been updated to handle start options and + test specification terms for test case groups (and test + cases in groups). Also, an option named 'label', has been + added that associates the test run with a name that + Common Test prints in the overview HTML logs.</p> + <p> + Own Id: OTP-8725 Aux Id: OTP-8727 </p> + </item> + <item> + <p> + It is now possible to skip all tests in a suite, or a + group, by returning {fail,Reason} from the end_tc/5 + framework function for init_per_suite, or init_per_group.</p> + <p> + Own Id: OTP-8805 Aux Id: seq11664 </p> + </item> + </list> + </section> + +</section> + +<section><title>Test_Server 3.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Returning {fail,Reason} from the framework end_tc + function was not handled properly by Test Server for all + test suite functions.</p> + <p> + Own Id: OTP-8492 Aux Id: seq11502 </p> + </item> + <item> + <p> + If the framework end_tc function would hang and get + aborted by Test Server, there was no indication of + failure in the logs. This has been fixed.</p> + <p> + Own Id: OTP-8682 Aux Id: seq11504 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + It is now possible for the Test Server framework end_tc + function to change the status of the test case from ok or + auto-skipped to failed by returning {fail,Reason}.</p> + <p> + Own Id: OTP-8495 Aux Id: seq11502 </p> + </item> + <item> + <p> + Test Server will now call the end_per_testcase/2 function + even if the test case has been terminated explicitly + (with abort_current_testcase/1), or after a timetrap + timeout. Under these circumstances the return value of + end_per_testcase is completely ignored. Therefore the + function will not be able to change the reason for test + case termination by returning {fail,Reason}, nor will it + be able to save data with {save_config,Data}.</p> + <p> + Own Id: OTP-8500 Aux Id: seq11521 </p> + </item> + <item> + <p> + Previously, a repeat property of a test case group + specified the number of times the group should be + repeated after the main test run. I.e. {repeat,N} would + case the group to execute 1+N times. To be consistent + with the behaviour of the run_test repeat option, this + has been changed. N now specifies the absolute number of + executions instead.</p> + <p> + Own Id: OTP-8689 Aux Id: seq11502 </p> + </item> + </list> + </section> + +</section> + <section><title>Test_Server 3.3.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/test_server/doc/src/test_server.xml b/lib/test_server/doc/src/test_server.xml index 6e75425862..78bb922cc5 100644 --- a/lib/test_server/doc/src/test_server.xml +++ b/lib/test_server/doc/src/test_server.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2007</year> - <year>2008</year> + <year>2011</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -167,6 +167,22 @@ </desc> </func> <func> + <name>adjusted_sleep(MSecs) -> ok</name> + <fsummary>Suspens the calling task for a specified time.</fsummary> + <type> + <v>MSecs = integer() | float() | infinity</v> + <d>The default number of milliseconds to sleep</d> + </type> + <desc> + <p>This function suspends the calling process for at least the + supplied number of milliseconds. The function behaves the same + way as <c>test_server:sleep/1</c>, only <c>MSecs</c> + will be multiplied by the 'multiply_timetraps' value, if set, + and also automatically scaled up if 'scale_timetraps' is set + to true (which it is by default).</p> + </desc> + </func> + <func> <name>hours(N) -> MSecs</name> <name>minutes(N) -> MSecs</name> <name>seconds(N) -> MSecs</name> diff --git a/lib/test_server/doc/src/test_server_ctrl.xml b/lib/test_server/doc/src/test_server_ctrl.xml index 0ed5f88544..9028a67ecb 100644 --- a/lib/test_server/doc/src/test_server_ctrl.xml +++ b/lib/test_server/doc/src/test_server_ctrl.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2007</year> - <year>2008</year> + <year>2011</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -376,6 +376,31 @@ Optional, if not given the test server controller node </desc> </func> <func> + <name>scale_timetraps(Bool) -> ok</name> + <fsummary>.</fsummary> + <type> + <v>Bool = true | false</v> + </type> + <desc> + <p>This function should be called before a test is started. + The parameter specifies if test_server should attempt + to automatically scale the timetrap value in order to compensate + for delays caused by e.g. the cover tool.</p> + </desc> + </func> + <func> + <name>get_timetrap_parameters() -> {N,Bool} </name> + <fsummary>Read the parameter values that affect timetraps.</fsummary> + <type> + <v>N = integer() | infinity</v> + <v>Bool = true | false</v> + </type> + <desc> + <p>This function may be called to read the values set by + <c>multiply_timetraps/1</c> and <c>scale_timetraps/1</c>.</p> + </desc> + </func> + <func> <name>cover(Application,Analyse) -> ok</name> <name>cover(CoverFile,Analyse) -> ok</name> <name>cover(App,CoverFile,Analyse) -> ok</name> @@ -538,9 +563,6 @@ Optional, if not given the test server controller node test server controller node. The log must be formatted using <c>ttb:format/1/2</c>. </p> - <p>This is valid for all targets except the OSE/Delta target - for which all nodes will be logged and automatically formatted - in one single text file called <c>allnodes-test_server</c>.</p> </desc> </func> <func> @@ -610,7 +632,7 @@ Optional, if not given the test server controller node <tag><c>NAME name</c></tag> <item>Names the test suite to something else than the default name. This does not apply to <c>SPEC</c> which keeps - it's names. + its names. </item> <tag><c>PARAMETERS parameterfile</c></tag> <item>Specifies the parameter file to use when starting diff --git a/lib/test_server/doc/src/ts.xml b/lib/test_server/doc/src/ts.xml index 0f91d3eea2..f9b48d8372 100644 --- a/lib/test_server/doc/src/ts.xml +++ b/lib/test_server/doc/src/ts.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2007</year> - <year>2008</year> + <year>2011</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -77,7 +77,7 @@ </p> <p><c>ts:install/0</c> is used if the target platform is the same as the controller host, i.e. if you run on "local target" - and no options are needed. Then running <c>ts:install/0</c><c>ts</c> + and no options are needed. Then running <c>ts:install/0</c> <c>ts</c> will run an autoconf script for your current environment and set up the necessary variables needed by the test suites. @@ -250,7 +250,7 @@ running test suites. If a remote host is to be used, the <c>TargetSystem</c> argument must be given so that "cross installation" can be done. This should be used for testing on - VxWorks or OSE/Delta. Installation is required for any of the + VxWorks. Installation is required for any of the functions in <c>ts</c> to work. </p> <p>Opts may be one or more of @@ -275,7 +275,7 @@ strings. </item> <item><c>{slavetargets, SlaveTarges}</c><br></br> - For VxWorks and OSE/Delta only. This is a list of + For VxWorks only. This is a list of available hosts where slave nodes can be started. This is necessary because only one node can run per host in the VxWorks environment. This is not the same as |