diff options
Diffstat (limited to 'lib/test_server/doc/src')
| -rw-r--r-- | lib/test_server/doc/src/basics_chapter.xml | 12 | ||||
| -rw-r--r-- | lib/test_server/doc/src/notes.xml | 249 | ||||
| -rw-r--r-- | lib/test_server/doc/src/part.xml | 4 | ||||
| -rw-r--r-- | lib/test_server/doc/src/part_notes.xml | 4 | ||||
| -rw-r--r-- | lib/test_server/doc/src/part_notes_history.xml | 4 | ||||
| -rw-r--r-- | lib/test_server/doc/src/ref_man.xml | 4 | ||||
| -rw-r--r-- | lib/test_server/doc/src/test_server.xml | 23 | ||||
| -rw-r--r-- | lib/test_server/doc/src/test_server_app.xml | 5 | ||||
| -rw-r--r-- | lib/test_server/doc/src/test_server_ctrl.xml | 148 | ||||
| -rw-r--r-- | lib/test_server/doc/src/test_spec_chapter.xml | 5 | ||||
| -rw-r--r-- | lib/test_server/doc/src/ts.xml | 35 | ||||
| -rw-r--r-- | lib/test_server/doc/src/write_framework_chapter.xml | 13 | ||||
| -rw-r--r-- | lib/test_server/doc/src/write_test_chapter.xml | 3 |
13 files changed, 356 insertions, 153 deletions
diff --git a/lib/test_server/doc/src/basics_chapter.xml b/lib/test_server/doc/src/basics_chapter.xml index a96cc88075..2e60d412e9 100644 --- a/lib/test_server/doc/src/basics_chapter.xml +++ b/lib/test_server/doc/src/basics_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2002</year><year>2009</year> + <year>2002</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -105,14 +105,14 @@ <p>The Test Server consists of three parts: </p> <list type="bulleted"> - <item>The part that executes the test suites on target and + <item>The part that executes the test suites and provides support for the test suite author is called <c>test_server</c>. This is described in the chapter about writing test cases in this user's guide, and in the reference manual for the <c>test_server</c> module.</item> <item>The controlling part, which provides the low level - operator interface, starts and stops the target node (if remote - target) and slave nodes and writes log files, is called + operator interface, starts and stops slave nodes and writes + log files, is called <c>test_server_ctrl</c>. The Test Server Controller should not be used directly when running tests. Instead a framework built on top of it should be used. More information @@ -176,9 +176,7 @@ shall return an empty list, a test specification or <c>{skip,Reason}</c>. If an empty list is returned, it means that the test case shall be executed, and so it must also have - an execution clause. Note that the specification clause is - always executed on the controller node, i.e. not on the target - node. + an execution clause. </item> <tag><em>test case</em></tag> <item>A single test included in a test suite. Typically it tests diff --git a/lib/test_server/doc/src/notes.xml b/lib/test_server/doc/src/notes.xml index 66c4d3d494..c6c166c796 100644 --- a/lib/test_server/doc/src/notes.xml +++ b/lib/test_server/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2012</year> + <year>2004</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,21 +32,250 @@ <file>notes.xml</file> </header> -<section><title>Test_Server 3.5.3.1</title> +<section><title>Test_Server 3.6.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Some unused code related to remote targets is removed, + and documentation is updated.</p> + <p> + Own Id: OTP-10607 Aux Id: kunagi-338 [249] </p> + </item> + <item> + <p> + A bug in test_server_gl caused io requests containing + invalid data (i.e. not unicode:chardata()) to hang, since + no io reply was sent. This has been corrected.</p> + <p> + Own Id: OTP-10991</p> + </item> + <item> + <p> + Common Test would, in case of timetrap error, print a + warning in the log if end_per_testcase wasn't implemented + in the suite, even though it's an optional function. This + printout has been removed.</p> + <p> + Own Id: OTP-11052</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The '-force_stop' flag to use with time-limited repeats + of test runs can now be used with a new 'skip_rest' + option which causes the rest of the test cases in the + ongoing test job to be skipped when the time limit is + reached. E.g. 'ct_run -spec xxx -duration 010000 + -force_stop skip_rest'</p> + <p> + Own Id: OTP-10856 Aux Id: OTP-10832 </p> + </item> + </list> + </section> + +</section> + +<section><title>Test_Server 3.6.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The unicode update of test_server for R16A introduced a + few potential errors when logging to files. Sometimes ~tp + or ~ts was used for formatting also when writing to files + that were not opened with the {encoding,utf8} option. If + then the argument contained unicode characters above 255, + the file descriptor would crash. This has been corrected + by the following modifications: <list> <item> Since the + 'unexpected_io' log file is used only when the test case + HTML file is not available (e.g. between test cases), + this file is now also a HTML file and as other + test_server HTML logs it is always UTF-8 encoded </item> + <item> Since it is possible to change which information + is going to which log file (with + test_server_ctrl:set_levels/3), we do not have full + control over which information is written to which file. + This means that any printout could be written to the + 'major' log file (suite.log), which was earlier encoded + as latin1. To avoid crashing this file descriptor due to + unicode strings, the 'major' log file is now also encoded + in UTF-8 (possible incopatibility). </item> <item> The + cross_cover.info file is no longer a text file which can + be read with file:consult/1, instead it is written as a + pure binary file using term_to_binary when writing and + binary_to_term when reading. </item> <item> The encoding + of the file named 'last_name', which only content is the + path to the last run.<timestamp> directory, is now + dependent on the file name mode of the VM. If file names + are expected to be unicode, then the 'last_name' file is + UTF-8 encoded, else it is latin1 encoded. </item> </list></p> + <p> + Also, ~tp has been changed back to ~p unless it is + somehow likely that the argument includes strings. It is + not obvious that this is the correct thing to do, but + some decission had to be taken...</p> + <p> + Own Id: OTP-10780</p> + </item> + <item> + <p> + Using the force_stop flag/option to interrupt a test run + caused a crash in Common Test. This problem has been + solved.</p> + <p> + Own Id: OTP-10832</p> + </item> + </list> + </section> + +</section> + +<section><title>Test_Server 3.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Line numbering of erlang files that were not correctly + indented could be wrong after coverting to html with + erl2html2:convert/[2,3] (the source code pointed to from + the test case). This has been corrected.</p> + <p> + Also, there are now link targets for each line and not + only for each 10th line, and link targets for functions + now include the arity and not only the function name + (e.g. func/1 has a link target "func-1").</p> + <p> + Own Id: OTP-9710 Aux Id: seq11945, kunagi-201 [112] </p> + </item> + <item> + <p> + Severe errors detected by <c>test_server</c> (e.g. if log + files directories cannot be created) will now be reported + to <c>common_test</c> and noted in the <c>common_test</c> + logs.</p> + <p> + Own Id: OTP-9769 Aux Id: kunagi-202 [113] </p> + </item> + <item> + <p> + The earlier undocumented cross cover feature for + accumulating cover data over multiple tests has now been + fixed and documented.</p> + <p> + Own Id: OTP-9870 Aux Id: kunagi-206 [117] </p> + </item> + <item> + <p> + If the test suite itself was included in code coverage + analysis, then the test_server would not manage to set + data_dir correctly for the test. This has been corrected.</p> + <p> + Own Id: OTP-9956 Aux Id: kunagi-207 [118] </p> + </item> + <item> + <p> + Any call to test_server:break/1 should cancel all active + timetramps. However, in some cases + Suite:end_per_testcase/2 is executed on a different + process than the test case itself, and if + test_server:break/1 was called from there, the timetraps + were not cancelled. This has been corrected.</p> + <p> + Own Id: OTP-10046 Aux Id: kunagi-174 [85] </p> + </item> + <item> + <p>When a test case failed because of a timetrap time + out, the <c>Config</c> data for the case was lost in the + following call to <c>end_per_testcase/2</c>, and also in + calls to the CT Hook function + <c>post_end_per_testcase/4</c>. This problem has been + solved and the <c>Config</c> data is now correctly passed + to the above functions after a timetrap timeout + failure.</p> + <p> + Own Id: OTP-10070 Aux Id: kunagi-175 [86] </p> + </item> + <item> + <p>In test_server, the same process would supervise the + currently running test case and be group leader (and IO + server) for the test case. Furthermore, when running + parallel test cases, new temporary supervisor/group + leader processes were spawned and the process that was + group leader for sequential test cases would not be + active. That would lead to several problems:</p> + <p>* Processes started by init_per_suite will inherit the + group leader of the init_per_suite process (and that + group leader would not process IO requests when parallel + test cases was running). If later a parallel test case + caused such a processto print using (for example) + io:format/2, the calling would hang.</p> + <p>* Similarly, if a process was spawned from a parallel + test case, it would inherit the temporary group leader + for that parallel test case. If that spawned process + later - when the group of parallel tests have finished - + attempted to print something, its group leader would be + dead and there would be <c>badarg</c> exception.</p> + <p>Those problems have been solved by having group + leaders separate from the processes that supervises the + test cases, and keeping temporary group leader process + for parallel test cases alive until no more process in + the system use them as group leaders.</p> + <p>Also, a new <c>unexpected_io.log</c> log file + (reachable from the summary page of each test suite) has + been introduced. All unexpected IO will be printed into + it(for example, IO to a group leader for a parallel test + case that has finished).</p> + <p> + Own Id: OTP-10101 Aux Id: OTP-10125 </p> + </item> + <item> + <p> + The stability of <c>common_test</c> and + <c>test_server</c> when running test cases in parallel + has been improved.</p> + <p> + Own Id: OTP-10480 Aux Id: kunagi-318 [229] </p> + </item> + </list> + </section> + <section><title>Improvements and New Features</title> <list> <item> <p> - A new option to erlang:system_monitor, - {long_schedule,Millis} allows you to monitor if your - system performs any lengthy NIF, BIF or driver operations - that might result in bad resource utilization and - therefore bad performance. To better test this - functionality, the test_server has been updated to - include configure tests for usleep.</p> + Added a general framework for executing benchmarks of + Erlang/OTP. Benchmarks for the Erlang VM and mnesia have + been incorporated in the framework. </p> + <p> + For details about how to add more benchmarks see + $ERL_TOP/HOWTO/BENCHMARKS.md in the source distribution.</p> + <p> + Own Id: OTP-10156</p> + </item> + <item> <p> - Own Id: OTP-11067</p> + Update common test modules to handle unicode <list> + <item> Use UTF-8 encoding for all HTML files, except the + HTML version of the test suite generated with + erl2html2:convert, which will have the same encoding as + the original test suite (.erl) file. </item> <item> + Encode link targets in HTML files with + test_server_ctrl:uri_encode/1. </item> <item> Use unicode + modifier 't' with ~s when appropriate. </item> <item> Use + unicode:characters_to_list and + unicode:characters_to_binary for conversion between + binaries and strings instead of binary_to_list and + list_to_binary. </item> </list></p> </item> </list> </section> diff --git a/lib/test_server/doc/src/part.xml b/lib/test_server/doc/src/part.xml index fdcd3d274e..a20835e035 100644 --- a/lib/test_server/doc/src/part.xml +++ b/lib/test_server/doc/src/part.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2002</year><year>2009</year> + <year>2002</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -30,7 +30,7 @@ <description> <p><em>Test Server</em> is a portable test server for automated application testing. The server can run test suites - on local or remote targets and log progress and results to HTML + and log progress and results to HTML pages. The main purpose of Test Server is to act as engine inside customized test tools. A callback interface for such framework applications is provided.</p> diff --git a/lib/test_server/doc/src/part_notes.xml b/lib/test_server/doc/src/part_notes.xml index 2347f64ca1..abf949dfda 100644 --- a/lib/test_server/doc/src/part_notes.xml +++ b/lib/test_server/doc/src/part_notes.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2004</year><year>2009</year> + <year>2004</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -30,7 +30,7 @@ <description> <p>The <em>Test Server</em> is a portable test server for application testing. The test server can run automatic test suites - on local or remote target and log progress and results to HTML + and log progress and results to HTML pages. It also provides some support for test suite authors.</p> <p>For information about older versions, see <url href="part_notes_history_frame.html">Release Notes History</url>.</p> diff --git a/lib/test_server/doc/src/part_notes_history.xml b/lib/test_server/doc/src/part_notes_history.xml index 556d172755..4746f583b9 100644 --- a/lib/test_server/doc/src/part_notes_history.xml +++ b/lib/test_server/doc/src/part_notes_history.xml @@ -4,7 +4,7 @@ <part> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -30,7 +30,7 @@ <description> <p>The <em>Test Server</em> is a portable test server for application testing. The test server can run automatic test suites - on local or remote target and log progress and results to HTML + and log progress and results to HTML pages. It also provides some support for test suite authors.</p> </description> <include file="notes_history"></include> diff --git a/lib/test_server/doc/src/ref_man.xml b/lib/test_server/doc/src/ref_man.xml index 17d6093dc0..ca0b38e0c8 100644 --- a/lib/test_server/doc/src/ref_man.xml +++ b/lib/test_server/doc/src/ref_man.xml @@ -4,7 +4,7 @@ <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2002</year><year>2009</year> + <year>2002</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,7 +31,7 @@ <description> <p><em>Test Server</em> is a portable test server for automated application testing. The server can run test suites - on local or remote targets and log progress and results to HTML + and log progress and results to HTML pages. The main purpose of Test Server is to act as engine inside customized test tools. A callback interface for such framework applications is provided.</p> diff --git a/lib/test_server/doc/src/test_server.xml b/lib/test_server/doc/src/test_server.xml index 5bfa42c36f..5ef43dd628 100644 --- a/lib/test_server/doc/src/test_server.xml +++ b/lib/test_server/doc/src/test_server.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2007</year> - <year>2011</year> + <year>2013</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -73,8 +73,8 @@ <d>This is the same as returned from <c>os:type/0</c></d> </type> <desc> - <p>This function can be called on controller or target node, and - it will always return the OS type of the target node.</p> + <p>This function is equivalent to <c>os:type/0</c>. It is kept + for backwards compatibility.</p> </desc> </func> <func> @@ -465,9 +465,7 @@ Tries running clock_sanity_check() up to 8 times,and <taglist> <tag><c>{remote, true}</c></tag> <item>Start the node on a remote host. If not specified, the - node will be started on the local host (with some - exceptions, as for the case of VxWorks, where - all nodes are started on a remote host). Test cases that + node will be started on the local host. Test cases that require a remote host will fail with a reasonable comment if no remote hosts are available at the time they are run. </item> @@ -529,6 +527,18 @@ Only valid for peer nodes. Note that slave nodes always analogy with <c>os:getenv/1</c>), which removes the environment variable. Only valid for peer nodes. Not available on VxWorks.</item> + <tag><c>{start_cover, false}</c></tag> + <item>By default the test server will start cover on all nodes + when the test is run with code coverage analysis. To make + sure cover is not started on a new node, set this option to + <c>false</c>. This can be necessary if the connection to + the node at some point will be broken but the node is + expected to stay alive. The reason is that a remote cover + node can not continue to run without its main node. Another + solution would be to explicitly stop cover on the node + before breaking the connection, but in some situations (if + old code resides in one or more processes) this is not + possible.</item> </taglist> </desc> </func> @@ -736,7 +746,6 @@ Only valid for peer nodes. Note that slave nodes always test case or <c>{skip,Comment}</c>. The syntax of a test specification is described in the Test Server User's Guide. </p> - <p><em>Note that the specification clause always is executed on the controller host.</em></p> <p>The <em>execution clause</em> (argument <c>Config</c>) is only called if the specification clause returns an empty list. The execution clause is the real test case. Here you must call diff --git a/lib/test_server/doc/src/test_server_app.xml b/lib/test_server/doc/src/test_server_app.xml index 924cdc886b..ea4b5ffecf 100644 --- a/lib/test_server/doc/src/test_server_app.xml +++ b/lib/test_server/doc/src/test_server_app.xml @@ -4,7 +4,7 @@ <appref> <header> <copyright> - <year>2002</year><year>2009</year> + <year>2002</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -36,14 +36,13 @@ <description> <p><em>Test Server</em> is a portable test server for automated application testing. The server can run test suites - on local or remote targets and log progress and results to HTML + and log progress and results to HTML pages. The main purpose of Test Server is to act as engine inside customized test tools. A callback interface for such framework applications is provided.</p> <p>In brief the test server supports:</p> <list type="bulleted"> <item>Running multiple, concurrent test suites</item> - <item>Running tests on remote and even diskless targets</item> <item>Test suites may contain other test suites, in a tree fashion</item> <item>Logging of the events in a test suite, on both suite and case levels</item> <item>HTML presentation of test suite results</item> diff --git a/lib/test_server/doc/src/test_server_ctrl.xml b/lib/test_server/doc/src/test_server_ctrl.xml index 41bc0bcc75..9252cdce4f 100644 --- a/lib/test_server/doc/src/test_server_ctrl.xml +++ b/lib/test_server/doc/src/test_server_ctrl.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2007</year> - <year>2012</year> + <year>2013</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -57,64 +57,19 @@ <funcs> <func> <name>start() -> Result</name> - <name>start(ParameterFile) -> Result</name> <fsummary>Starts the test server.</fsummary> <type> <v>Result = ok | {error, {already_started, pid()}</v> - <v>ParameterFile = atom() | string()</v> </type> <desc> - <p>This function starts the test server. If the parameter file - is given, it indicates that the target is remote. In that case - the target node is started and a socket connection is - established between the controller and the target node. - </p> - <p>The parameter file is a text file containing key-value - tuples. Each tuple must be followed by a dot-newline - sequence. The following key-value tuples are allowed: - </p> - <taglist> - <tag><c>{type,PlatformType}</c></tag> - <item>This is an atom indicating the target platform type, - currently supported: <c>PlatformType = vxworks</c> <br></br> -Mandatory - </item> - <tag><c>{target,TargetHost}</c></tag> - <item>This is the name of the target host, can be atom or - string. - <br></br> -Mandatory - </item> - <tag><c>{slavetargets,SlaveTargets}</c></tag> - <item>This is a list of available hosts where slave nodes - can be started. The hostnames are given as atoms or strings. - <br></br> -Optional, default <c>SlaveTargets = []</c></item> - <tag><c>{longnames,Bool}</c></tag> - <item>This indicates if longnames shall be used, i.e. if the - <c>-name</c> option should be used for the target node - instead of <c>-sname</c> <br></br> -Optional, default <c>Bool = false</c></item> - <tag><c>{master, {MasterHost, MasterCookie}}</c></tag> - <item>If target is remote and the target node is started as - a slave node, this option indicates which master and - cookie to use. The given master - will also be used as master for slave nodes started with - <c>test_server:start_node/3</c>. It is expected that the - <c>erl_boot_server</c> is started on the master node before - the <c>test_server_ctrl:start/1</c> function is called. - <br></br> -Optional, if not given the test server controller node - is used as master and the <c>erl_boot_server</c> is - automatically started.</item> - </taglist> + <p>This function starts the test server.</p> </desc> </func> <func> <name>stop() -> ok</name> <fsummary>Stops the test server immediately.</fsummary> <desc> - <p>This stops the test server (both controller and target) and + <p>This stops the test server and all its activity. The running test suite (if any) will be halted.</p> </desc> @@ -427,11 +382,21 @@ Optional, if not given the test server controller node <p>A <c>CoverFile</c> can have the following entries:</p> <code type="none"> {exclude, all | ExcludeModuleList}. -{include, IncludeModuleList}. </code> +{include, IncludeModuleList}. +{cross, CrossCoverInfo}.</code> <p>Note that each line must end with a full stop. <c>ExcludeModuleList</c> and <c>IncludeModuleList</c> are lists of atoms, where each atom is a module name. </p> + + <p><c>CrossCoverInfo</c> is used when collecting cover data + over multiple tests. Modules listed here are compiled, but + they will not be analysed when the test is finished. See + <seealso + marker="#cross_cover_analyse-2">cross_cover_analyse/2</seealso> + for more information about the cross cover mechanism and the + format of <c>CrossCoverInfo</c>. + </p> <p>If both an <c>Application</c> and a <c>CoverFile</c> is given, all modules in the application are cover compiled, except for the modules listed in <c>ExcludeModuleList</c>. The @@ -467,30 +432,71 @@ Optional, if not given the test server controller node </desc> </func> <func> - <name>cross_cover_analyse(Level) -> ok</name> - <fsummary>Analyse cover data collected from all tests</fsummary> + <name>cross_cover_analyse(Level, Tests) -> ok</name> + <fsummary>Analyse cover data collected from multiple tests</fsummary> <type> <v>Level = details | overview</v> + <v>Tests = [{Tag,LogDir}]</v> + <v>Tag = atom()</v> + <d>Test identifier.</d> + <v>LogDir = string()</v> + <d>Log directory for the test identified by <c>Tag</c>. This + can either be the <c>run.<timestamp></c> directory or + the parent directory of this (in which case the latest + <c>run.<timestamp></c> directory is chosen.</d> </type> <desc> - <p>Analyse cover data collected from all tests. The modules - analysed are the ones listed in the cross cover file - <c>cross.cover</c> in the current directory of the test - server.</p> - <p>The modules listed in the <c>cross.cover</c> file are - modules that are heavily used by other applications than the - one they belong to. This function should be run after all - tests are completed, and the result will be stored in a file - called cross_cover.html in the run.<timestamp> - directory of the application the modules belong to. - </p> - <p>The <c>cross.cover</c> file contains elements like this:</p> - <pre> -{App,Modules}. </pre> - <p>where <c>App</c> can be an application name or the atom - <c>all</c>. The application (or all applications) will cover - compile the listed <c>Modules</c>. - </p> + <p>Analyse cover data collected from multiple tests. The modules + analysed are the ones listed in <c>cross</c> statements in + the cover files. These are modules that are heavily used by + other tests than the one where they belong or are explicitly + tested. They should then be listed as cross modules in the + cover file for the test where they are used but do not + belong. Se example below.</p> + <p>This function should be run after all tests are completed, + and the result will be stored in a file called + <c>cross_cover.html</c> in the <c>run.<timestamp></c> + directory of the test the modules belong to.</p> + <p>Note that the function can be executed on any node, and it + does not require <c>test_server_ctrl</c> to be started first.</p> + <p>The <c>cross</c> statement in the cover file must be like this:</p> + <code type="none"> +{cross,[{Tag,Modules}]}.</code> + <p>where <c>Tag</c> is the same as <c>Tag</c> in the + <c>Tests</c> parameter to this function and <c>Modules</c> is a + list of module names (atoms).</p> + <p><em>Example:</em></p> + <p>If the module <c>m1</c> belongs to system <c>s1</c> but is + heavily used also in the tests for another system <c>s2</c>, + then the cover files for the two systems' tests could be like + this:</p> +<code type="none"> +s1.cover: + {include,[m1]}. + +s2.cover: + {include,[....]}. % modules belonging to system s2 + {cross,[{s1,[m1]}]}.</code> + <p>When the tests for both <c>s1</c> and <c>s2</c> are completed, run</p> +<code type="none"> +test_server_ctrl:cross_cover_analyse(Level,[{s1,S1LogDir},{s2,S2LogDir}]) +</code> + + <p>and the accumulated cover data for <c>m1</c> will be written to + <c>S1LogDir/[run.<timestamp>/]cross_cover.html</c>.</p> + <p>Note that the <c>m1</c> module will also be presented in the + normal coverage log for <c>s1</c> (due to the include statement in + <c>s1.cover</c>), but that only includes the coverage achieved by the + <c>s1</c> test itself.</p> + <p>The Tag in the <c>cross</c> statement in the cover file has + no other purpose than mapping the list of modules + (<c>[m1]</c> in the example above) to the correct log + directory where it should be included in the + <c>cross_cover.html</c> file (<c>S1LogDir</c> in the example + above). I.e. the value of <c>Tag</c> has no meaning, it + could be <c>foo</c> as well as <c>s1</c> above, as long as + the same <c>Tag</c> is used in the cover file and in the + call to this function.</p> </desc> </func> <func> @@ -634,10 +640,6 @@ Optional, if not given the test server controller node default name. This does not apply to <c>SPEC</c> which keeps its names. </item> - <tag><c>PARAMETERS parameterfile</c></tag> - <item>Specifies the parameter file to use when starting - remote target - </item> <tag><c>COVER app cover_file analyse</c></tag> <item>Indicates that the test should be run with cover analysis. <c>app</c>, <c>cover_file</c> and <c>analyse</c> diff --git a/lib/test_server/doc/src/test_spec_chapter.xml b/lib/test_server/doc/src/test_spec_chapter.xml index 3a7730d61e..c197e03bed 100644 --- a/lib/test_server/doc/src/test_spec_chapter.xml +++ b/lib/test_server/doc/src/test_spec_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2002</year><year>2009</year> + <year>2002</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -99,8 +99,7 @@ function fails, all tests in the test suite are skipped. The difference between this "make case" and a normal conf case is that for the make case, <c>Init</c> and <c>Fin</c> are given with - arguments (<c>{Mod,Func,Args}</c>), and that they are executed - on the controller node (i.e. not on target). + arguments (<c>{Mod,Func,Args}</c>). </item> <tag><c>Case</c></tag> <item>This can only be used inside a module, i.e. not a test diff --git a/lib/test_server/doc/src/ts.xml b/lib/test_server/doc/src/ts.xml index f9b48d8372..82ba3a5017 100644 --- a/lib/test_server/doc/src/ts.xml +++ b/lib/test_server/doc/src/ts.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2007</year> - <year>2011</year> + <year>2012</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -85,8 +85,7 @@ <p><c>ts:install/1</c> or <c>ts:install/2</c> is used if the target platform is different from the controller host, i.e. if you run on "remote target" or if special options are required - for your system. VxWorks is currently supported - as remote target platform. + for your system. </p> <p>See the reference manual for detailed information about <c>ts:install/0/1/2</c>. @@ -249,9 +248,8 @@ <p>Installs and configures the Test Server Framework for running test suites. If a remote host is to be used, the <c>TargetSystem</c> argument must be given so that "cross - installation" can be done. This should be used for testing on - VxWorks. Installation is required for any of the - functions in <c>ts</c> to work. + installation" can be done. Installation is required for + any of the functions in <c>ts</c> to work. </p> <p>Opts may be one or more of </p> @@ -452,7 +450,7 @@ This option is mandatory for remote targets <desc> <p>Analyse cover data collected from all tests. </p> - <p>See test_server_ctrl:cross_cover_analyse/1 + <p>See test_server_ctrl:cross_cover_analyse/2 </p> </desc> </func> @@ -500,29 +498,6 @@ This option is mandatory for remote targets </desc> </func> <func> - <name>index() -> ok | {error, Reason}</name> - <fsummary>Updates local index page</fsummary> - <type> - <v>Reason = term()</v> - </type> - <desc> - <p>This function updates the local index page. This can be - useful if a previous test run was not completed and the index - is incomplete.</p> - </desc> - </func> - <func> - <name>clean() -> ok</name> - <name>clean(all) -> ok</name> - <fsummary>Cleans up the log directories created when running tests. </fsummary> - <desc> - <p>This function cleans up log directories created when - running test cases. <c>clean/0</c> cleans up all but the last - run of each application. <c>clean/1</c> cleans up all test - runs found.</p> - </desc> - </func> - <func> <name>estone() -> ok | {error, Reason}</name> <name>estone(Opts) -> ok</name> <fsummary>Runs the EStone test</fsummary> diff --git a/lib/test_server/doc/src/write_framework_chapter.xml b/lib/test_server/doc/src/write_framework_chapter.xml index 8a20e9afec..5c1c657d9e 100644 --- a/lib/test_server/doc/src/write_framework_chapter.xml +++ b/lib/test_server/doc/src/write_framework_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2002</year><year>2009</year> + <year>2002</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -46,15 +46,8 @@ <title>Interfacing the test server controller from Erlang</title> <p>Using the test server from Erlang means that you have to start the test server and then add test jobs. Use - <c>test_server_ctrl:start/0</c> to start a local target or - <c>test_server_ctrl:start/1</c> to start a remote target. The test - server is stopped by <c>test_server_ctrl:stop/0</c>. - </p> - <p>The argument to <c>test_server_ctrl:start/1</c> is the name of a - parameter file. The parameter file specifies what type of target - to start and where to start it, as well as some additional - parameters needed for different target types. See the reference - manual for a detailed description of all valid parameters. + <c>test_server_ctrl:start/0</c> to start the test server, and + <c>test_server_ctrl:stop/0</c> to stop it. </p> <section> diff --git a/lib/test_server/doc/src/write_test_chapter.xml b/lib/test_server/doc/src/write_test_chapter.xml index 12f0dfc361..086ed2a18d 100644 --- a/lib/test_server/doc/src/write_test_chapter.xml +++ b/lib/test_server/doc/src/write_test_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2002</year><year>2009</year> + <year>2002</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -130,7 +130,6 @@ case. If the test specification is an empty list, this indicates that the test case is a leaf test case, i.e. one to be executed. </p> - <p><em>Note that the specification clause of a test case is executed on the test server controller host. This means that if target is remote, the specification clause is probably executed on a different platform than the one tested.</em></p> <p>The execution clause implements the actual test case. It takes one argument, <c>Config</c>, which contain configuration information like <c>data_dir</c> and <c>priv_dir</c>. See <seealso marker="#data_priv_dir">Data and Private Directories</seealso> for |