diff options
Diffstat (limited to 'lib/common_test/doc/src')
| -rw-r--r-- | lib/common_test/doc/src/cover_chapter.xml | 87 | ||||
| -rw-r--r-- | lib/common_test/doc/src/notes.xml | 116 |
2 files changed, 202 insertions, 1 deletions
diff --git a/lib/common_test/doc/src/cover_chapter.xml b/lib/common_test/doc/src/cover_chapter.xml index b2e64bfff0..4fa92d5583 100644 --- a/lib/common_test/doc/src/cover_chapter.xml +++ b/lib/common_test/doc/src/cover_chapter.xml @@ -108,8 +108,8 @@ specifications</seealso>).</p> </section> + <marker id="cover_stop"></marker> <section> - <marker id="cover_stop"></marker> <title>Stopping the cover tool when tests are completed</title> <p>By default the Cover tool is automatically stopped when the tests are completed. This causes the original (non cover @@ -175,6 +175,11 @@ %% Specific modules to exclude in cover. {excl_mods, Mods}. + + %% Cross cover compilation + %% Tag = atom(), an identifier for a test run + %% Mod = [atom()], modules to compile for accumulated analysis + {cross,[{Tag,Mods}]}. </pre> <p>The <c>incl_dirs_r</c> and <c>excl_dirs_r</c> terms tell Common @@ -190,6 +195,81 @@ specification file for Common Test).</p> </section> + <marker id="cross_cover"/> + <section> + <title>Cross cover analysis</title> + <p>The cross cover mechanism allows cover analysis of modules + across multiple tests. It is useful if some code, e.g. a library + module, is used by many different tests and the accumulated cover + result is desirable.</p> + + <p>This can of course also be achieved in a more customized way by + using the <c>export</c> parameter in the cover specification and + analysing the result off line, but the cross cover mechanism is a + build in solution which also provides the logging.</p> + + <p>The mechanism is easiest explained via an example:</p> + + <p>Let's say that there are two systems, <c>s1</c> and <c>s2</c>, + which are tested in separate test runs. System <c>s1</c> contains + a library module <c>m1</c> which is tested by the <c>s1</c> test + run and is included in <c>s1</c>'s cover specification:</p> + +<code type="none"> +s1.cover: + {incl_mods,[m1]}.</code> + + <p>When analysing code coverage, the result for <c>m1</c> can be + seen in the cover log in the <c>s1</c> test result.</p> + + <p>Now, let's imagine that since <c>m1</c> is a library module, it + is also used quite a bit by system <c>s2</c>. The <c>s2</c> test + run does not specifically test <c>m1</c>, but it might still be + interesting to see which parts of <c>m1</c> is actually covered by + the <c>s2</c> tests. To do this, <c>m1</c> could be included also + in <c>s2</c>'s cover specification:</p> + +<code type="none"> +s2.cover: + {incl_mods,[m1]}.</code> + + <p>This would give an entry for <c>m1</c> also in the cover log + for the <c>s2</c> test run. The problem is that this would only + reflect the coverage by <c>s2</c> tests, not the accumulated + result over <c>s1</c> and <c>s2</c>. And this is where the cross + cover mechanism comes in handy.</p> + + <p>If instead the cover specification for <c>s2</c> was like + this:</p> + +<code type="none"> +s2.cover: + {cross,[{s1,[m1]}]}.</code> + + <p>then <c>m1</c> would be cover compiled in the <c>s2</c> test + run, but not shown in the coverage log. Instead, if + <c>ct_cover:cross_cover_analyse/2</c> is called after both + <c>s1</c> and <c>s2</c> test runs are completed, the accumulated + result for <c>m1</c> would be available in the cross cover log for + the <c>s1</c> test run.</p> + + <p>The call to the analyse function must be like this:</p> + +<code type="none"> +ct_cover:cross_cover_analyse(Level, [{s1,S1LogDir},{s2,S2LogDir}]).</code> + + <p>where <c>S1LogDir</c> and <c>S2LogDir</c> are the directories + named <c><TestName>.logs</c> for each test respectively.</p> + + <p>Note the tags <c>s1</c> and <c>s2</c> which are used in the + cover specification file and in the call to + <c>ct_cover:cross_cover_analyse/2</c>. The point of these are only + to map the modules specified in the cover specification to the log + directory specified in the call to the analyse function. The name + of the tag has no meaning beyond this.</p> + + </section> + <section> <title>Logging</title> <p>To view the result of a code coverage test, follow the @@ -197,6 +277,11 @@ takes you to the code coverage overview page. If you have successfully performed a detailed coverage analysis, you find links to each individual module coverage page here.</p> + + <p>If cross cover analysis has been performed, and there are + accumulated coverage results for the current test, then the - + "Coverdata collected over all tests" link will take you to these + results.</p> </section> </chapter> diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 8c3b13951d..0345fab8e8 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -32,6 +32,122 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Severe errors detected by <c>test_server</c> (e.g. if log + files directories cannot be created) will now be reported + to <c>common_test</c> and noted in the <c>common_test</c> + logs.</p> + <p> + Own Id: OTP-9769 Aux Id: kunagi-202 [113] </p> + </item> + <item> + <p> + If a busy test case generated lots of error messages, + cth_log_redirect:post_end_per_testcase would crash with a + timeout while waiting for the error logger to finish + handling all error reports. The default timer was 5 + seconds. This has now been extended to 5 minutes.</p> + <p> + Own Id: OTP-10040 Aux Id: kunagi-173 [84] </p> + </item> + <item> + <p> + Some bugfixes in <c>ct_snmp:</c></p> + <p> + <list> <item> ct_snmp will now use the value of the + 'agent_vsns' config variable when setting the 'variables' + parameter to snmp application agent configuration. + Earlier this had to be done separately - i.e. the + supported versions had to be specified twice. </item> + <item> Snmp application failed to write notify.conf since + ct_snmp gave the notify type as a string instead of an + atom. This has been corrected. </item> </list></p> + <p> + Own Id: OTP-10432</p> + </item> + <item> + <p> + Some bugfixes in <c>ct_snmp</c>:</p> + <p> + <list> <item> Functions <c>register_users/2</c>, + <c>register_agents/2</c> and <c>register_usm_users/2</c>, + and the corresponding <c>unregister_*/1</c> functions + were not executable. These are corrected/rewritten. + </item> <item> Function <c>update_usm_users/2</c> is + removed, and an unregister function is added instead. + Update can now be done with unregister_usm_users and then + register_usm_users. </item> <item> Functions + <c>unregister_*/2</c> are added, so specific + users/agents/usm users can be unregistered. </item> + <item> Function <c>unload_mibs/1</c> is added for + completeness. </item> <item> Overriding configuration + files did not work, since the files were written in + priv_dir instead of in the configuration dir + (priv_dir/conf). This has been corrected. </item> <item> + Arguments to <c>register_usm_users/2</c> were faulty + documented. This has been corrected. </item> </list></p> + <p> + Own Id: OTP-10434 Aux Id: kunagi-264 [175] </p> + </item> + <item> + <p> + Faulty exported specs in common test has been corrected + to <c>ct_netconfc:hook_options/0</c> and + <c>inet:hostname/0</c></p> + <p> + Own Id: OTP-10601</p> + </item> + <item> + <p> + The netconf client in common_test did not adjust the + window after receiving data. Due to this, the client + stopped receiving data after a while. This has been + corrected.</p> + <p> + Own Id: OTP-10646</p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + The earlier undocumented cross cover feature for + accumulating cover data over multiple tests has now been + fixed and documented.</p> + <p> + Own Id: OTP-9870 Aux Id: kunagi-206 [117] </p> + </item> + <item> + <p> + CT drops error reason when groups/0 crashes.</p> + <p> + Own Id: OTP-10631 Aux Id: kunagi-345 [256] </p> + </item> + <item> + <p> + Problem opening sftp connection with ct_ssh.</p> + <p> + Own Id: OTP-10632 Aux Id: kunagi-346 [257] </p> + </item> + <item> + <p> + Event handler on a ct_master node causes hanging.</p> + <p> + Own Id: OTP-10634 Aux Id: kunagi-347 [258] </p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.6.3.1</title> <section><title>Known Bugs and Problems</title> |