Improve cover analysis via common_test

This addresses several bugs in common_test (ct) when using the cover
analysis mechanism:

In a ct test run, one can give a cover spec file which indicates that
cover analysis shall be run, including a number of modules and a
number of nodes. During the ct test run, multiple jobs may be started
in test_server, and when the cover option is used, test_server would
cover compile and analyse all given modules for each job. This commit
instead allows the compilation and analysis to be explicitly ordered
by ct for each test run. This way each module will only be cover
compiled and analysed once. The cover log will be located in the
common_test log directory (ct_run.<timestamp>), and the "Coverage log"
link in each suite.log.html will point to this file. A new button is
also added in the top level ct index file, which points to the cover
log.

This change also reduces the need of using the 'export' and 'import'
options, since there is no longer any need to accumulate cover data
over multiple test_server jobs. However, these options may still be
used for importing and exporting cover data in order to store or
accumulate data from multiple ct test runs.

The 'nodes' option was earlier only used by ct to start cover on the
given nodes before starting the first test_server job. After this job
was completed, test_server would stop cover completely and then start
it again for the next job without any knowledge of the 'nodes'
options. For the next test_server jobs cover would therefore no longer
be running on these nodes. Explcit calls to ct_cover:add_nodes had to
be done in order to collect data through all test_server jobs. This
bug has now been solved, since cover is no longer stopped between each
test_server job.

Finally, ct no longer stores cover data using
ct_util:set_testdata. This was earlier used by ct_cover:add_nodes to
make sure no node was added twice.This did, however, cause some
problems when ct and cover were out of sync. ct could belive that a
node was running cover and thus reject adding this node, while in
reality cover had been stopped on the node (e.g. by test_server) so no
cover data was collected. ct_cover:add_nodes will now instead use
cover:which_nodes to check if a node is already running.

1 files changed, 16 insertions, 9 deletions

diff --git a/lib/common_test/doc/src/cover_chapter.xml b/lib/common_test/doc/src/cover_chapter.xml
index a215c8c2f3..accb94e1a9 100644
--- a/lib/common_test/doc/src/cover_chapter.xml
+++ b/lib/common_test/doc/src/cover_chapter.xml
@@ -4,7 +4,7 @@
 <chapter>
   <header>
     <copyright>
-      <year>2006</year><year>2013</year>
+      <year>2006</year><year>2014</year>
       <holder>Ericsson AB. All Rights Reserved.</holder>
     </copyright>
     <legalnotice>
@@ -81,10 +81,7 @@
       specify that previously exported data should be imported and
       included in the analysis for a test (you can specify multiple
       import files). This way it is possible to analyse total code coverage
-      without necessarily running all tests at once. Note that even if
-      you run separate tests in one test run, code coverage data will
-      not be passed on from one test to another unless you specify an
-      export file for Common Test to use for this purpose.</p>
+      without necessarily running all tests at once.</p>
 
     <p>To activate the code coverage support, you simply specify the
       name of the cover specification file as you start Common Test.
@@ -266,10 +263,20 @@ ct_cover:cross_cover_analyse(Level, [{s1,S1LogDir},{s2,S2LogDir}]).</code>
 
   <section>
     <title>Logging</title>
-    <p>To view the result of a code coverage test, follow the
-      "Coverage log" link on the test suite results page. This 
-      takes you to the code coverage overview page. If you have 
-      successfully performed a detailed coverage analysis, you 
+    <p>To view the result of a code coverage test, click the button
+      labled "COVER LOG" in the top level index page for the test run.</p>
+
+    <p>Prior to Erlang/OTP 17.1, if your test run consisted of
+      multiple tests, cover would be started and stopped for each test
+      within the test run. Separate logs would be available via the
+      "Coverage log" link on the test suite result pages. These links
+      are still available, but now they all point to the same page as
+      the button on the top level index page. The log contains the
+      accumulated results for the complete test run. See the release
+      notes for more information about this change.</p>
+
+    <p>The buttonc takes you to the code coverage overview page. If you
+      have successfully performed a detailed coverage analysis, you
       find links to each individual module coverage page here.</p>
 
     <p>If cross cover analysis has been performed, and there are