From 3f1db1cd7d1ba9cf77a10d767b2d82f2542e01a0 Mon Sep 17 00:00:00 2001 From: Siri Hansen Date: Tue, 4 Dec 2012 11:34:09 +0100 Subject: [common_test] Add documentation for cross cover analysis --- lib/common_test/doc/src/cover_chapter.xml | 87 ++++++++++++++++++++++++++++++- 1 file changed, 86 insertions(+), 1 deletion(-) (limited to 'lib/common_test/doc/src') diff --git a/lib/common_test/doc/src/cover_chapter.xml b/lib/common_test/doc/src/cover_chapter.xml index b2e64bfff0..4fa92d5583 100644 --- a/lib/common_test/doc/src/cover_chapter.xml +++ b/lib/common_test/doc/src/cover_chapter.xml @@ -108,8 +108,8 @@ specifications).

+
- Stopping the cover tool when tests are completed

By default the Cover tool is automatically stopped when the tests are completed. This causes the original (non cover @@ -175,6 +175,11 @@ %% Specific modules to exclude in cover. {excl_mods, Mods}. + + %% Cross cover compilation + %% Tag = atom(), an identifier for a test run + %% Mod = [atom()], modules to compile for accumulated analysis + {cross,[{Tag,Mods}]}.

The incl_dirs_r and excl_dirs_r terms tell Common @@ -190,6 +195,81 @@ specification file for Common Test).

+ +
+ Cross cover analysis +

The cross cover mechanism allows cover analysis of modules + across multiple tests. It is useful if some code, e.g. a library + module, is used by many different tests and the accumulated cover + result is desirable.

+ +

This can of course also be achieved in a more customized way by + using the export parameter in the cover specification and + analysing the result off line, but the cross cover mechanism is a + build in solution which also provides the logging.

+ +

The mechanism is easiest explained via an example:

+ +

Let's say that there are two systems, s1 and s2, + which are tested in separate test runs. System s1 contains + a library module m1 which is tested by the s1 test + run and is included in s1's cover specification:

+ + +s1.cover: + {incl_mods,[m1]}. + +

When analysing code coverage, the result for m1 can be + seen in the cover log in the s1 test result.

+ +

Now, let's imagine that since m1 is a library module, it + is also used quite a bit by system s2. The s2 test + run does not specifically test m1, but it might still be + interesting to see which parts of m1 is actually covered by + the s2 tests. To do this, m1 could be included also + in s2's cover specification:

+ + +s2.cover: + {incl_mods,[m1]}. + +

This would give an entry for m1 also in the cover log + for the s2 test run. The problem is that this would only + reflect the coverage by s2 tests, not the accumulated + result over s1 and s2. And this is where the cross + cover mechanism comes in handy.

+ +

If instead the cover specification for s2 was like + this:

+ + +s2.cover: + {cross,[{s1,[m1]}]}. + +

then m1 would be cover compiled in the s2 test + run, but not shown in the coverage log. Instead, if + ct_cover:cross_cover_analyse/2 is called after both + s1 and s2 test runs are completed, the accumulated + result for m1 would be available in the cross cover log for + the s1 test run.

+ +

The call to the analyse function must be like this:

+ + +ct_cover:cross_cover_analyse(Level, [{s1,S1LogDir},{s2,S2LogDir}]). + +

where S1LogDir and S2LogDir are the directories + named <TestName>.logs for each test respectively.

+ +

Note the tags s1 and s2 which are used in the + cover specification file and in the call to + ct_cover:cross_cover_analyse/2. The point of these are only + to map the modules specified in the cover specification to the log + directory specified in the call to the analyse function. The name + of the tag has no meaning beyond this.

+ +
+
Logging

To view the result of a code coverage test, follow the @@ -197,6 +277,11 @@ takes you to the code coverage overview page. If you have successfully performed a detailed coverage analysis, you find links to each individual module coverage page here.

+ +

If cross cover analysis has been performed, and there are + accumulated coverage results for the current test, then the - + "Coverdata collected over all tests" link will take you to these + results.

-- cgit v1.2.3