Although Common Test was created primarly for the purpose of black box testing, nothing prevents it from working perfectly as a white box testing tool as well. This is especially true when the application to test is written in Erlang. Then the test ports are easily realized by means of Erlang function calls.
When white box testing an Erlang application, it is useful to be able to measure the code coverage of the test. Common Test provides simple access to the OTP Cover tool for this purpose. Common Test handles all necessary communication with the Cover tool (starting, compiling, analysing, etc). All the Common Test user needs to do is to specify the extent of the code coverage analysis.
To specify what modules should be included in the code coverage test, you provide a cover specification file. Using this file you can point out specific modules or specify directories that contain modules which should all be included in the analysis. You can also, in the same fashion, specify modules that should be excluded from the analysis.
If you are testing a distributed Erlang application, it is
likely that code you want included in the code coverage analysis
gets executed on an Erlang node other than the one Common Test
is running on. If this is the case you need to specify these
other nodes in the cover specification file or add them
dynamically to the code coverage set of nodes. See the
In the cover specification file you can also specify your
required level of the code coverage analysis;
You can choose to export and import code coverage data between tests. If you specify the name of an export file in the cover specification file, Common Test will export collected coverage data to this file at the end of the test. You may similarly specify that previously exported data should be imported and included in the analysis for a test (you can specify multiple import files). This way it is possible to analyse total code coverage without necessarily running all tests at once.
To activate the code coverage support, you simply specify the
name of the cover specification file as you start Common Test.
This you do either by using the
You may also pass the cover specification file name in a
call to
By default the Cover tool is automatically stopped when the
tests are completed. This causes the original (non cover
compiled) modules to be loaded back in to the test node. If a
process at this point is still running old code of any of the
modules that are cover compiled, meaning that it has not done
any fully qualified function call after the cover compilation,
the process will now be killed. To avoid this it is possible to
set the value of the
The option can be set by using the
These are the terms allowed in a cover specification file:
%% List of Nodes on which cover will be active during test. %% Nodes = [atom()] {nodes, Nodes}. %% Files with previously exported cover data to include in analysis. %% CoverDataFiles = [string()] {import, CoverDataFiles}. %% Cover data file to export from this session. %% CoverDataFile = string() {export, CoverDataFile}. %% Cover analysis level. %% Level = details | overview {level, Level}. %% Directories to include in cover. %% Dirs = [string()] {incl_dirs, Dirs}. %% Directories, including subdirectories, to include. {incl_dirs_r, Dirs}. %% Specific modules to include in cover. %% Mods = [atom()] {incl_mods, Mods}. %% Directories to exclude in cover. {excl_dirs, Dirs}. %% Directories, including subdirectories, to exclude. {excl_dirs_r, Dirs}. %% Specific modules to exclude in cover. {excl_mods, Mods}. %% Cross cover compilation %% Tag = atom(), an identifier for a test run %% Mod = [atom()], modules to compile for accumulated analysis {cross,[{Tag,Mods}]}.
The
Note: Directories containing Erlang modules that are to be included in a code coverage test must exist in the code server path, or the cover tool will fail to recompile the modules. (It is not sufficient to specify these directories in the cover specification file for Common Test).
The cross cover mechanism allows cover analysis of modules across multiple tests. It is useful if some code, e.g. a library module, is used by many different tests and the accumulated cover result is desirable.
This can of course also be achieved in a more customized way by
using the
The mechanism is easiest explained via an example:
Let's say that there are two systems,
s1.cover:
{incl_mods,[m1]}.
When analysing code coverage, the result for
Now, let's imagine that since
s2.cover:
{incl_mods,[m1]}.
This would give an entry for
If instead the cover specification for
s2.cover:
{cross,[{s1,[m1]}]}.
then
The call to the analyse function must be like this:
ct_cover:cross_cover_analyse(Level, [{s1,S1LogDir},{s2,S2LogDir}]).
where
Note the tags
To view the result of a code coverage test, click the button labled "COVER LOG" in the top level index page for the test run.
Prior to Erlang/OTP 17.1, if your test run consisted of multiple tests, cover would be started and stopped for each test within the test run. Separate logs would be available via the "Coverage log" link on the test suite result pages. These links are still available, but now they all point to the same page as the button on the top level index page. The log contains the accumulated results for the complete test run. See the release notes for more information about this change.
The buttonc takes you to the code coverage overview page. If you have successfully performed a detailed coverage analysis, you find links to each individual module coverage page here.
If cross cover analysis has been performed, and there are accumulated coverage results for the current test, then the - "Coverdata collected over all tests" link will take you to these results.