Merge branch 'peppe/common_test/misc_r16b02_docs' into maint

* peppe/common_test/misc_r16b02_docs:
  Add misc documentation

OTP-11272
OTP-11263
OTP-11244
OTP-11238
OTP-11176

4 files changed, 84 insertions, 15 deletions

diff --git a/lib/common_test/doc/src/ct_hooks_chapter.xml b/lib/common_test/doc/src/ct_hooks_chapter.xml
index a706bbf9e6..60cd9be918 100644
--- a/lib/common_test/doc/src/ct_hooks_chapter.xml
+++ b/lib/common_test/doc/src/ct_hooks_chapter.xml
@@ -322,6 +322,34 @@ post_end_per_testcase(_TC, Config, Return, CTHState) ->
 
   </section>
 
+  <marker id="synchronizing"/>
+  <section>
+    <title>Synchronizing external user applications with Common Test</title>
+    <p>CTHs can be used to synchronize test runs with external user applications.
+    The init function may e.g. start and/or communicate with an application that
+    has the purpose of preparing the SUT for an upcoming test run, or maybe
+    initialize a database for saving test data to during the test run. The
+    terminate function may similarly order such an application to reset the SUT
+    after the test run, and/or tell the application to finish active sessions
+    and terminate.
+    Any system error- or progress reports generated during the init- or
+    termination stage will be saved in the 
+    <seealso marker="run_test_chapter#pre_post_test_io_log">Pre-
+    and post test I/O log</seealso>. (This is also true for any printouts made
+    with <c>ct:log/2</c> and <c>ct:pal/2</c>).</p>
+    <p>In order to ensure that Common Test doesn't start executing tests, or
+    closes its log files and shuts down, before the external application
+    is ready for it, Common Test may be synchronized with the application. 
+    During startup and shutdown, Common Test can be suspended, simply by
+    having a CTH evaluate a <c>receive</c> expression in the init- or terminate
+    function. The macros <c>?CT_HOOK_INIT_PROCESS</c> (the process executing the hook
+    init function) and <c>?CT_HOOK_TERMINATE_PROCESS</c> (the process executing
+    the hook terminate function), each specifies the name of the correct Common Test
+    process to send a message to in order to return from the <c>receive</c>.
+    These macros are defined in <c>ct.hrl</c>.
+    </p>
+  </section>
+
   <marker id="example"/>
   <section>
      <title>Example CTH</title>
diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml
index 657a72ef8c..6b37e183dd 100644
--- a/lib/common_test/doc/src/run_test_chapter.xml
+++ b/lib/common_test/doc/src/run_test_chapter.xml
@@ -21,7 +21,7 @@
 
     </legalnotice>
 
-    <title>Running Tests</title>
+    <title>Running Tests and Analyzing Results</title>
     <prepared>Peter Andersson, Kenneth Lundin</prepared>
     <docno></docno>
     <date></date>
@@ -1087,18 +1087,18 @@
       </list>
       
       <p>On the test run index page there is a link to the Common Test
-	Framework log file in which information about imported
-	configuration data and general test progress is written. This
-	log file is useful to get snapshot information about the test
-	run during execution. It can also be very helpful when
-	analyzing test results or debugging test suites.</p>
+        Framework Log file in which information about imported
+        configuration data and general test progress is written. This
+        log file is useful to get snapshot information about the test
+        run during execution. It can also be very helpful when
+        analyzing test results or debugging test suites.</p>
 
       <p>On the test run index page it is noted if a test has missing
         suites (i.e. suites that Common Test has failed to
         compile). Names of the missing suites can be found in the
-        Common Test Framework log file.</p>
+        Common Test Framework Log file.</p>
 
-      <p>The major logfile shows a detailed report of the test run. It
+      <p>The major log file shows a detailed report of the test run. It
         includes test suite and test case names, execution time, the 
 	exact reason for failures etc. The information is available in both
 	a file with textual and with HTML representation. The HTML file shows a 
@@ -1172,6 +1172,40 @@
 	    <url href="http://tablesorter.com">tablesorter</url> plugin, with customized sorting
 	    functions, for this implementation.</p>
 	</section>
+
+	<section>
+	  <title>The Unexpected I/O Log</title>
+	  <p>On the test suites overview page you find a link to the Unexpected I/O Log.
+	  In this log, Common Test saves printouts made with
+	  <c>ct:log/2</c> and <c>ct:pal/2</c>, as well as captured system error- and
+	  progress reports, that cannot be associated with particular test cases and
+	  therefore cannot be written to individual test case log files. This happens e.g.
+	  if a log printout is made from an external process (not a test case process),
+	  or if an error- or progress report comes in, during a short interval while Common
+	  Test is not executing a test case or configuration function, <em>or</em> while
+	  Common Test is currently executing a parallell test case group.</p>
+	</section>
+
+	<section>
+	  <marker id="pre_post_test_io_log"></marker>
+	  <title>The Pre- and Post Test I/O Log</title>
+	  <p>On the Common Test Framework Log page you find links to the so called
+	  Pre- and Post Test I/O Log. In this log, Common Test saves printouts made with
+	  <c>ct:log/2</c> and <c>ct:pal/2</c>, as well as captured system error-
+	  and progress reports, that take place before - and after - the actual test run.
+	  Examples of this are printouts from a CT hook init- or terminate function, or
+	  progress reports generated when an OTP application is started from a CT hook
+	  init function. Another example is an error report generated due to
+	  a failure when an external application is stopped from a CT hook terminate function.
+	  All information in these examples ends up in the Pre- and Post Test I/O Log.
+	  For more information on how to synchronize test runs with external user
+	  applications, please see the
+	  <seealso marker="ct_hooks_chapter#synchronizing">Synchronizing</seealso>
+	  section in the Common Test Hooks chapter.</p>
+	  <p>Note that logging to file with <c>ct:log/2</c> or <c>ct:pal/2</c>
+	  only works when Common Test is running. Printouts with <c>ct:pal/2</c>
+	  are however always displayed on screen.</p>
+	</section>
       </section>
   
       <section>
diff --git a/lib/common_test/include/ct.hrl b/lib/common_test/include/ct.hrl
index bde2709ad1..44cc33f01e 100644
--- a/lib/common_test/include/ct.hrl
+++ b/lib/common_test/include/ct.hrl
@@ -32,3 +32,7 @@
 -define(STD_VERBOSITY, 50 ).
 -define(HI_VERBOSITY,  75 ).
 -define(MAX_VERBOSITY, 100).
+
+%% name of process executing the CT Hook init and terminate function
+-define(CT_HOOK_INIT_PROCESS, ct_util_server).
+-define(CT_HOOK_TERMINATE_PROCESS, ct_util_server).
diff --git a/lib/common_test/test/ct_pre_post_test_io_SUITE_data/cth_ctrl.erl b/lib/common_test/test/ct_pre_post_test_io_SUITE_data/cth_ctrl.erl
index a9ea7b14dd..c8c08a5735 100644
--- a/lib/common_test/test/ct_pre_post_test_io_SUITE_data/cth_ctrl.erl
+++ b/lib/common_test/test/ct_pre_post_test_io_SUITE_data/cth_ctrl.erl
@@ -21,6 +21,8 @@
 -export([proceed/0,
 	 init/2, terminate/1]).
 
+-include_lib("common_test/include/ct.hrl").
+
 %%%===================================================================
 %%% API
 %%%===================================================================
@@ -39,12 +41,12 @@ init(_Id, _Opts) ->
 	    ok
     end,
     WhoAmI = self(),
+    WhoAmI = whereis(?CT_HOOK_INIT_PROCESS),
     DispPid = spawn_link(fun() -> dispatcher(WhoAmI) end),
     register(?MODULE, DispPid),
-    io:format(user,
-	      "~n~n+++ Startup of ~w on ~p finished, "
-	      "call ~w:proceed() to run tests...~n",
-	      [?MODULE,node(),?MODULE]),
+    ct:pal("~n~n+++ Startup of ~w on ~p finished, "
+	   "call ~w:proceed() to run tests...~n",
+	   [?MODULE,node(),?MODULE]),
     start_external_logger(cth_logger),
     receive
 	{?MODULE,proceed} -> ok
@@ -55,9 +57,10 @@ init(_Id, _Opts) ->
     {ok,[],ct_last}.
 
 terminate(_State) ->
-    io:format(user,
-	      "~n~n+++ Tests finished, call ~w:proceed() to shut down...~n",
-	      [?MODULE]),
+    WhoAmI = whereis(?CT_HOOK_TERMINATE_PROCESS),
+    WhoAmI = self(),
+    ct:pal("~n~n+++ Tests finished, call ~w:proceed() to shut down...~n",
+	   [?MODULE]),
     receive
 	{?MODULE,proceed} -> ok
     after