diff options
Diffstat (limited to 'lib/common_test/doc')
| -rw-r--r-- | lib/common_test/doc/src/ct_hooks_chapter.xml | 28 | ||||
| -rw-r--r-- | lib/common_test/doc/src/notes.xml | 93 | ||||
| -rw-r--r-- | lib/common_test/doc/src/run_test_chapter.xml | 50 |
3 files changed, 163 insertions, 8 deletions
diff --git a/lib/common_test/doc/src/ct_hooks_chapter.xml b/lib/common_test/doc/src/ct_hooks_chapter.xml index a706bbf9e6..60cd9be918 100644 --- a/lib/common_test/doc/src/ct_hooks_chapter.xml +++ b/lib/common_test/doc/src/ct_hooks_chapter.xml @@ -322,6 +322,34 @@ post_end_per_testcase(_TC, Config, Return, CTHState) -> </section> + <marker id="synchronizing"/> + <section> + <title>Synchronizing external user applications with Common Test</title> + <p>CTHs can be used to synchronize test runs with external user applications. + The init function may e.g. start and/or communicate with an application that + has the purpose of preparing the SUT for an upcoming test run, or maybe + initialize a database for saving test data to during the test run. The + terminate function may similarly order such an application to reset the SUT + after the test run, and/or tell the application to finish active sessions + and terminate. + Any system error- or progress reports generated during the init- or + termination stage will be saved in the + <seealso marker="run_test_chapter#pre_post_test_io_log">Pre- + and post test I/O log</seealso>. (This is also true for any printouts made + with <c>ct:log/2</c> and <c>ct:pal/2</c>).</p> + <p>In order to ensure that Common Test doesn't start executing tests, or + closes its log files and shuts down, before the external application + is ready for it, Common Test may be synchronized with the application. + During startup and shutdown, Common Test can be suspended, simply by + having a CTH evaluate a <c>receive</c> expression in the init- or terminate + function. The macros <c>?CT_HOOK_INIT_PROCESS</c> (the process executing the hook + init function) and <c>?CT_HOOK_TERMINATE_PROCESS</c> (the process executing + the hook terminate function), each specifies the name of the correct Common Test + process to send a message to in order to return from the <c>receive</c>. + These macros are defined in <c>ct.hrl</c>. + </p> + </section> + <marker id="example"/> <section> <title>Example CTH</title> diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 2d6bcc0d8b..fce5401f13 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -32,6 +32,99 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.7.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Documentation is added for ct_netconfc:send and + ct_netconfc:send_rpc.</p> + <p> + Own Id: OTP-11132</p> + </item> + <item> + <p> + ct_netconfc:create_subscription only allowed one XML + element inside the 'filter' element. According to RFC5277 + it should be allowed to add any number of elements inside + the filter, so this is now corrected.</p> + <p> + Own Id: OTP-11166</p> + </item> + <item> + <p> + The error handler installed by the Common Test hook + cth_log_redirect did not respond to init:stop/1/2. This + has been corrected.</p> + <p> + Own Id: OTP-11175 Aux Id: seq12356 </p> + </item> + <item> + <p> + Calling ct:pal/2 or ct:print/2 when Common Test was not + running, would cause an exit. This has been changed and + the string is now simply printed to stdout instead.</p> + <p> + Own Id: OTP-11176</p> + </item> + <item> + <p> + Fixed problem with the cth_log_redirect hook making calls + to an undefined function in ct_logs.</p> + <p> + Own Id: OTP-11238</p> + </item> + <item> + <p> + When running tests with the 'repeat' option, the Common + Test utility process did not always terminate quickly + enough after a test run, causing the start of the next + run to fail. A monitor is now used to ensure termination + of the utility process after each test run.</p> + <p> + Own Id: OTP-11244 Aux Id: seq12396 </p> + </item> + <item> + <p> + Test Server installed an error handler (test_server_h) + only to be able to write the name of the current test + case to stdout whenever it received an error- or progress + report. This functionality was not useful and has been + removed. The built-in Common Test hook, cth_log_redirect, + has instead been improved to now also tag all error- and + progress reports in the log with suite-, group-, and/or + test case name.</p> + <p> + Own Id: OTP-11263 Aux Id: seq12251 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + A new log, the "Pre- and Post Test I/O Log", has been + introduced, which makes it possible to capture error- and + progress reports, as well as printouts made with ct:log/2 + and ct:pal/2, before and after a test run. (Some minor + improvements of the logging system have been made at the + same time). Links to the new log are found on the Common + Test Framework Log page. The Common Test User's Guide has + been updated with information about the new log and also + with a new section on how to synchronize external + applications with Common Test by means of the CT Hook + init and terminate functions.</p> + <p> + Own Id: OTP-11272</p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.7.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index 657a72ef8c..6b37e183dd 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -21,7 +21,7 @@ </legalnotice> - <title>Running Tests</title> + <title>Running Tests and Analyzing Results</title> <prepared>Peter Andersson, Kenneth Lundin</prepared> <docno></docno> <date></date> @@ -1087,18 +1087,18 @@ </list> <p>On the test run index page there is a link to the Common Test - Framework log file in which information about imported - configuration data and general test progress is written. This - log file is useful to get snapshot information about the test - run during execution. It can also be very helpful when - analyzing test results or debugging test suites.</p> + Framework Log file in which information about imported + configuration data and general test progress is written. This + log file is useful to get snapshot information about the test + run during execution. It can also be very helpful when + analyzing test results or debugging test suites.</p> <p>On the test run index page it is noted if a test has missing suites (i.e. suites that Common Test has failed to compile). Names of the missing suites can be found in the - Common Test Framework log file.</p> + Common Test Framework Log file.</p> - <p>The major logfile shows a detailed report of the test run. It + <p>The major log file shows a detailed report of the test run. It includes test suite and test case names, execution time, the exact reason for failures etc. The information is available in both a file with textual and with HTML representation. The HTML file shows a @@ -1172,6 +1172,40 @@ <url href="http://tablesorter.com">tablesorter</url> plugin, with customized sorting functions, for this implementation.</p> </section> + + <section> + <title>The Unexpected I/O Log</title> + <p>On the test suites overview page you find a link to the Unexpected I/O Log. + In this log, Common Test saves printouts made with + <c>ct:log/2</c> and <c>ct:pal/2</c>, as well as captured system error- and + progress reports, that cannot be associated with particular test cases and + therefore cannot be written to individual test case log files. This happens e.g. + if a log printout is made from an external process (not a test case process), + or if an error- or progress report comes in, during a short interval while Common + Test is not executing a test case or configuration function, <em>or</em> while + Common Test is currently executing a parallell test case group.</p> + </section> + + <section> + <marker id="pre_post_test_io_log"></marker> + <title>The Pre- and Post Test I/O Log</title> + <p>On the Common Test Framework Log page you find links to the so called + Pre- and Post Test I/O Log. In this log, Common Test saves printouts made with + <c>ct:log/2</c> and <c>ct:pal/2</c>, as well as captured system error- + and progress reports, that take place before - and after - the actual test run. + Examples of this are printouts from a CT hook init- or terminate function, or + progress reports generated when an OTP application is started from a CT hook + init function. Another example is an error report generated due to + a failure when an external application is stopped from a CT hook terminate function. + All information in these examples ends up in the Pre- and Post Test I/O Log. + For more information on how to synchronize test runs with external user + applications, please see the + <seealso marker="ct_hooks_chapter#synchronizing">Synchronizing</seealso> + section in the Common Test Hooks chapter.</p> + <p>Note that logging to file with <c>ct:log/2</c> or <c>ct:pal/2</c> + only works when Common Test is running. Printouts with <c>ct:pal/2</c> + are however always displayed on screen.</p> + </section> </section> <section> |