diff options
Diffstat (limited to 'lib/common_test/doc/src/notes.xml')
| -rw-r--r-- | lib/common_test/doc/src/notes.xml | 443 |
1 files changed, 442 insertions, 1 deletions
diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 8c3b13951d..2d6bcc0d8b 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2012</year> + <year>2004</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,447 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.7.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A design flaw in the generic connection handling in + Common Test made it impossible to implement a connection + handler that could map multiple connection names (i.e. + configuration variable aliases) to single connection + pids. This problem has been solved.</p> + <p> + Own Id: OTP-10126 Aux Id: kunagi-178 [89] </p> + </item> + <item> + <p> + If a telnet connection is hanging, then a call to + ct_telnet:close/1 will time out after 5 seconds and the + connection process is brutally killed. In some cases the + connection would not be unregistered and attempts at + opening a new connection with the same name would make + common_test try to reuse the same connection since it + believed that it was still alive. This has been corrected + - a killed connection is now always unregistered.</p> + <p> + Own Id: OTP-10648 Aux Id: seq12212 </p> + </item> + <item> + <p> + Test performance has been improved by means of a cache + for the top level HTML index logs (all_runs.html and + index.html, in the logdir directory). This solves + problems with slow start up times and test execution + times increasing with the number of ct_run directories + stored in logdir. The cached index entries are stored in + RAM during test execution and are saved to file in logdir + (for faster start up times) whenever a test run finishes.</p> + <p> + Own Id: OTP-10855</p> + </item> + <item> + <p> + Testing of the test specification functionality has been + improved and a couple of minor bugs have been discovered + and corrected.</p> + <p> + Own Id: OTP-10857</p> + </item> + <item> + <p> + Links to the top level index files in some HTML footers + had disappeared. This error has been corrected. Also, a + problem with the suite overview log file not being closed + properly has been solved.</p> + <p> + Own Id: OTP-11046</p> + </item> + <item> + <p> + Common Test would, in case of timetrap error, print a + warning in the log if end_per_testcase wasn't implemented + in the suite, even though it's an optional function. This + printout has been removed.</p> + <p> + Own Id: OTP-11052</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + If it could not be decided which test case a certain log + printout belonged to, the common test framework log was + earlier used. Such printouts are now instead sent to + unexpected_io.log.html in test_server so that there is + only one place to look for "missing" printouts.</p> + <p> + Own Id: OTP-10494 Aux Id: kunagi-319 [230] </p> + </item> + <item> + <p> + Make cover smarter about finding source from beam.</p> + <p> + In particular, search using the source path in + module_info if the current heuristic fails.</p> + <p> + Own Id: OTP-10902</p> + </item> + <item> + <p> + Add a variant of ct_slave:start/2 that starts a node with + specified options on the local host.</p> + <p> + Own Id: OTP-10920</p> + </item> + <item> + <p> + Integrate elliptic curve contribution from Andreas + Schultz </p> + <p> + In order to be able to support elliptic curve cipher + suites in SSL/TLS, additions to handle elliptic curve + infrastructure has been added to public_key and crypto.</p> + <p> + This also has resulted in a rewrite of the crypto API to + gain consistency and remove unnecessary overhead. All OTP + applications using crypto has been updated to use the new + API.</p> + <p> + Impact: Elliptic curve cryptography (ECC) offers + equivalent security with smaller key sizes than other + public key algorithms. Smaller key sizes result in + savings for power, memory, bandwidth, and computational + cost that make ECC especially attractive for constrained + environments.</p> + <p> + Own Id: OTP-11009</p> + </item> + <item> + <p> Postscript files no longer needed for the generation + of PDF files have been removed. </p> + <p> + Own Id: OTP-11016</p> + </item> + <item> + <p> + A link is added from the red error printout in a test + case log (for a failed test case) to the full error + description at the end of the log. The reason for this is + that the error description in the red field is sometimes + truncated at 50 characters in order to keep the log as + short and easy to read as possible.</p> + <p> + Own Id: OTP-11044 Aux Id: seq12304 </p> + </item> + <item> + <p> + A new option 'no_prompt_check' is added to + ct_telnet:expect/3. If this option is used, ct_telnet + will not wait for a prompt or a newline before attempting + to match the given pattern.</p> + <p> + Own Id: OTP-11095</p> + </item> + </list> + </section> + +</section> + +<section><title>Common_Test 1.7.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If an event handler installed in the CT Master event + manager took too long to respond during the termination + phase, CT Master crashed because of a timeout after 5 + secs. This would leave the system in a bad state. The + problem has been solved by means of a 30 min timeout + value and if CT Master gets a timeout after that time, it + now kills the event manager and shuts down properly.</p> + <p> + Own Id: OTP-10634 Aux Id: kunagi-347 [258] </p> + </item> + <item> + <p> + Printing with any of the ct printout functions from an + event handler installed by Common Test, would cause a + deadlock. This problem has been solved.</p> + <p> + Own Id: OTP-10826 Aux Id: seq12250 </p> + </item> + <item> + <p> + Using the force_stop flag/option to interrupt a test run + caused a crash in Common Test. This problem has been + solved.</p> + <p> + Own Id: OTP-10832</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Removed depricated run_test program, use ct_run instead.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-9052</p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + Test case execution time increases with size of test run.</p> + <p> + Own Id: OTP-10855</p> + </item> + </list> + </section> + +</section> + +<section><title>Common_Test 1.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Severe errors detected by <c>test_server</c> (e.g. if log + files directories cannot be created) will now be reported + to <c>common_test</c> and noted in the <c>common_test</c> + logs.</p> + <p> + Own Id: OTP-9769 Aux Id: kunagi-202 [113] </p> + </item> + <item> + <p> + The earlier undocumented cross cover feature for + accumulating cover data over multiple tests has now been + fixed and documented.</p> + <p> + Own Id: OTP-9870 Aux Id: kunagi-206 [117] </p> + </item> + <item> + <p> + If a busy test case generated lots of error messages, + cth_log_redirect:post_end_per_testcase would crash with a + timeout while waiting for the error logger to finish + handling all error reports. The default timer was 5 + seconds. This has now been extended to 5 minutes.</p> + <p> + Own Id: OTP-10040 Aux Id: kunagi-173 [84] </p> + </item> + <item> + <p>When a test case failed because of a timetrap time + out, the <c>Config</c> data for the case was lost in the + following call to <c>end_per_testcase/2</c>, and also in + calls to the CT Hook function + <c>post_end_per_testcase/4</c>. This problem has been + solved and the <c>Config</c> data is now correctly passed + to the above functions after a timetrap timeout + failure.</p> + <p> + Own Id: OTP-10070 Aux Id: kunagi-175 [86] </p> + </item> + <item> + <p> + Some calls to deprecated and removed functions in snmp + are removed from ct_snmp.</p> + <p> + Own Id: OTP-10088 Aux Id: kunagi-176 [87] </p> + </item> + <item> + <p>In test_server, the same process would supervise the + currently running test case and be group leader (and IO + server) for the test case. Furthermore, when running + parallel test cases, new temporary supervisor/group + leader processes were spawned and the process that was + group leader for sequential test cases would not be + active. That would lead to several problems:</p> + <p>* Processes started by init_per_suite will inherit the + group leader of the init_per_suite process (and that + group leader would not process IO requests when parallel + test cases was running). If later a parallel test case + caused such a processto print using (for example) + io:format/2, the calling would hang.</p> + <p>* Similarly, if a process was spawned from a parallel + test case, it would inherit the temporary group leader + for that parallel test case. If that spawned process + later - when the group of parallel tests have finished - + attempted to print something, its group leader would be + dead and there would be <c>badarg</c> exception.</p> + <p>Those problems have been solved by having group + leaders separate from the processes that supervises the + test cases, and keeping temporary group leader process + for parallel test cases alive until no more process in + the system use them as group leaders.</p> + <p>Also, a new <c>unexpected_io.log</c> log file + (reachable from the summary page of each test suite) has + been introduced. All unexpected IO will be printed into + it(for example, IO to a group leader for a parallel test + case that has finished).</p> + <p> + Own Id: OTP-10101 Aux Id: OTP-10125 </p> + </item> + <item> + <p> + Some bugfixes in <c>ct_snmp:</c></p> + <p> + <list> <item> ct_snmp will now use the value of the + 'agent_vsns' config variable when setting the 'variables' + parameter to snmp application agent configuration. + Earlier this had to be done separately - i.e. the + supported versions had to be specified twice. </item> + <item> Snmp application failed to write notify.conf since + ct_snmp gave the notify type as a string instead of an + atom. This has been corrected. </item> </list></p> + <p> + Own Id: OTP-10432</p> + </item> + <item> + <p> + Some bugfixes in <c>ct_snmp</c>:</p> + <p> + <list> <item> Functions <c>register_users/2</c>, + <c>register_agents/2</c> and <c>register_usm_users/2</c>, + and the corresponding <c>unregister_*/1</c> functions + were not executable. These are corrected/rewritten. + </item> <item> Function <c>update_usm_users/2</c> is + removed, and an unregister function is added instead. + Update can now be done with unregister_usm_users and then + register_usm_users. </item> <item> Functions + <c>unregister_*/2</c> are added, so specific + users/agents/usm users can be unregistered. </item> + <item> Function <c>unload_mibs/1</c> is added for + completeness. </item> <item> Overriding configuration + files did not work, since the files were written in + priv_dir instead of in the configuration dir + (priv_dir/conf). This has been corrected. </item> <item> + Arguments to <c>register_usm_users/2</c> were faulty + documented. This has been corrected. </item> </list></p> + <p> + Own Id: OTP-10434 Aux Id: kunagi-264 [175] </p> + </item> + <item> + <p> + Faulty exported specs in common test has been corrected + to <c>ct_netconfc:hook_options/0</c> and + <c>inet:hostname/0</c></p> + <p> + Own Id: OTP-10601</p> + </item> + <item> + <p> + The netconf client in common_test did not adjust the + window after receiving data. Due to this, the client + stopped receiving data after a while. This has been + corrected.</p> + <p> + Own Id: OTP-10646</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>It is now possible to let a test specification include + other test specifications. Included specs can either be + joined with the source spec (and all other joined specs), + resulting in one single test run, or they can be executed + in separate test runs. Also, a start flag/option, + <c>join_specs</c>, has been introduced, to be used in + combination with the <c>spec</c> option. With + <c>join_specs</c>, Common Test can be told to either join + multiple test specifications, or run them separately. + Without <c>join_specs</c>, the latter behaviour is + default. Note that this is a change compared to earlier + versions of Common Test, where specifications could only + be joined. More information can be found in the Running + Tests chapter in the User's Guide (see the Test + Specifications section).</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-9881 Aux Id: kunagi-350 [261] </p> + </item> + <item> + <p> + The <c>ct_slave:start/3</c> function now supports an + <c>{env,[{Var,Value}]}</c> option to extend environment + for the slave node.</p> + <p> + Own Id: OTP-10469 Aux Id: kunagi-317 [228] </p> + </item> + <item> + <p> Some examples overflowing the width of PDF pages have + been corrected. </p> + <p> + Own Id: OTP-10665</p> + </item> + <item> + <p> + Update common test modules to handle unicode <list> + <item> Use UTF-8 encoding for all HTML files, except the + HTML version of the test suite generated with + erl2html2:convert, which will have the same encoding as + the original test suite (.erl) file. </item> <item> + Encode link targets in HTML files with + test_server_ctrl:uri_encode/1. </item> <item> Use unicode + modifier 't' with ~s when appropriate. </item> <item> Use + unicode:characters_to_list and + unicode:characters_to_binary for conversion between + binaries and strings instead of binary_to_list and + list_to_binary. </item> </list></p> + <p> + Own Id: OTP-10783</p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + CT drops error reason when groups/0 crashes.</p> + <p> + Own Id: OTP-10631 Aux Id: kunagi-345 [256] </p> + </item> + <item> + <p> + Event handler on a ct_master node causes hanging.</p> + <p> + Own Id: OTP-10634 Aux Id: kunagi-347 [258] </p> + </item> + <item> + <p> + CT fails to open telnet conn after a timetrap timeout.</p> + <p> + Own Id: OTP-10648 Aux Id: seq12212 </p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.6.3.1</title> <section><title>Known Bugs and Problems</title> |