From dcda9b507bf14391c8bed91bfa9c56355342b681 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Gustavsson?= Date: Tue, 16 Feb 2016 06:45:27 +0100 Subject: Remove test_server as a standalone application The test_server application has previously been deprecated. In OTP 19, we will move relevant parts of test_server into the common_test application. Test suites that include test_server.hrl must be updated to include ct.hrl instead. Test suites that include test_server_line.hrl must removed that inclusion. Test suites that call the test_server module directly will continue to work in OTP 19. The test suites for Erlang/OTP are built and executed in exactly the same way as previously. Here are some more details. The modules test_server*.erl and erl2html2.erl in lib/test_server/src have been moved to common_test/src. The test_server.hrl and test_server_line.hrl include files have been deleted. The macros in test_server.hrl have been copied into lib/common_test/include/ct.hrl. The ts*.erl modules and their associated data files in lib/test_server/src has been been moved to the new directory lib/common_test/test_server. The ts* modules are no longer built to lib/common_test/ebin. They will only built when 'make release_tests' is executed. The test suite for test_server has been moved to lib/common_test/test. The rest of the files have been deleted. --- lib/test_server/doc/src/notes.xml | 1694 ------------------------------------- 1 file changed, 1694 deletions(-) delete mode 100644 lib/test_server/doc/src/notes.xml (limited to 'lib/test_server/doc/src/notes.xml') diff --git a/lib/test_server/doc/src/notes.xml b/lib/test_server/doc/src/notes.xml deleted file mode 100644 index b48bda94d0..0000000000 --- a/lib/test_server/doc/src/notes.xml +++ /dev/null @@ -1,1694 +0,0 @@ - - - - -
- - 20042013 - Ericsson AB. All Rights Reserved. - - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - - - - APPLICATION Release Notes - Peter Andersson - Peter Andersson - - - - 2007-11-30 - A - notes.xml -
- -
Test_Server 3.9.1 - -
Fixed Bugs and Malfunctions - - -

- When generating Makefile from Makefile.src, - ts_lib:get_arg/4 earlier removed all spaces in the - extracted argument. The code was probably meant for - removing leading and trailing spaces only, and is now - corrected to do so.

-

- Own Id: OTP-13015

- - -

- With the Common Test 'create_priv_dir' start option set - to 'auto_per_tc', the name of the priv directory for a - configuration function could clash with the name of the - priv directory for a test case, which would cause Test - Server failure. This error has been corrected.

-

- Own Id: OTP-13181

- - -
- -
- -
Test_Server 3.9 - -
Fixed Bugs and Malfunctions - - -

- The status of an aborted test due to test suite - compilation error has changed from 'auto_skipped' to - 'failed'. This affects both the textual log file, event - handling and CT hook callbacks. The logging of - compilation failures has also been improved, especially - in the case of multiple test suites failing compilation.

-

- Own Id: OTP-10816

- - -

- The Test Server source code parser (erl2html2) failed to - handle the macro tuple in the syntax tree returned by - epp_dodger. This error has been corrected.

-

- Own Id: OTP-12740

- - -
- - -
Improvements and New Features - - -

- The Test Server application has been marked as obsolete - and will be removed from OTP in the next major release - (OTP 19.0).

-

- Own Id: OTP-10923 Aux Id: OTP-12705

- - -

- When running OTP tests using the ts interface, it is now - possible to specify so called test categories per OTP - application. A test category is represented by a CT test - specification and defines an arbitrary subset of existing - test suites, groups and cases. Examples of test - categories are 'smoke' (smoke tests) and 'bench' - (benchmarks). (Call ts:help() for more info). Also, - functions for reading terms from the current test - specification during test, ct:get_testspec_terms/0 and - ct:get_testspec_terms/1, have been implemented.

-

- Own Id: OTP-11962

- - -
- -
- -
Test_Server 3.8.1 - -
Fixed Bugs and Malfunctions - - -

- If the last expression in a test case causes a timetrap - timeout, the stack trace is ignored and not printed to - the test case log file. This happens because the - {Suite,TestCase,Line} info is not available in the stack - trace in this scenario, due to tail call elimination. - Common Test has been modified to handle this situation by - inserting a {Suite,TestCase,last_expr} tuple in the - correct place and printing the stack trace as expected.

-

- Own Id: OTP-12697 Aux Id: seq12848

- - -
- -
- -
Test_Server 3.8 - -
Fixed Bugs and Malfunctions - - -

- When installing test suites in a cross compilation - environment, ts_install was not able to read the values - of the environment variables specified in the - configuration file. This has been fixed.

-

- Own Id: OTP-11441

- - -

- Printouts by means of ct:log/2/3 or ct:pal/2/3 from the - hook functions on_tc_fail/2 and on_tc_skip/2 would (quite - unexpectedly) end up in the "unexpected i/o" log file - instead of in the test case log file. This behaviour has - been changed so that now, all printouts (including stdio - printouts) from these hook functions will be routed to - the test case log file.

-

- Own Id: OTP-12468

- - -
- - -
Improvements and New Features - - -

- The format of the information printed on top of the test - case (and configuration function) log file has been - slightly modified, mainly in order to make the start - configuration data easier to read and interpret.

-

- Own Id: OTP-12518 Aux Id: seq12808

- - -
- -
- -
Test_Server 3.7.2 - -
Fixed Bugs and Malfunctions - - -

- The source code to html code generator in Test Server - (and Common Test) would fail to generate anchors in the - html code for functions with non-expandable macros, - resulting in bad html links to such functions. This - correction lets the code generator ignore macros that - can't be expanded (i.e. not pre-process them), so that - correct anchors will always be produced.

-

- Own Id: OTP-11766 Aux Id: seq12556

- - -

- Make sure to install .hrl files when needed

-

- Own Id: OTP-12197

- - -
- - -
Improvements and New Features - - -

- Distribute autoconf helpers to applications at - build time instead of having multiple identical copies - committed in the repository.

-

- Own Id: OTP-12348

- - -
- -
- -
Test_Server 3.7.1 - -
Fixed Bugs and Malfunctions - - -

- The mechanism for running code cover analysis with - common_test has been improved. Earlier, if a test run - consisted of multiple tests, cover would be started and - stopped for each test. This would give "intermediate" - cover logs available from the "Coverage log" link on the - test suite result pages. To accumulate cover data over - all tests, the 'export' option had to be used in the - cover spec file. This was not well documented, and the - functionality was quite confusing.

-

- Using the 'nodes' option in the cover spec file would - fail when the test run consisted of multiple tests, since - the specified nodes would only be included in the cover - analysis of the first test.

-

- The repeated compilation and analysis of the same modules - was also very time consuming.

-

- To overcome these problems, ct will now only cover - compile and analyze modules once per test run, i.e. once - for each cover spec file. The log file is available via a - new button on the top level index page. The old "Coverage - log" links on the test suite result pages still exist, - but they all point to the same log containing the - accumulated result.

-

- Own Id: OTP-11971

- - -
- -
- -
Test_Server 3.7 - -
Fixed Bugs and Malfunctions - - -

- Application upgrade (appup) files are corrected for the - following applications:

-

- asn1, common_test, compiler, crypto, debugger, - dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, - inets, observer, odbc, os_mon, otp_mibs, parsetools, - percept, public_key, reltool, runtime_tools, ssh, - syntax_tools, test_server, tools, typer, webtool, wx, - xmerl

-

- A new test utility for testing appup files is added to - test_server. This is now used by most applications in - OTP.

-

- (Thanks to Tobias Schlager)

-

- Own Id: OTP-11744

- - -
- - -
Improvements and New Features - - -

- Calls to erlang:open_port/2 with 'spawn' are updated to - handle space in the command path.

-

- Own Id: OTP-10842

- - -
- -
- -
Test_Server 3.6.4 - -
Fixed Bugs and Malfunctions - - -

The way Common Test handles skipping of test cases has - been updated. In previous versions, returning - {skip,Reason} from a configuration function (such - as init_per_suite or init_per_group), resulted in all - affected test cases getting skipped with status - auto_skipped. This was inappropriate, since this - status is supposed to be used to inform that Common Test - has taken the initiative to skip something (e.g. a test - case group if init_per_group failed). Therefore, in this - version of Common Test, whenever the user skips a suite, - group, or individual test case (by means of a - configuration function or test specification term), the - affected test cases get the status user_skipped - instead.

This update has meant a few changes that - may affect Common Test users in various ways:

- The test results and statistics will be affected, - which is important to know when running regression tests - and comparing results to previous test runs. - Users that read or parse the textual log file - suite.log will notice that an auto skipped - function is now reported as auto_skipped rather - than skipped as before. When - require fails in an info function (such as suite/0 - or group/1), all affected configuration functions and - test cases are marked as auto_skipped. - If Common Test detects an error in the test suite - (such as e.g. an invalid all/0 function), all affected - configuration functions and test cases are marked as - auto_skipped. If a repeated test run - session reaches a deadline with force_stop - enabled, all remaining test cases are marked as - auto_skipped rather than user_skipped as - before. The event messages that Common Test - generates during test runs have been affected by this - update. For details see OTP-11524. -

- Own Id: OTP-11305 Aux Id: OTP-11524

- - -
- -
- -
Test_Server 3.6.3 - -
Fixed Bugs and Malfunctions - - -

- Test Server installed an error handler (test_server_h) - only to be able to write the name of the current test - case to stdout whenever it received an error- or progress - report. This functionality was not useful and has been - removed. The built-in Common Test hook, cth_log_redirect, - has instead been improved to now also tag all error- and - progress reports in the log with suite-, group-, and/or - test case name.

-

- Own Id: OTP-11263 Aux Id: seq12251

- - -
- - -
Improvements and New Features - - -

- A new log, the "Pre- and Post Test I/O Log", has been - introduced, which makes it possible to capture error- and - progress reports, as well as printouts made with ct:log/2 - and ct:pal/2, before and after a test run. (Some minor - improvements of the logging system have been made at the - same time). Links to the new log are found on the Common - Test Framework Log page. The Common Test User's Guide has - been updated with information about the new log and also - with a new section on how to synchronize external - applications with Common Test by means of the CT Hook - init and terminate functions.

-

- Own Id: OTP-11272

- - -
- - -
Known Bugs and Problems - - -

- Test Server: Report auto_skipped in major log.

-

- Own Id: OTP-11297

- - -
- -
- -
Test_Server 3.6.2 - -
Fixed Bugs and Malfunctions - - -

- Some unused code related to remote targets is removed, - and documentation is updated.

-

- Own Id: OTP-10607 Aux Id: kunagi-338 [249]

- - -

- A bug in test_server_gl caused io requests containing - invalid data (i.e. not unicode:chardata()) to hang, since - no io reply was sent. This has been corrected.

-

- Own Id: OTP-10991

- - -

- Common Test would, in case of timetrap error, print a - warning in the log if end_per_testcase wasn't implemented - in the suite, even though it's an optional function. This - printout has been removed.

-

- Own Id: OTP-11052

- - -
- - -
Improvements and New Features - - -

- The '-force_stop' flag to use with time-limited repeats - of test runs can now be used with a new 'skip_rest' - option which causes the rest of the test cases in the - ongoing test job to be skipped when the time limit is - reached. E.g. 'ct_run -spec xxx -duration 010000 - -force_stop skip_rest'

-

- Own Id: OTP-10856 Aux Id: OTP-10832

- - -
- -
- -
Test_Server 3.6.1 - -
Fixed Bugs and Malfunctions - - -

- The unicode update of test_server for R16A introduced a - few potential errors when logging to files. Sometimes ~tp - or ~ts was used for formatting also when writing to files - that were not opened with the {encoding,utf8} option. If - then the argument contained unicode characters above 255, - the file descriptor would crash. This has been corrected - by the following modifications:

Since the - 'unexpected_io' log file is used only when the test case - HTML file is not available (e.g. between test cases), - this file is now also a HTML file and as other - test_server HTML logs it is always UTF-8 encoded - Since it is possible to change which information - is going to which log file (with - test_server_ctrl:set_levels/3), we do not have full - control over which information is written to which file. - This means that any printout could be written to the - 'major' log file (suite.log), which was earlier encoded - as latin1. To avoid crashing this file descriptor due to - unicode strings, the 'major' log file is now also encoded - in UTF-8 (possible incopatibility). The - cross_cover.info file is no longer a text file which can - be read with file:consult/1, instead it is written as a - pure binary file using term_to_binary when writing and - binary_to_term when reading. The encoding - of the file named 'last_name', which only content is the - path to the last run.<timestamp> directory, is now - dependent on the file name mode of the VM. If file names - are expected to be unicode, then the 'last_name' file is - UTF-8 encoded, else it is latin1 encoded. -

- Also, ~tp has been changed back to ~p unless it is - somehow likely that the argument includes strings. It is - not obvious that this is the correct thing to do, but - some decission had to be taken...

-

- Own Id: OTP-10780

- - -

- Using the force_stop flag/option to interrupt a test run - caused a crash in Common Test. This problem has been - solved.

-

- Own Id: OTP-10832

- - -
- -
- -
Test_Server 3.6 - -
Fixed Bugs and Malfunctions - - -

- Line numbering of erlang files that were not correctly - indented could be wrong after coverting to html with - erl2html2:convert/[2,3] (the source code pointed to from - the test case). This has been corrected.

-

- Also, there are now link targets for each line and not - only for each 10th line, and link targets for functions - now include the arity and not only the function name - (e.g. func/1 has a link target "func-1").

-

- Own Id: OTP-9710 Aux Id: seq11945, kunagi-201 [112]

- - -

- Severe errors detected by test_server (e.g. if log - files directories cannot be created) will now be reported - to common_test and noted in the common_test - logs.

-

- Own Id: OTP-9769 Aux Id: kunagi-202 [113]

- - -

- The earlier undocumented cross cover feature for - accumulating cover data over multiple tests has now been - fixed and documented.

-

- Own Id: OTP-9870 Aux Id: kunagi-206 [117]

- - -

- If the test suite itself was included in code coverage - analysis, then the test_server would not manage to set - data_dir correctly for the test. This has been corrected.

-

- Own Id: OTP-9956 Aux Id: kunagi-207 [118]

- - -

- Any call to test_server:break/1 should cancel all active - timetramps. However, in some cases - Suite:end_per_testcase/2 is executed on a different - process than the test case itself, and if - test_server:break/1 was called from there, the timetraps - were not cancelled. This has been corrected.

-

- Own Id: OTP-10046 Aux Id: kunagi-174 [85]

- - -

When a test case failed because of a timetrap time - out, the Config data for the case was lost in the - following call to end_per_testcase/2, and also in - calls to the CT Hook function - post_end_per_testcase/4. This problem has been - solved and the Config data is now correctly passed - to the above functions after a timetrap timeout - failure.

-

- Own Id: OTP-10070 Aux Id: kunagi-175 [86]

- - -

In test_server, the same process would supervise the - currently running test case and be group leader (and IO - server) for the test case. Furthermore, when running - parallel test cases, new temporary supervisor/group - leader processes were spawned and the process that was - group leader for sequential test cases would not be - active. That would lead to several problems:

-

* Processes started by init_per_suite will inherit the - group leader of the init_per_suite process (and that - group leader would not process IO requests when parallel - test cases was running). If later a parallel test case - caused such a processto print using (for example) - io:format/2, the calling would hang.

-

* Similarly, if a process was spawned from a parallel - test case, it would inherit the temporary group leader - for that parallel test case. If that spawned process - later - when the group of parallel tests have finished - - attempted to print something, its group leader would be - dead and there would be badarg exception.

-

Those problems have been solved by having group - leaders separate from the processes that supervises the - test cases, and keeping temporary group leader process - for parallel test cases alive until no more process in - the system use them as group leaders.

-

Also, a new unexpected_io.log log file - (reachable from the summary page of each test suite) has - been introduced. All unexpected IO will be printed into - it(for example, IO to a group leader for a parallel test - case that has finished).

-

- Own Id: OTP-10101 Aux Id: OTP-10125

- - -

- The stability of common_test and - test_server when running test cases in parallel - has been improved.

-

- Own Id: OTP-10480 Aux Id: kunagi-318 [229]

- - -
- - -
Improvements and New Features - - -

- Added a general framework for executing benchmarks of - Erlang/OTP. Benchmarks for the Erlang VM and mnesia have - been incorporated in the framework.

-

- For details about how to add more benchmarks see - $ERL_TOP/HOWTO/BENCHMARKS.md in the source distribution.

-

- Own Id: OTP-10156

- - -

- Update common test modules to handle Unicode:

- Use UTF-8 encoding for all HTML files, except the - HTML version of the test suite generated with - erl2html2:convert, which will have the same encoding as - the original test suite (.erl) file. - Encode link targets in HTML files with - test_server_ctrl:uri_encode/1. Use unicode - modifier 't' with ~s when appropriate. Use - unicode:characters_to_list and - unicode:characters_to_binary for conversion between - binaries and strings instead of binary_to_list and - list_to_binary. - - -
- -
- -
Test_Server 3.5.3 - -
Improvements and New Features - - -

- test_server_h will now recognize info_reports written by - ct connection handlers (according to the description in - cth_conn_log) and ignore them as they will be completely - handled by by ct_conn_log_h.

-

- Earlier test_server_h would print a tag (testcase name) - before forwarding the report to error_logger_tty_h. This - would cause lots of tags in the log with no info report - following (since error_logger_tty_h did not handle them).

-

- Own Id: OTP-10571

- - -
- - -
Known Bugs and Problems - - -

- Restore Config data if lost when test case fails.

-

- Own Id: OTP-10070 Aux Id: kunagi-175 [86]

- - -

- IO server error in test_server.

-

- Own Id: OTP-10125 Aux Id: OTP-10101, kunagi-177 [88]

- - -
- -
- -
Test_Server 3.5.2 - -
Fixed Bugs and Malfunctions - - -

- The documentation has been updated with the latest - changes for the test_server_ctrl:report/2 function.

-

- Own Id: OTP-10086 Aux Id: seq12066

- - -

- The ct:get_status/0 function failed to report status if a - parallel test case group was running at the time of the - call. This has been fixed and the return value for the - function has been updated. Please see the ct reference - manual for details.

-

- Own Id: OTP-10172

- - -
- - -
Improvements and New Features - - -

- It is now possible to sort the HTML tables by clicking on - the header elements. In order to reset a sorted table, - the browser window should simply be refreshed. This - feature requires that the browser supports javascript, - and has javascript execution enabled. If the 'ct_run - -basic_html' flag is used, no javascript code is included - in the generated HTML code.

-

- Own Id: OTP-9896 Aux Id: seq12034, OTP-9835

- - -

- Verbosity levels for log printouts has been added. This - makes it possible to specify preferred verbosity for - different categories of log printouts, as well as general - printouts (such as standard IO), to allow control over - which strings get printed and which get ignored. New - versions of the Common Test logging functions, ct:log, - ct:pal and ct:print, have been introduced, with a new - Importance argument added. The Importance value is - compared to the verbosity level at runtime. More - information can be found in the chapter about Logging in - the Common Test User's Guide.

-

- Own Id: OTP-10067 Aux Id: seq12050

- - -

- The Erlang/OTP test runner ts has been extended to allow - cross compilation of test suites. To cross compile the - test suites first follow the normal cross compilation - procedures and release the tests on the build host. Then - install ts using an xcomp specification file and compile - test suites using ts:compile_testcases/0. For more - details see $ERL_TOP/xcomp/README.md.

-

- Own Id: OTP-10074

- - -
- -
- -
Test_Server 3.5.1 - -
Fixed Bugs and Malfunctions - - -

- After a test case timeout or abortion, the - end_per_testcase function executes on a new dedicated - process. The group leader for this process should be set - to the IO server for the test case, which was not done - properly. The result of this error was that no warnings - about end_per_testcase failing or timing out were ever - printed in the test case log. Also, help functions such - as e.g. test_server:stop_node/1, attempting to - synchronize with the IO server, would hang. The fault has - been corrected.

-

- Own Id: OTP-9666

- - -

- A deadlock situation could occur if Common Test is - forwarding error_handler printouts to Test Server at the - same time a new test case is starting. This error has - been fixed.

-

- Own Id: OTP-9894

- - -

- When a test case was killed because of a timetrap - timeout, the current location (suite, case and line) was - not printed correctly in the log files. This has been - corrected.

-

- Own Id: OTP-9930 Aux Id: seq12002

- - -

- Test Server and Common Test would add new error handlers - with each test run and fail to remove previously added - ones. In the case of Test Server, this would only happen - if SASL was not running on the test node. This has been - fixed.

-

- Own Id: OTP-9941 Aux Id: seq12009

- - -

- If a test case process was terminated due to an exit - signal from a linked process, Test Server failed to - report the correct name of the suite and case to the - framework. This has been corrected.

-

- Own Id: OTP-9958 Aux Id: OTP-9855

- - -
- - -
Improvements and New Features - - -

- A new optional feature has been introduced that enables - Common Test to generate priv_dir directory names that are - unique for each test case or config function. The name of - the option/flag is 'create_priv_dir' and it can be set to - value 'auto_per_run' (which is the default, existing, - behaviour), or 'auto_per_tc' or 'manual_per_tc'. If - 'auto_per_tc' is used, Test Server creates a dedicated - priv_dir automatically for each test case (which can be - very expensive in case of many and/or repeated cases). If - 'manual_per_tc' is used, the user needs to create the - priv_dir explicitly by calling the new function - ct:make_priv_dir/0.

-

- Own Id: OTP-9659 Aux Id: seq11930

- - -

- A column for test case group name has been added to the - suite overview HTML log file.

-

- Own Id: OTP-9730 Aux Id: seq11952

- - -

- It is now possible to use the post_end_per_testcase CT - hook function to print a comment for a test case in the - overview log file, even if the test case gets killed by a - timetrap or unknown exit signal, or if the - end_per_testcase function times out.

-

- Own Id: OTP-9855 Aux Id: seq11979

- - -

- Common Test will now print error information (with a time - stamp) in the test case log file immediately when a test - case fails. This makes it easier to see when, in time, - the fault actually occured, and aid the job of locating - relevant trace and debug printouts in the log.

-

- Own Id: OTP-9904 Aux Id: seq11985, OTP-9900

- - -

- Test Server has been modified to check the SASL - errlog_type parameter when receiving an error logger - event, so that it doesn't print reports of type that the - user has disabled.

-

- Own Id: OTP-9955 Aux Id: seq12013

- - -

- If an application cannot be found by ts it is - automatically skipped when testing.

-

- Own Id: OTP-9971

- - -

- By specifying a user defined function ({M,F,A} or fun) as - timetrap value, either by means of an info function or by - calling ct:timetrap/1, it is now possible to set a - timetrap that will be triggered when the user function - returns.

-

- Own Id: OTP-9988 Aux Id: OTP-9501, seq11894

- - -

- If the optional configuration functions init_per_suite/1 - and end_per_suite/1 are not implemented in the test - suite, local Common Test versions of these functions are - called instead, and will be displayed in the overview log - file. Any printouts made by the pre- or - post_init_per_suite and pre- or post_end_per_suite hook - functions are saved in the log files for these functions.

-

- Own Id: OTP-9992

- - -
- -
- -
Test_Server 3.5 - -
Improvements and New Features - - -

- The test case group info function has been implemented in - Common Test. Before execution of a test case group, a - call is now made to TestSuite:group(GroupName). - The function returns a list of test properties, e.g. to - specify timetrap values, require configuration data, etc - (analogue to the test suite- and test case info - function). The scope of the properties set by - group(GroupName) is all test cases and sub-groups - of group GroupName.

-

- Own Id: OTP-9235

- - -

- The look of the HTML log files generated by Common Test - and Test Server has been improved (and made easier to - customize) by means of a CSS file.

-

- Own Id: OTP-9706

- - -
- - -
Known Bugs and Problems - - -

- Fix problems in CT/TS due to line numbers in exceptions.

-

- Own Id: OTP-9203

- - -
- -
- -
Test_Server 3.4.5 - -
Fixed Bugs and Malfunctions - - -

- An error in how comments are colored in the test suite - overview html log file has been corrected. As result, a - new framework callback function, format_comment/1, has - been introduced.

-

- Own Id: OTP-9237

- - -

- Test Server did not release SASL TTY handlers - (sasl_report_tty_h and error_logger_tty_h) properly after - each test run. This error has been fixed.

-

- Own Id: OTP-9311

- - -

- Automatically generated init- and end-configuration - functions for test case groups caused incorrect execution - order of test cases. This has been corrected.

-

- Own Id: OTP-9369

- - -

- If ct:log/2 was called with bad arguments, this could - cause the Common Test IO handling process to crash. This - fault has been corrected.

-

- Own Id: OTP-9371 Aux Id: OTP-8933

- - -

- A bug has been fixed that made Test Server call the - end_tc/3 framework function with an incorrect module name - as first argument.

-

- Own Id: OTP-9379 Aux Id: seq11863

- - -

- If end_per_testcase caused a timetrap timeout, the actual - test case status was discarded and the test case logged - as successful (even if the case had actually failed - before the call to end_per_testcase). This fault has been - fixed.

-

- Own Id: OTP-9397

- - -

- If a timetrap timeout occured during execution of of a - function in a lib module (i.e. a function called directly - or indirectly from a test case), the Suite argument in - the end_tc/3 framework callback function would not - correctly contain the name of the test suite, but the lib - module. (This would only happen if the lib module was - compiled with ct.hrl included). This error has been - solved.

-

- Own Id: OTP-9398

- - -

- Add a proplist() type

-

- Recently I was adding specs to an API and found that - there is no canonical proplist() type defined. (Thanks to - Ryan Zezeski)

-

- Own Id: OTP-9499

- - -

XML files have been corrected.

-

- Own Id: OTP-9550 Aux Id: OTP-9541

- - -

- If a test suite would start with a test case group - defined without the init_per_group/2 and end_per_group/2 - function, init_per_suite/1 would not execute initially - and logging of the test run would fail. This error has - been fixed.

-

- Own Id: OTP-9584

- - -
- - -
Improvements and New Features - - -

- A new option, 'logopts', has been introduced, to make it - possible to modify some aspects of the logging behaviour - in Common Test (or Test Server). For example, whenever an - io printout is made, test_server adds newline (\n) to the - end of the output string. This may not always be a - preferred action and can therefore be disabled by means - of "ct_run ... -logopts no_nl" (or ct:run_test([..., - {logopts,[no_nl]}])). A new framework callback function, - get_logopts/0, has been introduced (see the ct_framework - module for details).

-

- Own Id: OTP-9372 Aux Id: OTP-9396

- - -

- A new option, 'logopts', has been introduced, to make it - possible to modify some aspects of the logging behaviour - in Common Test (or Test Server). For example, if the html - version of the test suite source code should not be - generated during the test run (and consequently be - unavailable in the log file system), the feature may be - disabled by means of "ct_run ... -logopts no_src" (or - ct:run_test([..., {logopts,[no_src]}])). A new framework - callback function, get_logopts/0, has been introduced - (see the ct_framework module for details).

-

- Own Id: OTP-9396 Aux Id: seq11869, OTP-9372

- - -

- It is now possible to use a tuple {M,F,A}, or a fun, as - timetrap specification in the suite info function or test - case info functions. The function must return a valid - timeout value, as documented in the common_test man page - and in the User's Guide.

-

- Own Id: OTP-9501 Aux Id: seq11894

- - -
- -
- -
Test_Server 3.4.4 - -
Fixed Bugs and Malfunctions - - -

- It was previously not possible to use timetrap value - 'infinity' with ct:timetrap/1. This has been fixed.

-

- Own Id: OTP-9159

- - -

- A bug that made it impossible to cancel the previous - timetrap when calling ct:timetrap/1 has been corrected.

-

- Own Id: OTP-9233 Aux Id: OTP-9159

- - -
- - -
Improvements and New Features - - -

- When running tests with auto-compilation disabled, Common - Test could only display the test suite source code on - html format in the test case log if the source file was - located in the same directory as the pre-compiled suite. - This has been modified so that Common Test now tries to - locate the source file by means of the test suite module - info (Suite:module_info/1). As a result, a suite may now - be compiled to a different output directory (e.g. - $MYTEST/bin) than the source code directory (e.g. - $MYTEST/src), without the source-code-to-html generation - being affected.

-

- Own Id: OTP-9138

- - -

- It is now possible to return a tuple {fail,Reason} from - init_per_testcase/2. The result is that the associated - test case gets logged as failed without ever executing.

-

- Own Id: OTP-9160 Aux Id: seq11502

- - -

- Added DragonflyBSD check in test_server configure.

-

- Own Id: OTP-9249

- - -
- -
- -
Test_Server 3.4.3 - -
Fixed Bugs and Malfunctions - - -

- Updated the ts*.config files to contain information - relevant to testing Erlang/OTP in an open source - environment.

-

- Own Id: OTP-9017

- - -
- - -
Improvements and New Features - - -

- Alpha release of Common Test Hooks (CTH). CTHs allow the - users of common test to abtract out common behaviours - from test suites in a much more elegant and flexible way - than was possible before. Note that the addition of this - feature may introduce minor changes in the undocumented - behaviour of the interface inbetween common_test and - test_server.

-

- *** POTENTIAL INCOMPATIBILITY ***

-

- Own Id: OTP-8851

- - -
- -
- -
Test_Server 3.4.2 - -
Improvements and New Features - - -

Miscellaneous updates

-

- Own Id: OTP-8976

- - -
- -
- -
Test_Server 3.4.1 - -
Fixed Bugs and Malfunctions - - -

- Returning {return_group_result,failed} from end_per_group - in a group that is part of a sequence, did not cause the - proceeding cases (or groups) to get skipped. This has - been fixed.

-

- Own Id: OTP-8753 Aux Id: seq11644

- - -
- - -
Improvements and New Features - - -

- Common Test has been updated to handle start options and - test specification terms for test case groups (and test - cases in groups). Also, an option named 'label', has been - added that associates the test run with a name that - Common Test prints in the overview HTML logs.

-

- Own Id: OTP-8725 Aux Id: OTP-8727

- - -

- It is now possible to skip all tests in a suite, or a - group, by returning {fail,Reason} from the end_tc/5 - framework function for init_per_suite, or init_per_group.

-

- Own Id: OTP-8805 Aux Id: seq11664

- - -
- -
- -
Test_Server 3.4 - -
Fixed Bugs and Malfunctions - - -

- Returning {fail,Reason} from the framework end_tc - function was not handled properly by Test Server for all - test suite functions.

-

- Own Id: OTP-8492 Aux Id: seq11502

- - -

- If the framework end_tc function would hang and get - aborted by Test Server, there was no indication of - failure in the logs. This has been fixed.

-

- Own Id: OTP-8682 Aux Id: seq11504

- - -
- - -
Improvements and New Features - - -

- It is now possible for the Test Server framework end_tc - function to change the status of the test case from ok or - auto-skipped to failed by returning {fail,Reason}.

-

- Own Id: OTP-8495 Aux Id: seq11502

- - -

- Test Server will now call the end_per_testcase/2 function - even if the test case has been terminated explicitly - (with abort_current_testcase/1), or after a timetrap - timeout. Under these circumstances the return value of - end_per_testcase is completely ignored. Therefore the - function will not be able to change the reason for test - case termination by returning {fail,Reason}, nor will it - be able to save data with {save_config,Data}.

-

- Own Id: OTP-8500 Aux Id: seq11521

- - -

- Previously, a repeat property of a test case group - specified the number of times the group should be - repeated after the main test run. I.e. {repeat,N} would - case the group to execute 1+N times. To be consistent - with the behaviour of the run_test repeat option, this - has been changed. N now specifies the absolute number of - executions instead.

-

- Own Id: OTP-8689 Aux Id: seq11502

- - -
- -
- -
Test_Server 3.3.6 - -
Fixed Bugs and Malfunctions - - -

- The Test Server parse transform did not handle bit string - comprehensions. This has been fixed.

-

- Own Id: OTP-8458 Aux Id: OTP-8311

- - -
- - -
Improvements and New Features - - -

- The tc_status value in the Config list for a test case - that has failed because of a timetrap timeout, has - changed from {tc_status,timeout} to - {tc_status,timetrap_timeout}.

-

- Own Id: OTP-8302

- - -

The documentation is now possible to build in an open - source environment after a number of bugs are fixed and - some features are added in the documentation build - process.

-

- The arity calculation is updated.

-

- The module prefix used in the function names for - bif's are removed in the generated links so the links - will look like - "http://www.erlang.org/doc/man/erlang.html#append_element-2" - instead of - "http://www.erlang.org/doc/man/erlang.html#erlang:append_element-2".

-

- Enhanced the menu positioning in the html - documentation when a new page is loaded.

-

- A number of corrections in the generation of man - pages (thanks to Sergei Golovan)

-

- The legal notice is taken from the xml book file so - OTP's build process can be used for non OTP - applications.

-

- Own Id: OTP-8343

- - -
- -
- -
Test_Server 3.3.5 - -
Fixed Bugs and Malfunctions - - -

- If the init_per_testcase/2 function fails, the test case - now gets marked and counted as auto skipped, not user - skipped (which would previously happen).

-

- Own Id: OTP-8289

- - -
- - -
Improvements and New Features - - -

- The documentation is now built with open source tools - (xsltproc and fop) that exists on most platforms. One - visible change is that the frames are removed.

-

- Own Id: OTP-8201

- - -

- It is now possible to fail a test case from the - end_per_testcase/2 function, by returning {fail,Reason}.

-

- Own Id: OTP-8284

- - -

- It is now possible to fail a test case by having the - end_tc/3 framework function return {fail,Reason} for the - test case.

-

- Own Id: OTP-8285

- - -

- The test_server framework API (e.g. the end_tc/3 - function) has been modified. See the test_server_ctrl - documentation for details.

-

- Own Id: OTP-8286 Aux Id: OTP-8285, OTP-8287

- - -
- -
- -
Test_Server 3.3.4 - -
Fixed Bugs and Malfunctions - - -

- When running a suite starting with a test case group, - Test Server crashed if init_per_suite/1 exited or - returned skip. This has been fixed.

-

- Own Id: OTP-8105 Aux Id: OTP-8089

- - -
- - -
Improvements and New Features - - -

- Various updates and fixes in Common Test and Test Server.

-

- Own Id: OTP-8045 Aux Id: OTP-8089,OTP-8105,OTP-8163

- - -

- Errors in coverage data collection and analysis were - difficult to detect. The logging has been improved so - that more information about e.g. imported and missing - modules is printed to the html log files.

-

- Own Id: OTP-8163 Aux Id: seq11374

- - -

- The Common Test HTML overview pages have been improved. - It is now possible to see if a test case has been skipped - explicitly or because a configuration function has - failed. Also, the history page (all_runs.html) now has - scrolling text displaying the test names. The old format - (showing names as a truncated string) can still be - generated by means of the flag/option 'basic_html'.

-

- Own Id: OTP-8177

- - -
- -
- -
Test_Server 3.3.2 - -
Improvements and New Features - - -

- Various corrections and improvements of Common Test and - Test Server.

-

- Own Id: OTP-7981

- - -
- -
- -
Test_Server 3.3.1 - -
Improvements and New Features - - -

- Minor updates and corrections.

-

- Own Id: OTP-7897

- - -
- -
- -
Test_Server 3.3 - -
Improvements and New Features - - -

- The conf case in Test Server has been extended with - properties that make it possible to execute test cases in - parallel, in sequence and in shuffled order. It is now - also possible to repeat test cases according to different - criterias. The properties can be combined, making it - possible to e.g. repeat a conf case a certain number of - times and execute the test cases in different (random) - order every time. The properties are specified in a list - in the conf case definition: {conf, Properties, InitCase, - TestCases, EndCase}. The available properties are: - parallel, sequence, shuffle, repeat, repeat_until_all_ok, - repeat_until_any_ok, repeat_until_any_fail, - repeat_until_all_fail.

-

- Own Id: OTP-7511 Aux Id: OTP-7839

- - -

The test server starts Cover on nodes of the same - version as the test server itself only.

-

- Own Id: OTP-7699

- - -

- The Erlang mode for Emacs has been updated with new and - modified skeletons for Common Test and TS. Syntax for - test case groups in Common Test (and conf cases with - properties in TS) has been added and a new minimal Common - Test suite skeleton has been introduced.

-

- Own Id: OTP-7856

- - -
- -
-
Test_Server 3.2.4.1 - -
Fixed Bugs and Malfunctions - - -

- The step functionality in Common Test (based on - interaction with Debugger) was broken. This has been - fixed, and some new step features have also been added. - Please see the Common Test User's Guide for details.

-

- Own Id: OTP-7800 Aux Id: seq11106

- - -
- -
- -
Test_Server 3.2.4 - -
Improvements and New Features - - -

- Miscellaneous updates.

-

- Own Id: OTP-7527

- - -
- -
- -
Test_Server 3.2.3 - -
Fixed Bugs and Malfunctions - - -

- When a testcase terminated due to a timetrap, io sent to - the group leader from framework:end_tc/3 (using - ct:pal/2/3 or ct:log/2/3) would cause deadlock. This has - been fixed.

-

- Own Id: OTP-7447 Aux Id: seq11010

- - -
- - -
Improvements and New Features - - -

- Various updates and improvements, plus some minor bug - fixes, have been implemented in Common Test and Test - Server.

-

- Own Id: OTP-7112

- - -

- It is now possible, by means of the new function - ct:abort_current_testcase/1 or - test_server_ctrl:abort_current_testcase/1, to abort the - currently executing test case.

-

- Own Id: OTP-7518 Aux Id: OTP-7112

- - -
- -
- -
Test_Server 3.2.2 - -
Improvements and New Features - - -

erlang:system_info/1 now accepts the - logical_processors, and debug_compiled - arguments. For more info see the, erlang(3) - documentation.

The scale factor returned by - test_server:timetrap_scale_factor/0 is now also - effected if the emulator uses a larger amount of - scheduler threads than the amount of logical processors - on the system.

-

- Own Id: OTP-7175

- - -
- -
- -
Test_Server 3.2.1 - -
Improvements and New Features - - -

- When init_per_suite or end_per_suite terminated due to - runtime failure, test_server failed to format the line - number information properly and crashed. This error has - now been fixed.

-

- Own Id: OTP-7091

- - -
- -
- -
Test_Server 3.2.0 - -
Improvements and New Features - - -

- Test Server is a portable test server for automated - application testing. The server can run test suites on - local or remote targets and log progress and results to - HTML pages. The main purpose of Test Server is to act as - engine inside customized test tools. A callback interface - for such framework applications is provided.

-

- Own Id: OTP-6989

- - -
- -
- - - -- cgit v1.2.3