From dcda9b507bf14391c8bed91bfa9c56355342b681 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Gustavsson?= Date: Tue, 16 Feb 2016 06:45:27 +0100 Subject: Remove test_server as a standalone application The test_server application has previously been deprecated. In OTP 19, we will move relevant parts of test_server into the common_test application. Test suites that include test_server.hrl must be updated to include ct.hrl instead. Test suites that include test_server_line.hrl must removed that inclusion. Test suites that call the test_server module directly will continue to work in OTP 19. The test suites for Erlang/OTP are built and executed in exactly the same way as previously. Here are some more details. The modules test_server*.erl and erl2html2.erl in lib/test_server/src have been moved to common_test/src. The test_server.hrl and test_server_line.hrl include files have been deleted. The macros in test_server.hrl have been copied into lib/common_test/include/ct.hrl. The ts*.erl modules and their associated data files in lib/test_server/src has been been moved to the new directory lib/common_test/test_server. The ts* modules are no longer built to lib/common_test/ebin. They will only built when 'make release_tests' is executed. The test suite for test_server has been moved to lib/common_test/test. The rest of the files have been deleted. --- lib/test_server/doc/src/write_test_chapter.xml | 228 ------------------------- 1 file changed, 228 deletions(-) delete mode 100644 lib/test_server/doc/src/write_test_chapter.xml (limited to 'lib/test_server/doc/src/write_test_chapter.xml') diff --git a/lib/test_server/doc/src/write_test_chapter.xml b/lib/test_server/doc/src/write_test_chapter.xml deleted file mode 100644 index c3e1881b8a..0000000000 --- a/lib/test_server/doc/src/write_test_chapter.xml +++ /dev/null @@ -1,228 +0,0 @@ - - - - -
- - 20022013 - Ericsson AB. All Rights Reserved. - - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - - - - Writing Test Suites - Siri Hansen - - - - write_test_chapter.xml -
- -
- Support for test suite authors -

The test_server module provides some useful functions - to support the test suite author. This includes: -

- - Starting and stopping slave or peer nodes - Capturing and checking stdout output - Retrieving and flushing process message queue - Watchdog timers - Checking that a function crashes - Checking that a function succeeds at least m out of n times - Checking .app files - -

Please turn to the reference manual for the test_server - module for details about these functions. -

-
- -
- Test suites -

A test suite is an ordinary Erlang module that contains test - cases. It's recommended that the module has a name on the form - *_SUITE.erl. Otherwise, the directory function will not find the - modules (by default). -

-

For some of the test server support, the test server include - file test_server.hrl must be included. Never include it - with the full path, for portability reasons. Use the compiler - include directive instead. -

-

The special function all(suite) in each module is called - to get the test specification for that module. The function - typically returns a list of test cases in that module, but any - test specification could be returned. Please see the chapter - about test specifications for details about this. -

-
- -
- Init per test case -

In each test suite module, the functions - init_per_testcase/2 and end_per_testcase/2 must be - implemented. -

-

init_per_testcase is called before each test case in the - test suite, giving a (limited) possibility for initialization. -

-

end_per_testcase/2 is called after each test case is - completed, giving a possibility to clean up. -

-

The first argument to these functions is the name of the test - case. This can be used to do individual initialization and cleanup for - each test cases. -

-

The second argument is a list of tuples called - Config. The first element in a Config tuple - should be an atom - a key value to be used for searching. - init_per_testcase/2 may modify the Config - parameter or just return it as is. Whatever is retuned by - init_per_testcase/2 is given as Config parameter to - the test case itself. -

-

The return value of end_per_testcase/2 is ignored by the - test server. -

-
- -
- Test cases -

The smallest unit that the test server is concerned with is a - test case. Each test case can in turn test many things, for - example make several calls to the same interface function with - different parameters. -

-

It is possible to put many or few tests into each test - case. How many things each test case tests is up to the author, - but here are some things to keep in mind. -

-

Very small test cases often leads to more code, since - initialization has to be duplicated. Larger code, especially with - a lot of duplication, increases maintenance and reduces - readability. -

-

Larger test cases make it harder to tell what went wrong if it - fails, and force us to skip larger portions of test code if a - specific part fails. These effects are accentuated when running on - multiple platforms because test cases often have to be skipped. -

-

A test case generally consists of three parts, the - documentation part, the specification part and the execution - part. These are implemented as three clauses of the same function. -

-

The documentation clause matches the argument 'doc' and - returns a list for strings describing what the test case tests. -

-

The specification clause matches the argument 'suite' - and returns the test specification for this particular test - case. If the test specification is an empty list, this indicates - that the test case is a leaf test case, i.e. one to be executed. -

-

The execution clause implements the actual test case. It takes - one argument, Config, which contain configuration - information like data_dir and priv_dir. See Data and Private Directories for - more information about these. -

-

The Config variable can also contain the - nodenames key, if requested by the require_nodenames - command in the test suite specification file. All Config - items should be extracted using the ?config macro. This is - to ensure future compatibility if the Config format - changes. See the reference manual for test_server for - details about this macro. -

-

If the execution clause crashes or exits, it is considered a - failure. If it returns {skip,Reason}, the test case is - considered skipped. If it returns {comment,String}, - the string will be added in the 'Comment' field on the HTML - result page. If the execution clause returns anything else, it is - considered a success, unless it is {'EXIT',Reason} or - {'EXIT',Pid,Reason} which can't be distinguished from a - crash, and thus will be considered a failure. -

-
- -
- - Data and Private Directories -

The data directory (data_dir) is the directory where the test - module has its own files needed for the testing. A compiler test - case may have source files to feed into the compiler, a release - upgrade test case may have some old and new release of - something. A graphics test case may have some icons and a test - case doing a lot of math with bignums might store the correct - answers there. The name of the data_dir is the the name of - the test suite and then "_data". For example, - "some_path/foo_SUITE.beam" has the data directory - "some_path/foo_SUITE_data/". -

-

The priv_dir is the test suite's private directory. This - directory should be used when a test case needs to write to - files. The name of the private directory is generated by the test - server, which also creates the directory. -

-

Warning: Do not depend on current directory to be - writable, or to point to anything in particular. All scratch files - are to be written in the priv_dir, and all data files found - in data_dir. If the current directory has to be something - specific, it must be set with file:set_cwd/1. -

-
- -
- Execution environment -

Each time a test case is about to be executed, a new process is - created with spawn_link. This is so that the test case will - have no dependencies to earlier tests, with respect to process flags, - process links, messages in the queue, other processes having registered - the process, etc. As little as possible is done to change the initial - context of the process (what is created by plain spawn). Here is a - list of differences: -

- - It has a link to the test server. If this link is removed, - the test server will not know when the test case is finished, - just wait infinitely. - - It often holds a few items in the process dictionary, all - with names starting with 'test_server_'. This is to keep - track of if/where a test case fails. - - There is a top-level catch. All of the test case code is - catched, so that the location of a crash can be reported back to - the test server. If the test case process is killed by another - process (thus the catch code is never executed) the test server - is not able to tell where the test case was executing. - - It has a special group leader implemented by the test - server. This way the test server is able to capture the io that - the test case provokes. This is also used by some of the test - server support functions. - - -

There is no time limit for a test case, unless the test case - itself imposes such a limit, by calling - test_server:timetrap/1 for example. The call can be made - in each test case, or in the init_per_testcase/2 - function. Make sure to call the corresponding - test_server:timetrap_cancel/1 function as well, e.g in the - end_per_testcase/2 function, or else the test cases will - always fail. -

-
- - - -- cgit v1.2.3