aboutsummaryrefslogtreecommitdiffstats
path: root/lib/test_server/doc/src/basics_chapter.xml
diff options
context:
space:
mode:
authorBjörn Gustavsson <[email protected]>2016-02-16 06:45:27 +0100
committerBjörn Gustavsson <[email protected]>2016-02-17 10:35:22 +0100
commitdcda9b507bf14391c8bed91bfa9c56355342b681 (patch)
tree45658baf8d18d7f363a3044972201b957acaffb1 /lib/test_server/doc/src/basics_chapter.xml
parent40aaa8bfa8ec0776a4d70079cf34a5bd337d42fe (diff)
downloadotp-dcda9b507bf14391c8bed91bfa9c56355342b681.tar.gz
otp-dcda9b507bf14391c8bed91bfa9c56355342b681.tar.bz2
otp-dcda9b507bf14391c8bed91bfa9c56355342b681.zip
Remove test_server as a standalone application
The test_server application has previously been deprecated. In OTP 19, we will move relevant parts of test_server into the common_test application. Test suites that include test_server.hrl must be updated to include ct.hrl instead. Test suites that include test_server_line.hrl must removed that inclusion. Test suites that call the test_server module directly will continue to work in OTP 19. The test suites for Erlang/OTP are built and executed in exactly the same way as previously. Here are some more details. The modules test_server*.erl and erl2html2.erl in lib/test_server/src have been moved to common_test/src. The test_server.hrl and test_server_line.hrl include files have been deleted. The macros in test_server.hrl have been copied into lib/common_test/include/ct.hrl. The ts*.erl modules and their associated data files in lib/test_server/src has been been moved to the new directory lib/common_test/test_server. The ts* modules are no longer built to lib/common_test/ebin. They will only built when 'make release_tests' is executed. The test suite for test_server has been moved to lib/common_test/test. The rest of the files have been deleted.
Diffstat (limited to 'lib/test_server/doc/src/basics_chapter.xml')
-rw-r--r--lib/test_server/doc/src/basics_chapter.xml215
1 files changed, 0 insertions, 215 deletions
diff --git a/lib/test_server/doc/src/basics_chapter.xml b/lib/test_server/doc/src/basics_chapter.xml
deleted file mode 100644
index 9e9f38aab4..0000000000
--- a/lib/test_server/doc/src/basics_chapter.xml
+++ /dev/null
@@ -1,215 +0,0 @@
-<?xml version="1.0" encoding="utf-8" ?>
-<!DOCTYPE chapter SYSTEM "chapter.dtd">
-
-<chapter>
- <header>
- <copyright>
- <year>2002</year><year>2013</year>
- <holder>Ericsson AB. All Rights Reserved.</holder>
- </copyright>
- <legalnotice>
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
- </legalnotice>
-
- <title>Test Server Basics</title>
- <prepared>Siri Hansen</prepared>
- <docno></docno>
- <date></date>
- <rev></rev>
- <file>basics_chapter.xml</file>
- </header>
-
- <section>
- <title>Introduction</title>
- <p><em>Test Server</em> is a portable test tool for automated
- testing of Erlang programs and OTP applications. It provides an
- interface for running test programs directly with Test Server
- as well as an interface for integrating Test Server
- with a framework application. The latter makes it possible to use
- Test Server as the engine of a higher level test tool
- application.</p>
-
- <p>It is strongly recommended that Test Server be used from inside
- a framework application, rather than interfaced directly for
- running test programs. Test Server can be pretty difficult to use
- since it's a very general and quite extensive and complex
- application. Furthermore, the <c>test_server_ctrl</c> functions
- are not meant to be used from within the actual test programs. The
- framework should handle communication with Test Server and deal
- with the more complex aspects of this interaction automatically so
- that a higher level interface may be provided for the tester. For
- test tool usage to be productive, a simpler, more intuitive and
- (if required) more specific interface is required than what Test Server
- can provide.</p>
-
- <p>OTP delivers a general purpose framework for Test Server, called
- <em>Common Test</em>. This application is a tool well suited for
- automated black box testing of target systems of <em>any kind</em>
- (not necessarily implemented in Erlang). Common Test is also a very
- useful tool for white box testing of Erlang programs and OTP
- applications. Unless a more specific functionality and/or user
- interface is required (in which case you might need to implement
- your own framework), Common Test should do the job for
- you. Please read the Common Test User's Guide and reference manual
- for more information.</p>
-
- <p>Under normal circumstances, knowledge about the Test Server
- application is not required for using the Common Test framework.
- However, if you want to use Test Server without a framework,
- or learn how to integrate it with your own framework, please read on...
- </p>
- </section>
- <section>
- <title>Getting started</title>
- <p>Testing when using Test Server is done by running test
- suites. A test suite is a number of test cases, where each test
- case tests one or more things. The test case is the smallest unit
- that the test server deals with. One or more test cases are
- grouped together into one ordinary Erlang module, which is called
- a test suite. Several test suite modules can be grouped together
- in special test specification files representing whole application
- and/or system test "jobs".
- </p>
- <p>The test suite Erlang module must follow a certain interface,
- which is specified by Test Server. See the section on writing
- test suites for details about this.
- </p>
- <p>Each test case is considered a success if it returns to the
- caller, no matter what the returned value is. An exception to this
- is the return value <c>{skip, Reason}</c> which indicates that the
- test case is skipped. A failure is specified as a crash, no matter
- what the crash reason is.
- </p>
- <p>As a test suite runs, all information (including output to
- stdout) is recorded in several different log files. A minimum of
- information is displayed to the user console. This only include
- start and stop information, plus a note for each failed test case.
- </p>
- <p>The result from each test case is recorded in an HTML log file
- which is created for each test run. Every test case gets one row
- in a table presenting total time, whether the case was successful
- or not, if it was skipped, and possibly also a comment. The HTML
- file has links to each test case's logfile, which may be viewed
- from e.g. Netscape or any other HTML capable browser.
- </p>
- <p>The Test Server consists of three parts:
- </p>
- <list type="bulleted">
- <item>The part that executes the test suites and
- provides support for the test suite author is called
- <c>test_server</c>. This is described in the chapter about
- writing test cases in this user's guide, and in the reference
- manual for the <c>test_server</c> module.</item>
- <item>The controlling part, which provides the low level
- operator interface, starts and stops slave nodes and writes
- log files, is called
- <c>test_server_ctrl</c>. The Test Server Controller should not
- be used directly when running tests. Instead a framework built
- on top of it should be used. More information
- about how to write your own framework can be found
- in this user's guide and in the reference manual for the
- <c>test_server_ctrl</c> module.</item>
- </list>
- </section>
-
- <section>
- <title>Definition of terms</title>
- <taglist>
- <tag><em>conf(iguration) case</em></tag>
- <item>This is a group of test cases which need some specific
- configuration. A conf case contains an initiation function which
- sets up a specific configuration, one or more test cases using
- this configuration, and a cleanup function which restores the
- configuration. A conf case is specified in a test specification
- either like this:<c>{conf,InitFunc,ListOfCases,CleanupFunc}</c>,
- or this: <c>{conf,Properties,InitFunc,ListOfCases,CleanupFunc}</c>
- </item>
- <tag><em>datadir</em></tag>
- <item>Data directory for a test suite. This directory contains
- any files used by the test suite, e.g. additional erlang
- modules, c code or data files. If the data directory contains
- code which must be compiled before the test suite is run, it
- should also contain a makefile source called Makefile.src
- defining how to compile.
- </item>
- <tag><em>documentation clause</em></tag>
- <item>One of the function clauses in a test case. This clause
- shall return a list of strings describing what the test case
- tests.
- </item>
- <tag><em>execution clause</em></tag>
- <item>One of the function clauses in a test case. This clause
- implements the actual test case, i.e. calls the functions that
- shall be tested and checks results. The clause shall crash if it
- fails.
- </item>
- <tag><em>major log file</em></tag>
- <item>This is the test suites log file.
- </item>
- <tag><em>Makefile.src</em></tag>
- <item>This file is used by the test server framework to generate
- a makefile for a datadir. It contains some special characters
- which are replaced according to the platform currently tested.
- </item>
- <tag><em>minor log file</em></tag>
- <item>This is a separate log file for each test case.
- </item>
- <tag><em>privdir</em></tag>
- <item>Private directory for a test suite. This directory should
- be used when the test suite needs to write to files.
- </item>
- <tag><em>skip case</em></tag>
- <item>A test case which shall be skipped.
- </item>
- <tag><em>specification clause</em></tag>
- <item>One of the function clauses in a test case. This clause
- shall return an empty list, a test specification or
- <c>{skip,Reason}</c>. If an empty list is returned, it means
- that the test case shall be executed, and so it must also have
- an execution clause.
- </item>
- <tag><em>test case</em></tag>
- <item>A single test included in a test suite. Typically it tests
- one function in a module or application. A test case is
- implemented as a function in a test suite module. The function
- can have three clauses, the documentation-, specification- and
- execution clause.
- </item>
- <tag><em>test specification</em></tag>
- <item>A specification of which test suites and test cases to
- run. There can be test specifications on three different levels
- in a test. The top level is a test specification file which
- roughly specifies what to test for a whole application. Then
- there is a test specification for each test suite returned from
- the <c>all(suite)</c> function in the suite. And there can also
- be a test specification returned from the specification clause
- of a test case.
- </item>
- <tag><em>test specification file</em></tag>
- <item>This is a text file containing the test specification for
- an application. The file has the extension ".spec" or
- ".spec.Platform", where Platform is e.g. "vxworks".
- </item>
- <tag><em>test suite</em></tag>
- <item>An erlang module containing a collection of test cases for
- a specific application or module.
- </item>
- <tag><em>topcase</em></tag>
- <item>The first "command" in a test specification file. This
- command contains the test specification, like this:
- <c>{topcase,TestSpecification}</c></item>
- </taglist>
- </section>
-</chapter>
-