The R13B03 release.OTP_R13B03

1 files changed, 216 insertions, 0 deletions

diff --git a/lib/test_server/doc/src/basics_chapter.xml b/lib/test_server/doc/src/basics_chapter.xml
new file mode 100644
index 0000000000..a96cc88075
--- /dev/null
+++ b/lib/test_server/doc/src/basics_chapter.xml
@@ -0,0 +1,216 @@
+<?xml version="1.0" encoding="latin1" ?>
+<!DOCTYPE chapter SYSTEM "chapter.dtd">
+
+<chapter>
+  <header>
+    <copyright>
+      <year>2002</year><year>2009</year>
+      <holder>Ericsson AB. All Rights Reserved.</holder>
+    </copyright>
+    <legalnotice>
+      The contents of this file are subject to the Erlang Public License,
+      Version 1.1, (the "License"); you may not use this file except in
+      compliance with the License. You should have received a copy of the
+      Erlang Public License along with this software. If not, it can be
+      retrieved online at http://www.erlang.org/.
+    
+      Software distributed under the License is distributed on an "AS IS"
+      basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
+      the License for the specific language governing rights and limitations
+      under the License.
+    
+    </legalnotice>
+
+    <title>Test Server Basics</title>
+    <prepared>Siri Hansen</prepared>
+    <docno></docno>
+    <date></date>
+    <rev></rev>
+    <file>basics_chapter.xml</file>
+  </header>
+
+  <section>
+    <title>Introduction</title>
+    <p><em>Test Server</em> is a portable test tool for automated
+    testing of Erlang programs and OTP applications. It provides an
+    interface for running test programs directly with Test Server 
+    as well as an interface for integrating Test Server
+    with a framework application. The latter makes it possible to use
+    Test Server as the engine of a higher level test tool
+    application.</p>
+
+    <p>It is strongly recommended that Test Server be used from inside
+    a framework application, rather than interfaced directly for
+    running test programs. Test Server can be pretty difficult to use
+    since it's a very general and quite extensive and complex
+    application. Furthermore, the <c>test_server_ctrl</c> functions
+    are not meant to be used from within the actual test programs. The
+    framework should handle communication with Test Server and deal
+    with the more complex aspects of this interaction automatically so
+    that a higher level interface may be provided for the tester. For
+    test tool usage to be productive, a simpler, more intuitive and 
+    (if required) more specific interface is required than what Test Server
+    can provide.</p>
+
+    <p>OTP delivers a general purpose framework for Test Server, called
+    <em>Common Test</em>. This application is a tool well suited for
+    automated black box testing of target systems of <em>any kind</em> 
+    (not necessarily implemented in Erlang). Common Test is also a very
+    useful tool for white box testing of Erlang programs and OTP
+    applications. Unless a more specific functionality and/or user 
+    interface is required (in which case you might need to implement 
+    your own framework), Common Test should do the job for 
+    you. Please read the Common Test User's Guide and reference manual 
+    for more information.</p>
+
+    <p>Under normal circumstances, knowledge about the Test Server
+    application is not required for using the Common Test framework.
+    However, if you want to use Test Server without a framework,
+    or learn how to integrate it with your own framework, please read on...
+    </p>
+    </section>
+    <section>
+    <title>Getting started</title>
+    <p>Testing when using Test Server is done by running test
+      suites. A test suite is a number of test cases, where each test
+      case tests one or more things. The test case is the smallest unit
+      that the test server deals with. One or more test cases are
+      grouped together into one ordinary Erlang module, which is called
+      a test suite. Several test suite modules can be grouped together
+      in special test specification files representing whole application
+      and/or system test "jobs".
+      </p>
+    <p>The test suite Erlang module must follow a certain interface,
+      which is specified by Test Server. See the section on writing
+      test suites for details about this.
+      </p>
+    <p>Each test case is considered a success if it returns to the
+      caller, no matter what the returned value is. An exception to this
+      is the return value <c>{skip, Reason}</c> which indicates that the
+      test case is skipped. A failure is specified as a crash, no matter
+      what the crash reason is.
+      </p>
+    <p>As a test suite runs, all information (including output to
+      stdout) is recorded in several different log files. A minimum of
+      information is displayed to the user console. This only include
+      start and stop information, plus a note for each failed test case.
+      </p>
+    <p>The result from each test case is recorded in an HTML log file
+      which is created for each test run. Every test case gets one row
+      in a table presenting total time, whether the case was successful
+      or not, if it was skipped, and possibly also a comment. The HTML
+      file has links to each test case's logfile, which may be viewed
+      from e.g. Netscape or any other HTML capable browser.
+      </p>
+    <p>The Test Server consists of three parts: 
+      </p>
+    <list type="bulleted">
+      <item>The part that executes the test suites on target and
+       provides support for the test suite author is called
+      <c>test_server</c>. This is described in the chapter about
+       writing test cases in this user's guide, and in the reference
+       manual for the <c>test_server</c> module.</item>
+      <item>The controlling part, which provides the low level
+       operator interface, starts and stops the target node (if remote
+       target) and slave nodes and writes log files, is called
+      <c>test_server_ctrl</c>. The Test Server Controller should not
+       be used directly when running tests. Instead a framework built
+       on top of it should be used. More information
+       about how to write your own framework can be found
+       in this user's guide and in the reference manual for the
+      <c>test_server_ctrl</c> module.</item>
+    </list>
+  </section>
+
+  <section>
+    <title>Definition of terms</title>
+    <taglist>
+      <tag><em>conf(iguration) case</em></tag>
+      <item>This is a group of test cases which need some specific
+       configuration. A conf case contains an initiation function which
+       sets up a specific configuration, one or more test cases using
+       this configuration, and a cleanup function which restores the
+       configuration. A conf case is specified in a test specification
+       either like this:<c>{conf,InitFunc,ListOfCases,CleanupFunc}</c>,
+       or this: <c>{conf,Properties,InitFunc,ListOfCases,CleanupFunc}</c>
+       </item>
+      <tag><em>datadir</em></tag>
+      <item>Data directory for a test suite. This directory contains
+       any files used by the test suite, e.g. additional erlang
+       modules, c code or data files. If the data directory contains
+       code which must be compiled before the test suite is run, it
+       should also contain a makefile source called Makefile.src
+       defining how to compile.
+      </item>
+      <tag><em>documentation clause</em></tag>
+      <item>One of the function clauses in a test case. This clause
+       shall return a list of strings describing what the test case
+       tests.
+      </item>
+      <tag><em>execution clause</em></tag>
+      <item>One of the function clauses in a test case. This clause
+       implements the actual test case, i.e. calls the functions that
+       shall be tested and checks results. The clause shall crash if it
+       fails.
+      </item>
+      <tag><em>major log file</em></tag>
+      <item>This is the test suites log file.
+      </item>
+      <tag><em>Makefile.src</em></tag>
+      <item>This file is used by the test server framework to generate
+       a makefile for a datadir. It contains some special characters
+       which are replaced according to the platform currently tested.
+      </item>
+      <tag><em>minor log file</em></tag>
+      <item>This is a separate log file for each test case.
+      </item>
+      <tag><em>privdir</em></tag>
+      <item>Private directory for a test suite. This directory should
+       be used when the test suite needs to write to files.
+      </item>
+      <tag><em>skip case</em></tag>
+      <item>A test case which shall be skipped.
+      </item>
+      <tag><em>specification clause</em></tag>
+      <item>One of the function clauses in a test case. This clause
+       shall return an empty list, a test specification or
+      <c>{skip,Reason}</c>. If an empty list is returned, it means
+       that the test case shall be executed, and so it must also have
+       an execution clause. Note that the specification clause is
+       always executed on the controller node, i.e. not on the target
+       node.
+      </item>
+      <tag><em>test case</em></tag>
+      <item>A single test included in a test suite. Typically it tests
+       one function in a module or application. A test case is
+       implemented as a function in a test suite module. The function
+       can have three clauses, the documentation-, specification- and
+       execution clause.
+      </item>
+      <tag><em>test specification</em></tag>
+      <item>A specification of which test suites and test cases to
+       run. There can be test specifications on three different levels
+       in a test. The top level is a test specification file which
+       roughly specifies what to test for a whole application. Then
+       there is a test specification for each test suite returned from
+       the <c>all(suite)</c> function in the suite. And there can also
+       be a test specification returned from the specification clause
+       of a test case.
+      </item>
+      <tag><em>test specification file</em></tag>
+      <item>This is a text file containing the test specification for
+       an application. The file has the extension ".spec" or
+       ".spec.Platform", where Platform is e.g. "vxworks".
+      </item>
+      <tag><em>test suite</em></tag>
+      <item>An erlang module containing a collection of test cases for
+       a specific application or module.
+      </item>
+      <tag><em>topcase</em></tag>
+      <item>The first "command" in a test specification file. This
+       command contains the test specification, like this:
+      <c>{topcase,TestSpecification}</c></item>
+    </taglist>
+  </section>
+</chapter>
+