The R13B03 release.OTP_R13B03

1 files changed, 199 insertions, 0 deletions

diff --git a/lib/common_test/doc/src/ct_master_chapter.xml b/lib/common_test/doc/src/ct_master_chapter.xml
new file mode 100644
index 0000000000..79288cfe4c
--- /dev/null
+++ b/lib/common_test/doc/src/ct_master_chapter.xml
@@ -0,0 +1,199 @@
+<?xml version="1.0" encoding="latin1" ?>
+<!DOCTYPE chapter SYSTEM "chapter.dtd">
+
+<chapter>
+  <header>
+    <copyright>
+      <year>2006</year><year>2009</year>
+      <holder>Ericsson AB. All Rights Reserved.</holder>
+    </copyright>
+    <legalnotice>
+      The contents of this file are subject to the Erlang Public License,
+      Version 1.1, (the "License"); you may not use this file except in
+      compliance with the License. You should have received a copy of the
+      Erlang Public License along with this software. If not, it can be
+      retrieved online at http://www.erlang.org/.
+    
+      Software distributed under the License is distributed on an "AS IS"
+      basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
+      the License for the specific language governing rights and limitations
+      under the License.
+    
+    </legalnotice>
+
+    <title>Using Common Test for Large Scale Testing</title>
+    <prepared>Peter Andersson</prepared>
+    <docno></docno>
+    <date></date>
+    <rev></rev>
+    <file>ct_master_chapter.xml</file>
+  </header>
+
+  <section>
+    <title>General</title>
+    <p>Large scale automated testing requires running multiple independent 
+      test sessions in parallel. This is accomplished by running
+      a number of Common Test nodes on one or more hosts, testing
+      different target systems. Configuring, starting and controlling the
+      test nodes independently can be a cumbersome operation. To aid
+      this kind of automated large scale testing, CT offers a master test 
+      node component, CT Master, that handles central configuration and control
+      in a system of distributed CT nodes.</p>
+
+    <p>The CT Master server runs on one dedicated Erlang node and uses distributed
+      Erlang to communicate with any number of CT test nodes, each hosting a regular
+      CT server. Test specifications are used as input to specify what to test on which 
+      test nodes, using what configuration.</p> 
+
+    <p>The CT Master server writes progress information to HTML log files similarly 
+      to the regular CT server. The logs contain test statistics and links to the 
+      log files written by each independent CT server.</p>
+
+    <p>The CT master API is exported by the <c>ct_master</c> module.</p>
+  </section>
+  <section>
+    <title>Usage</title>
+    <p>CT Master requires all test nodes to be on the same network and share a common 
+      file system. As of this date, CT Master can not start test nodes
+      automatically. The nodes must have been started in advance for CT Master to be 
+      able to start test sessions on them.</p>
+      
+    <p>Tests are started by calling:</p>
+    
+    <p><c>ct_master:run(TestSpecs)</c> or 
+       <c>ct_master:run(TestSpecs, InclNodes, ExclNodes)</c></p>
+    
+    <p><c>TestSpecs</c> is either the name of a test specification file (string) or a list 
+      of test specifications. In case of a list, the specifications will be handled (and
+      the corresponding tests executed) in sequence. An element in a <c>TestSpecs</c> list 
+      can also be list of test specifications. The specifications in such a list will be 
+      merged into one combined specification prior to test execution. For example:</p>
+      
+    <p><c>ct_master:run(["ts1","ts2",["ts3","ts4"]])</c></p>
+
+    <p>means first the tests specified by "ts1" will run, then the tests specified by "ts2" 
+      and finally the tests specified by both "ts3" and "ts4".</p>
+
+    <p>The <c>InclNodes</c> argument to <c>run/3</c> is a list of node names. The <c>run/3</c> 
+      function runs the tests in <c>TestSpecs</c> just like <c>run/1</c> but will also 
+      take any test in <c>TestSpecs</c> that's not explicitly tagged with a particular 
+      node name and execute it on the nodes listed in <c>InclNodes</c>. By using <c>run/3</c> 
+      this way it is possible to use any test specification, with or without node information, 
+      in a large scale test environment! <c>ExclNodes</c> is a list of nodes that should be
+      excluded from the test. I.e. tests that have been specified in the test specification 
+      to run on a particular node will not be performed if that node is at runtime
+      listed in <c>ExclNodes</c>.</p>
+
+      <p>If CT Master fails initially to connect to any of the test nodes specified in a 
+      test specification or in the <c>InclNodes</c> list, the operator will be prompted with 
+      the option to either start over again (after manually checking the status of the 
+      node(s) in question), to run without the missing nodes, or to abort the operation.</p>
+
+      <p>When tests start, CT Master prints information to console about the nodes that are 
+      involved. CT Master also reports when tests finish, successfully or unsuccessfully. If
+      connection is lost to a node, the test on that node is considered finished. CT Master 
+      will not attempt to reestablish contact with the failing node. At any time to get the
+      current status of the test nodes, call the function:</p>
+
+      <p><c>ct_master:progress()</c></p>
+
+      <p>To stop one or more tests, use:</p>
+
+      <p><c>ct_master:abort()</c> (stop all) or <c>ct_master:abort(Nodes)</c></p>
+
+      <p>For detailed information about the <c>ct_master</c> API, please see the 
+      manual page for this module.</p>
+  </section>
+  <section>
+    <title>Test Specifications</title>
+    <p>The test specifications used as input to CT Master are fully compatible with the
+    specifications used as input to the regular CT server. The syntax is described in the 
+    <seealso marker="run_test_chapter#test_specifications">Running Test Suites</seealso>
+    chapter.</p>
+
+    <p>All test specification terms can have a <c>NodeRefs</c> element. This element
+    specifies which node or nodes a configuration operation or a test is to be executed 
+    on. <c>NodeRefs</c> is defined as:</p>
+    
+    <p><c>NodeRefs = all_nodes | [NodeRef] | NodeRef</c></p>
+    
+    <p>where</p>
+
+    <p><c>NodeRef = NodeAlias | node() | master</c></p>
+
+    <p>A <c>NodeAlias</c> (<c>atom()</c>) is used in a test specification as a 
+    reference to a node name (so the actual node name only needs to be declared once). 
+    The alias is declared with a <c>node</c> term:</p>
+
+    <p><c>{node, NodeAlias, NodeName}</c></p>
+
+    <p>If <c>NodeRefs</c> has the value <c>all_nodes</c>, the operation or test will
+    be performed on all given test nodes. (Declaring a term without a <c>NodeRefs</c> 
+    element actually has the same effect). If <c>NodeRefs</c> has the value 
+    <c>master</c>, the operation is only performed on the CT Master node (namely set 
+    the log directory or install an event handler).</p>
+
+    <p>Consider the example in the 
+    <seealso marker="run_test_chapter#test_specifications">Running Test Suites</seealso>
+    chapter, now extended with node information and intended to be executed by the
+    CT Master:</p>
+
+    <pre>
+      {node, node1, ct_node@host_x}.
+      {node, node2, ct_node@host_y}.
+ 
+      {logdir, master, "/home/test/master_logs"}.
+      {logdir, "/home/test/logs"}.
+      
+      {config, node1, "/home/test/t1/cfg/config.cfg"}.
+      {config, node2, "/home/test/t2/cfg/config.cfg"}.
+      {config, "/home/test/t3/cfg/config.cfg"}.
+      
+      {alias, t1, "/home/test/t1"}.
+      {alias, t2, "/home/test/t2"}.
+      {alias, t3, "/home/test/t3"}.
+      
+      {suites, node1, t1, all}.
+      {skip_suites, node1, t1, [t1B_SUITE,t1D_SUITE], "Not implemented"}.
+      {skip_cases, node1, t1, t1A_SUITE, [test3,test4], "Irrelevant"}.
+      {skip_cases, node1, t1, t1C_SUITE, [test1], "Ignore"}.
+      
+      {suites, node2, t2, [t2B_SUITE,t2C_SUITE]}.
+      {cases, node2, t2, t2A_SUITE, [test4,test1,test7]}.
+      
+      {skip_suites, t3, all, "Not implemented"}.
+    </pre>
+
+    <p>This example specifies the same tests as the original example. But 
+    now if started with a call to <c>ct_master:run(TestSpecName)</c>, the 
+    t1 test will be executed on node <c>ct_node@host_x</c> (node1), the
+    t2 test on <c>ct_node@host_y</c> (node2) and the t3 test on both
+    node1 and node2. The t1 config file will only be read on
+    node1 and the t2 config file only on node2, while the t3 config file
+    will be read on both node1 and node2. Both test nodes will write log 
+    files to the same directory. (The CT Master node will however use a 
+    different log directory than the test nodes).</p>
+
+    <p>If the test session is instead started with a call to 
+    <c>ct_master:run(TestSpecName, [ct_node@host_z], [ct_node@host_x])</c>, 
+    the result is that the t1 test does not run on 
+    <c>ct_node@host_x</c> (or any other node) while the t3 test runs on
+    <c>ct_node@host_y</c> and <c>ct_node@host_z</c>.</p>
+    
+    <p>A nice feature is that a test specification that includes node 
+      information can still be used as input to the regular Common Test server 
+      (as described in the
+    <seealso marker="run_test_chapter#test_specifications">Running Test Suites</seealso>
+    chapter). The result is that any test specified to run on a node with the same
+    name as the Common Test node in question (typically <c>ct@somehost</c> if started
+    with the <c>run_test</c> script), will be performed. Tests without explicit
+    node association will always be performed too of course!</p>
+
+    <note><p>It is recommended that absolute paths are used for log directories,
+    config files and test directory aliases in the test specifications so that
+    current working directory settings are not important.</p></note>
+  </section>
+
+</chapter>
+
+