From 84adefa331c4159d432d22840663c38f155cd4c1 Mon Sep 17 00:00:00 2001 From: Erlang/OTP Date: Fri, 20 Nov 2009 14:54:40 +0000 Subject: The R13B03 release. --- lib/common_test/doc/src/ct_master_chapter.xml | 199 ++++++++++++++++++++++++++ 1 file changed, 199 insertions(+) create mode 100644 lib/common_test/doc/src/ct_master_chapter.xml (limited to 'lib/common_test/doc/src/ct_master_chapter.xml') diff --git a/lib/common_test/doc/src/ct_master_chapter.xml b/lib/common_test/doc/src/ct_master_chapter.xml new file mode 100644 index 0000000000..79288cfe4c --- /dev/null +++ b/lib/common_test/doc/src/ct_master_chapter.xml @@ -0,0 +1,199 @@ + + + + +
+ + 20062009 + Ericsson AB. All Rights Reserved. + + + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + + + Using Common Test for Large Scale Testing + Peter Andersson + + + + ct_master_chapter.xml +
+ +
+ General +

Large scale automated testing requires running multiple independent + test sessions in parallel. This is accomplished by running + a number of Common Test nodes on one or more hosts, testing + different target systems. Configuring, starting and controlling the + test nodes independently can be a cumbersome operation. To aid + this kind of automated large scale testing, CT offers a master test + node component, CT Master, that handles central configuration and control + in a system of distributed CT nodes.

+ +

The CT Master server runs on one dedicated Erlang node and uses distributed + Erlang to communicate with any number of CT test nodes, each hosting a regular + CT server. Test specifications are used as input to specify what to test on which + test nodes, using what configuration.

+ +

The CT Master server writes progress information to HTML log files similarly + to the regular CT server. The logs contain test statistics and links to the + log files written by each independent CT server.

+ +

The CT master API is exported by the ct_master module.

+
+
+ Usage +

CT Master requires all test nodes to be on the same network and share a common + file system. As of this date, CT Master can not start test nodes + automatically. The nodes must have been started in advance for CT Master to be + able to start test sessions on them.

+ +

Tests are started by calling:

+ +

ct_master:run(TestSpecs) or + ct_master:run(TestSpecs, InclNodes, ExclNodes)

+ +

TestSpecs is either the name of a test specification file (string) or a list + of test specifications. In case of a list, the specifications will be handled (and + the corresponding tests executed) in sequence. An element in a TestSpecs list + can also be list of test specifications. The specifications in such a list will be + merged into one combined specification prior to test execution. For example:

+ +

ct_master:run(["ts1","ts2",["ts3","ts4"]])

+ +

means first the tests specified by "ts1" will run, then the tests specified by "ts2" + and finally the tests specified by both "ts3" and "ts4".

+ +

The InclNodes argument to run/3 is a list of node names. The run/3 + function runs the tests in TestSpecs just like run/1 but will also + take any test in TestSpecs that's not explicitly tagged with a particular + node name and execute it on the nodes listed in InclNodes. By using run/3 + this way it is possible to use any test specification, with or without node information, + in a large scale test environment! ExclNodes is a list of nodes that should be + excluded from the test. I.e. tests that have been specified in the test specification + to run on a particular node will not be performed if that node is at runtime + listed in ExclNodes.

+ +

If CT Master fails initially to connect to any of the test nodes specified in a + test specification or in the InclNodes list, the operator will be prompted with + the option to either start over again (after manually checking the status of the + node(s) in question), to run without the missing nodes, or to abort the operation.

+ +

When tests start, CT Master prints information to console about the nodes that are + involved. CT Master also reports when tests finish, successfully or unsuccessfully. If + connection is lost to a node, the test on that node is considered finished. CT Master + will not attempt to reestablish contact with the failing node. At any time to get the + current status of the test nodes, call the function:

+ +

ct_master:progress()

+ +

To stop one or more tests, use:

+ +

ct_master:abort() (stop all) or ct_master:abort(Nodes)

+ +

For detailed information about the ct_master API, please see the + manual page for this module.

+
+
+ Test Specifications +

The test specifications used as input to CT Master are fully compatible with the + specifications used as input to the regular CT server. The syntax is described in the + Running Test Suites + chapter.

+ +

All test specification terms can have a NodeRefs element. This element + specifies which node or nodes a configuration operation or a test is to be executed + on. NodeRefs is defined as:

+ +

NodeRefs = all_nodes | [NodeRef] | NodeRef

+ +

where

+ +

NodeRef = NodeAlias | node() | master

+ +

A NodeAlias (atom()) is used in a test specification as a + reference to a node name (so the actual node name only needs to be declared once). + The alias is declared with a node term:

+ +

{node, NodeAlias, NodeName}

+ +

If NodeRefs has the value all_nodes, the operation or test will + be performed on all given test nodes. (Declaring a term without a NodeRefs + element actually has the same effect). If NodeRefs has the value + master, the operation is only performed on the CT Master node (namely set + the log directory or install an event handler).

+ +

Consider the example in the + Running Test Suites + chapter, now extended with node information and intended to be executed by the + CT Master:

+ + 
+      {node, node1, ct_node@host_x}.
+      {node, node2, ct_node@host_y}.
+ 
+      {logdir, master, "/home/test/master_logs"}.
+      {logdir, "/home/test/logs"}.
+      
+      {config, node1, "/home/test/t1/cfg/config.cfg"}.
+      {config, node2, "/home/test/t2/cfg/config.cfg"}.
+      {config, "/home/test/t3/cfg/config.cfg"}.
+      
+      {alias, t1, "/home/test/t1"}.
+      {alias, t2, "/home/test/t2"}.
+      {alias, t3, "/home/test/t3"}.
+      
+      {suites, node1, t1, all}.
+      {skip_suites, node1, t1, [t1B_SUITE,t1D_SUITE], "Not implemented"}.
+      {skip_cases, node1, t1, t1A_SUITE, [test3,test4], "Irrelevant"}.
+      {skip_cases, node1, t1, t1C_SUITE, [test1], "Ignore"}.
+      
+      {suites, node2, t2, [t2B_SUITE,t2C_SUITE]}.
+      {cases, node2, t2, t2A_SUITE, [test4,test1,test7]}.
+      
+      {skip_suites, t3, all, "Not implemented"}.
+
+ +

This example specifies the same tests as the original example. But + now if started with a call to ct_master:run(TestSpecName), the + t1 test will be executed on node ct_node@host_x (node1), the + t2 test on ct_node@host_y (node2) and the t3 test on both + node1 and node2. The t1 config file will only be read on + node1 and the t2 config file only on node2, while the t3 config file + will be read on both node1 and node2. Both test nodes will write log + files to the same directory. (The CT Master node will however use a + different log directory than the test nodes).

+ +

If the test session is instead started with a call to + ct_master:run(TestSpecName, [ct_node@host_z], [ct_node@host_x]), + the result is that the t1 test does not run on + ct_node@host_x (or any other node) while the t3 test runs on + ct_node@host_y and ct_node@host_z.

+ +

A nice feature is that a test specification that includes node + information can still be used as input to the regular Common Test server + (as described in the + Running Test Suites + chapter). The result is that any test specified to run on a node with the same + name as the Common Test node in question (typically ct@somehost if started + with the run_test script), will be performed. Tests without explicit + node association will always be performed too of course!

+ +

It is recommended that absolute paths are used for log directories, + config files and test directory aliases in the test specifications so that + current working directory settings are not important.

 +
+ + + + -- cgit v1.2.3