From 6668a91b365b8390d8eb369f7d6c6294711a1fef Mon Sep 17 00:00:00 2001 From: Peter Andersson Date: Thu, 1 Sep 2011 16:32:36 +0200 Subject: Various corrections and updates to improve error handling and reporting OTP-8933 --- lib/common_test/doc/src/config_file_chapter.xml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'lib/common_test/doc') diff --git a/lib/common_test/doc/src/config_file_chapter.xml b/lib/common_test/doc/src/config_file_chapter.xml index 59151a73ec..6fc6638bf7 100644 --- a/lib/common_test/doc/src/config_file_chapter.xml +++ b/lib/common_test/doc/src/config_file_chapter.xml @@ -285,7 +285,7 @@ {ok, Config} - if the configuration variables are read successfully, - {error, Error, ErrorDetails} - if the callback module fails to + {error, {Error, ErrorDetails}} - if the callback module fails to proceed with the given configuration parameters. @@ -422,14 +422,14 @@ stop()-> call(Client, Request)-> case whereis(?REGISTERED_NAME) of undefined-> - {error, not_started, Request}; + {error, {not_started, Request}}; Pid-> Pid ! {Client, Request}, receive Reply-> {ok, Reply} after 4000-> - {error, timeout, Request} + {error, {timeout, Request}} end end. -- cgit v1.2.3 From e91ba727f534553a34c05ad54247a0485e2429ff Mon Sep 17 00:00:00 2001 From: Peter Andersson Date: Mon, 5 Sep 2011 19:51:59 +0200 Subject: Implement support for running suites with test case groups through the debugger OTP-9518 --- lib/common_test/doc/src/run_test_chapter.xml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'lib/common_test/doc') diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index e668568795..e944f215f6 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -322,8 +322,9 @@ are to be executed by Common Test, and those functions only. If the step option config is specified, breakpoints will also be initially set on the configuration functions in the suite, i.e. - init_per_suite/1, end_per_suite/1, init_per_testcase/2 - and end_per_testcase/2.

+ init_per_suite/1, end_per_suite/1, + init_per_group/2, end_per_group/2, + init_per_testcase/2 and end_per_testcase/2.

Common Test enables the Debugger auto attach feature, which means that for every new interpreted test case function that starts to execute, a new trace window will automatically pop up. (This is because each test -- cgit v1.2.3 From e78be5c09eda6a8738d07d801dd8935f3eaa5400 Mon Sep 17 00:00:00 2001 From: Peter Andersson Date: Thu, 29 Sep 2011 00:21:27 +0200 Subject: Add documentation on timetraps and start flags --- lib/common_test/doc/src/common_test_app.xml | 32 +++++++++++++++++++------- lib/common_test/doc/src/write_test_chapter.xml | 14 +++++++---- 2 files changed, 34 insertions(+), 12 deletions(-) (limited to 'lib/common_test/doc') diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml index 57b032b3fd..f58b2ab0a9 100644 --- a/lib/common_test/doc/src/common_test_app.xml +++ b/lib/common_test/doc/src/common_test_app.xml @@ -133,9 +133,15 @@ {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} | {ct_hooks, CTHs} - Time = MilliSec | {seconds,integer()} | {minutes,integer()} - | {hours,integer()} + Time = TimeVal | TimeFunc + TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | + {hours,integer()} + TimeFunc = {Mod,Func,Args} | Fun MilliSec = integer() + Mod = atom() + Func = atom() + Args = list() + Fun = fun() Required = Key | {Key,SubKeys} Key = atom() SubKeys = SubKey | [SubKey] @@ -161,7 +167,9 @@ test case is allowed to take (including init_per_testcase/2 and end_per_testcase/2). If the timetrap time is exceeded, the test case fails with reason - timetrap_timeout.

+ timetrap_timeout. If a TimeFunc function is specified, + it will be called initially and must return a value on + TimeVal format.

The require tag specifies configuration variables that are required by test cases in the suite. If the required @@ -248,7 +256,7 @@ -

MANDATORY (only if one or more groups are defined)

+

OPTIONAL

This function is called before execution of a test case group. It typically contains initialization which is common for @@ -279,7 +287,7 @@ -

MANDATORY (only if one or more groups are defined)

+

OPTIONAL

This function is called after the execution of a test case group is finished. It is meant to be used for cleaning up after init_per_group/2. @@ -353,9 +361,15 @@ Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns} - Time = MilliSec | {seconds,integer()} | {minutes,integer()} - | {hours,integer()} + Time = TimeVal | TimeFunc + TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | + {hours,integer()} + TimeFunc = {Mod,Func,Args} | Fun MilliSec = integer() + Mod = atom() + Func = atom() + Args = list() + Fun = fun() Required = Key | {Key,SubKeys} Key = atom() SubKeys = SubKey | [SubKey] @@ -378,7 +392,9 @@ exceeded, the test case fails with reason timetrap_timeout. init_per_testcase/2 and end_per_testcase/2 are included in the - timetrap time.

+ timetrap time. If a TimeFunc function is specified, + it will be called before the test case (or init_per_testcase/2) + and must return a value on TimeVal format.

The require tag specifies configuration variables that are required by the test case. If the required diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml index 3f9fdb7121..e35888e68f 100644 --- a/lib/common_test/doc/src/write_test_chapter.xml +++ b/lib/common_test/doc/src/write_test_chapter.xml @@ -280,6 +280,8 @@ the timetrap time is exceeded, the test case fails with reason timetrap_timeout. Note that init_per_testcase and end_per_testcase are included in the timetrap time. + Please see the Timetrap + section for more details.

userdata @@ -699,8 +701,8 @@
- Timetrap timeouts + Timetrap timeouts

The default time limit for a test case is 30 minutes, unless a timetrap is specified either by the suite info function or a test case info function. The timetrap timeout value defined @@ -723,6 +725,13 @@ multipled by multiply_timetraps, and possibly scaled up if scale_timetraps is enabled, the function ct:sleep/1 may be called.

+

A function (fun or MFA) may be specified as timetrap value + in the suite- and test case info function, e.g:

+

{timetrap,{test_utils,get_timetrap_value,[?MODULE,system_start]}}

+

The function will be called initially by Common Test (before execution + of the suite or the test case) and must return a time value such as an + integer (millisec), or a {SecMinOrHourTag,Time} tuple. More + information can be found in the common_test reference manual.

@@ -818,6 +827,3 @@
- - - -- cgit v1.2.3 From 09fbbccd5a8638601e44c605cd887f90b85d7d60 Mon Sep 17 00:00:00 2001 From: Peter Andersson Date: Fri, 30 Sep 2011 13:26:59 +0200 Subject: Update documentation OTP-9396 OTP-9372 OTP-9155 --- lib/common_test/doc/src/ct_run.xml | 7 ++++-- lib/common_test/doc/src/run_test_chapter.xml | 35 ++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 2 deletions(-) (limited to 'lib/common_test/doc') diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index 1ab563d74f..9045646733 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -83,7 +83,7 @@ Run tests from command line 
 	ct_run [-dir TestDir1 TestDir2 .. TestDirN] |
-	[-suite Suite1 Suite2 .. SuiteN
+	[[-dir TestDir] -suite Suite1 Suite2 .. SuiteN
 	 [[-group Group1 Group2 .. GroupN] [-case Case1 Case2 .. CaseN]]]
 	[-step [config | keep_inactive]]
 	[-config ConfigFile1 ConfigFile2 .. ConfigFileN]
@@ -92,6 +92,7 @@
 	[-decrypt_key Key] | [-decrypt_file KeyFile]
 	[-label Label]
 	[-logdir LogDir]
+	[-logopts LogOpts]
 	[-silent_connections [ConnType1 ConnType2 .. ConnTypeN]]
 	[-stylesheet CSSFile]
 	[-cover CoverCfgFile]
@@ -117,6 +118,7 @@
 	[-decrypt_key Key] | [-decrypt_file KeyFile]
 	[-label Label]
 	[-logdir LogDir]
+	[-logopts LogOpts]
 	[-allow_user_terms]
 	[-silent_connections [ConnType1 ConnType2 .. ConnTypeN]]
 	[-stylesheet CSSFile]
@@ -138,10 +140,11 @@
     
         ct_run -vts [-browser Browser]
         [-dir TestDir1 TestDir2 .. TestDirN] |
-        [-suite Suite [[-group Group] [-case Case]]]
+        [[dir TestDir] -suite Suite [[-group Group] [-case Case]]]
 	[-config ConfigFile1 ConfigFile2 .. ConfigFileN]
 	[-userconfig CallbackModule1 ConfigString1 and CallbackModule2
          ConfigString2 and .. and CallbackModuleN ConfigStringN]
+	[-logopts LogOpts]
 	[-decrypt_key Key] | [-decrypt_file KeyFile]
 	[-include InclDir1 InclDir2 .. InclDirN]
 	[-no_auto_compile]
diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml
index e944f215f6..d3c6847d85 100644
--- a/lib/common_test/doc/src/run_test_chapter.xml
+++ b/lib/common_test/doc/src/run_test_chapter.xml
@@ -128,6 +128,15 @@
     
$ ct_run -suite $SYS1_TEST/setup_SUITE -case start stop

     
$ ct_run -suite $SYS1_TEST/setup_SUITE -group installation -case start stop

     
+    
It is also possible to combine the dir, suite and group/case flags. E.g, to run
+    x_SUITE and y_SUITE in directory testdir:

+
+    
$ ct_run -dir ./testdir -suite x_SUITE y_SUITE

+
+    
This has the same effect as calling:

+
+    
$ ct_run -suite ./testdir/x_SUITE ./testdir/y_SUITE

+
     
Other flags that may be used with ct_run:

     
       ]]>, specifies where the HTML log files are to be written.
@@ -165,6 +174,8 @@
       ]]>, points out a file containing a decryption key for 
         encrypted configuration files.
       , switches off html enhancements that might not be compatible with older browsers.
+      ]]>, makes it possible to modify aspects of the logging behaviour, see
+        Log options below.
     
 
     
Directories passed to Common Test may have either relative or absolute paths.

@@ -645,6 +656,30 @@
 	to follow test progress simply by refreshing pages in the HTML browser.
 	Statistics totals are not presented until a test is complete however.

 
+	

+	  
+	  Log options
+	  
With the logopts start flag, it's possible to specify
+	  options that modify some aspects of the logging behaviour.
+	  Currently, the following options are available:

+	  
+	    no_src
+	    no_nl
+	  
+	  
With no_src, the html version of the test suite source
+	  code will not be generated during the test run (and consequently
+	  not be available in the log file system).

+	  
With no_nl, Common Test will not add a newline character
+	  (\n) to the end of an output string that it receives from a call to e.g.
+	  io:format/2, and which it prints to the test case log.

+	  
For example, if a test is started with:

+	  
$ ct_run -suite my_SUITE -logopts no_src

+	  
then printouts during the test made by successive calls to io:format("x"),
+	  will appear in the test case log as:

+	  
xxx

+	  
instead of each x printed on a new line, which is the default behaviour.

+	

+
       
   
       

-- 
cgit v1.2.3