diff options
Diffstat (limited to 'lib/common_test')
66 files changed, 3599 insertions, 1019 deletions
diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index 0750f560b3..198290c1be 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -126,6 +126,7 @@ <title>Run tests using test specification</title> <pre> ct_run -spec TestSpec1 TestSpec2 .. TestSpecN + [-join_specs] [-config ConfigFile1 ConfigFile2 .. ConfigFileN] [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 ConfigString2 and .. and CallbackModuleN ConfigStringN] diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 0345fab8e8..030de2efa9 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2012</year> + <year>2004</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index d5f5d89e05..35f89153d3 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -530,374 +530,469 @@ <marker id="test_specifications"></marker> <section> <title>Test Specifications</title> - - <p>The most flexible way to specify what to test, is to use a so - called test specification. A test specification is a sequence of - Erlang terms. The terms are normally declared in a text file (see - <c><seealso marker="ct#run_test-1">ct:run_test/1</seealso></c>), but - may also be passed to Common Test on the form of a list (see - <c><seealso marker="ct#run_testspec-1">ct:run_testspec/1</seealso></c>). - There are two general types of terms: configuration terms and test - specification terms.</p> - <p>With configuration terms it is possible to e.g. label the test - run (similar to <c>ct_run -label</c>), evaluate arbitrary expressions - before starting the test, import configuration data (similar to - <c>ct_run -config/-userconfig</c>), specify the top level HTML log - directory (similar to <c>ct_run -logdir</c>), enable code coverage - analysis (similar to <c>ct_run -cover</c>), install Common Test Hooks - (similar to <c>ct_run -ch_hooks</c>), install event_handler plugins - (similar to <c>ct_run -event_handler</c>), specify include directories - that should be passed to the compiler for automatic compilation - (similar to <c>ct_run -include</c>), disable the auto compilation - feature (similar to <c>ct_run -no_auto_compile</c>), set verbosity - levels (similar to <c>ct_run -verbosity</c>), and more.</p> - <p>Configuration terms can be combined with <c>ct_run</c> start flags, - or <c>ct:run_test/1</c> options. The result will for some flags/options - and terms be that the values are merged (e.g. configuration files, - include directories, verbosity levels, silent connections), and for - others that the start flags/options override the test specification - terms (e.g. log directory, label, style sheet, auto compilation).</p> - <p>With test specification terms it is possible to state exactly - which tests should run and in which order. A test term specifies - either one or more suites, one or more test case groups (possibly nested), - or one or more test cases in a group (or in multiple groups) or in a suite.</p> - <p>An arbitrary number of test terms may be declared in sequence. - Common Test will by default compile the terms into one or more tests - to be performed in one resulting test run. Note that a term that - specifies a set of test cases will "swallow" one that only - specifies a subset of these cases. E.g. the result of merging - one term that specifies that all cases in suite S should be - executed, with another term specifying only test case X and Y in - S, is a test of all cases in S. However, if a term specifying - test case X and Y in S is merged with a term specifying case Z - in S, the result is a test of X, Y and Z in S. To disable this - behaviour, i.e. to instead perform each test sequentially in a "script-like" - manner, the term <c>merge_tests</c> can be set to <c>false</c> in - the test specification.</p> - <p>A test term can also specify one or more test suites, groups, - or test cases to be skipped. Skipped suites, groups and cases - are not executed and show up in the HTML log files as - SKIPPED.</p> - <p>When a test case group is specified, the resulting test - executes the <c>init_per_group</c> function, followed by all test - cases and sub groups (including their configuration functions), and - finally the <c>end_per_group</c> function. Also if particular - test cases in a group are specified, <c>init_per_group</c> - and <c>end_per_group</c> for the group in question are - called. If a group which is defined (in <c>Suite:group/0</c>) to - be a sub group of another group, is specified (or if particular test - cases of a sub group are), Common Test will call the configuration - functions for the top level groups as well as for the sub group - in question (making it possible to pass configuration data all - the way from <c>init_per_suite</c> down to the test cases in the - sub group).</p> - <p>The test specification utilizes the same mechanism for specifying - test case groups by means of names and paths, as explained in the - <seealso marker="run_test_chapter#group_execution">Group Execution</seealso> - section above, with the addition of the <c>GroupSpec</c> element - described next.</p> - <p>The <c>GroupSpec</c> element makes it possible to specify - group execution properties that will override those in the - group definition (i.e. in <c>groups/0</c>). Execution properties for - sub-groups may be overridden as well. This feature makes it possible to - change properties of groups at the time of execution, - without even having to edit the test suite. The very same - feature is available for <c>group</c> elements in the <c>Suite:all/0</c> - list. Therefore, more detailed documentation, and examples, can be - found in the <seealso marker="write_test_chapter#test_case_groups"> - Test case groups</seealso> chapter.</p> - - <p>Below is the test specification syntax. Test specifications can - be used to run tests both in a single test host environment and - in a distributed Common Test environment (Large Scale - Testing). The node parameters in the <c>init</c> term are only - relevant in the latter (see the - <seealso marker="ct_master_chapter#test_specifications">Large - Scale Testing</seealso> chapter for information). For more information - about the various terms, please see the corresponding sections in the - User's Guide, such as e.g. the - <seealso marker="run_test_chapter#ct_run"><c>ct_run</c> - program</seealso> for an overview of available start flags - (since most flags have a corresponding configuration term), and - more detailed explanation of e.g. - <seealso marker="write_test_chapter#logging">Logging</seealso> - (for the <c>verbosity</c>, <c>stylesheet</c> and <c>basic_html</c> terms), - <seealso marker="config_file_chapter#top">External Configuration Data</seealso> - (for the <c>config</c> and <c>userconfig</c> terms), - <seealso marker="event_handler_chapter#event_handling">Event - Handling</seealso> (for the <c>event_handler</c> term), - <seealso marker="ct_hooks_chapter#installing">Common Test Hooks</seealso> - (for the <c>ct_hooks</c> term), etc.</p> - <p>Config terms:</p> - <pre> - {merge_tests, Bool}. - - {define, Constant, Value}. - - {node, NodeAlias, Node}. - - {init, InitOptions}. - {init, [NodeAlias], InitOptions}. - - {label, Label}. - {label, NodeRefs, Label}. - - {verbosity, VerbosityLevels}. - {verbosity, NodeRefs, VerbosityLevels}. - - {stylesheet, CSSFile}. - {stylesheet, NodeRefs, CSSFile}. - - {silent_connections, ConnTypes}. - {silent_connections, NodeRefs, ConnTypes}. - - {multiply_timetraps, N}. - {multiply_timetraps, NodeRefs, N}. - - {scale_timetraps, Bool}. - {scale_timetraps, NodeRefs, Bool}. - - {cover, CoverSpecFile}. - {cover, NodeRefs, CoverSpecFile}. - - {cover_stop, Bool}. - {cover_stop, NodeRefs, Bool}. - - {include, IncludeDirs}. - {include, NodeRefs, IncludeDirs}. - - {auto_compile, Bool}, - {auto_compile, NodeRefs, Bool}, - - {config, ConfigFiles}. - {config, ConfigDir, ConfigBaseNames}. - {config, NodeRefs, ConfigFiles}. - {config, NodeRefs, ConfigDir, ConfigBaseNames}. - - {userconfig, {CallbackModule, ConfigStrings}}. - {userconfig, NodeRefs, {CallbackModule, ConfigStrings}}. - - {logdir, LogDir}. - {logdir, NodeRefs, LogDir}. - - {logopts, LogOpts}. - {logopts, NodeRefs, LogOpts}. - - {create_priv_dir, PrivDirOption}. - {create_priv_dir, NodeRefs, PrivDirOption}. - - {event_handler, EventHandlers}. - {event_handler, NodeRefs, EventHandlers}. - {event_handler, EventHandlers, InitArgs}. - {event_handler, NodeRefs, EventHandlers, InitArgs}. - - {ct_hooks, CTHModules}. - {ct_hooks, NodeRefs, CTHModules}. - - {enable_builtin_hooks, Bool}. - - {basic_html, Bool}. - {basic_html, NodeRefs, Bool}. + <section> + <title>General description</title> + <p>The most flexible way to specify what to test, is to use a so + called test specification. A test specification is a sequence of + Erlang terms. The terms are normally declared in one or more text files + (see <c><seealso marker="ct#run_test-1">ct:run_test/1</seealso></c>), but + may also be passed to Common Test on the form of a list (see + <c><seealso marker="ct#run_testspec-1">ct:run_testspec/1</seealso></c>). + There are two general types of terms: configuration terms and test + specification terms.</p> + <p>With configuration terms it is possible to e.g. label the test + run (similar to <c>ct_run -label</c>), evaluate arbitrary expressions + before starting the test, import configuration data (similar to + <c>ct_run -config/-userconfig</c>), specify the top level HTML log + directory (similar to <c>ct_run -logdir</c>), enable code coverage + analysis (similar to <c>ct_run -cover</c>), install Common Test Hooks + (similar to <c>ct_run -ch_hooks</c>), install event_handler plugins + (similar to <c>ct_run -event_handler</c>), specify include directories + that should be passed to the compiler for automatic compilation + (similar to <c>ct_run -include</c>), disable the auto compilation + feature (similar to <c>ct_run -no_auto_compile</c>), set verbosity + levels (similar to <c>ct_run -verbosity</c>), and more.</p> + <p>Configuration terms can be combined with <c>ct_run</c> start flags, + or <c>ct:run_test/1</c> options. The result will for some flags/options + and terms be that the values are merged (e.g. configuration files, + include directories, verbosity levels, silent connections), and for + others that the start flags/options override the test specification + terms (e.g. log directory, label, style sheet, auto compilation).</p> + <p>With test specification terms it is possible to state exactly + which tests should run and in which order. A test term specifies + either one or more suites, one or more test case groups (possibly nested), + or one or more test cases in a group (or in multiple groups) or in a suite.</p> + <p>An arbitrary number of test terms may be declared in sequence. + Common Test will by default compile the terms into one or more tests + to be performed in one resulting test run. Note that a term that + specifies a set of test cases will "swallow" one that only + specifies a subset of these cases. E.g. the result of merging + one term that specifies that all cases in suite S should be + executed, with another term specifying only test case X and Y in + S, is a test of all cases in S. However, if a term specifying + test case X and Y in S is merged with a term specifying case Z + in S, the result is a test of X, Y and Z in S. To disable this + behaviour, i.e. to instead perform each test sequentially in a "script-like" + manner, the term <c>merge_tests</c> can be set to <c>false</c> in + the test specification.</p> + <p>A test term can also specify one or more test suites, groups, + or test cases to be skipped. Skipped suites, groups and cases + are not executed and show up in the HTML log files as + SKIPPED.</p> + </section> + <section> + <title>Using multiple test specification files</title> + + <p>When multiple test specification files are given at startup (either + with <c>ct_run -spec file1 file2 ...</c> or + <c>ct:run_test([{spec, [File1,File2,...]}])</c>), + Common Test will either execute one test run per specification file, or + join the files and perform all tests within one single test run. The first + behaviour is the default one. The latter requires that the start + flag/option <c>join_suites</c> is provided, e.g. + <c>run_test -spec ./my_tests1.ts ./my_tests2.ts -join_suites</c>.</p> + + <p>Joining a number of specifications, or running them separately, can + also be accomplished with (and may be combined with) test specification + file inclusion, described next.</p> + </section> + <section> + <title>Test specification file inclusion</title> + <p>With the <c>specs</c> term (see syntax below), it's possible to have + a test specification include other specifications. An included + specification may either be joined with the source specification, + or used to produce a separate test run (like with the <c>join_specs</c> + start flag/option above). Example:</p> + <pre> + %% In specification file "a.spec" + {specs, join, ["b.spec", "c.spec"]}. + {specs, separate, ["d.spec", "e.spec"]}. + %% Config and test terms follow + ...</pre> + <p>In this example, the test terms defined in files "b.spec" and "c.spec" + will be joined with the terms in the source specification "a.spec" + (if any). The inclusion of specifications "d.spec" and + "e.spec" will result in two separate, and independent, test runs (i.e. + one for each included specification).</p> + <p>Note that the <c>join</c> option does not imply that the test terms + will be merged (see <c>merge_tests</c> above), only that all tests are + executed in one single test run.</p> + <p>Joined specifications share common configuration settings, such as + the list of <c>config</c> files or <c>include</c> directories. + For configuration that can not be combined, such as settings for <c>logdir</c> + or <c>verbosity</c>, it is up to the user to ensure there are no clashes + when the test specifications are joined. Specifications included with + the <c>separate</c> option, do not share configuration settings with the + source specification. This is useful e.g. if there are clashing + configuration settings in included specifications, making it impossible + to join them.</p> + <p>If <c>{merge_tests,true}</c> is set in the source specification + (which is the default setting), terms in joined specifications will be + merged with terms in the source specification (according to the + description of <c>merge_tests</c> above).</p> + <p>Note that it is always the <c>merge_tests</c> setting in the source + specification that is used when joined with other specifications. + Say e.g. that a source specification A, with tests TA1 and TA2, has + <c>{merge_tests,false}</c> set, and it includes another specification, + B, with tests TB1 and TB2, that has <c>{merge_tests,true}</c> set. + The result will be that the test series: <c>TA1,TA2,merge(TB1,TB2)</c>, + is executed. The opposite <c>merge_tests</c> settings would result in the + following the test series: <c>merge(merge(TA1,TA2),TB1,TB2)</c>.</p> + <p>The <c>specs</c> term may of course be used to nest specifications, + i.e. have one specification include other specifications, which in turn + include others, etc.</p> + </section> + <section> + <title>Test case groups</title> + + <p>When a test case group is specified, the resulting test + executes the <c>init_per_group</c> function, followed by all test + cases and sub groups (including their configuration functions), and + finally the <c>end_per_group</c> function. Also if particular + test cases in a group are specified, <c>init_per_group</c> + and <c>end_per_group</c> for the group in question are + called. If a group which is defined (in <c>Suite:group/0</c>) to + be a sub group of another group, is specified (or if particular test + cases of a sub group are), Common Test will call the configuration + functions for the top level groups as well as for the sub group + in question (making it possible to pass configuration data all + the way from <c>init_per_suite</c> down to the test cases in the + sub group).</p> + <p>The test specification utilizes the same mechanism for specifying + test case groups by means of names and paths, as explained in the + <seealso marker="run_test_chapter#group_execution">Group Execution</seealso> + section above, with the addition of the <c>GroupSpec</c> element + described next.</p> + <p>The <c>GroupSpec</c> element makes it possible to specify + group execution properties that will override those in the + group definition (i.e. in <c>groups/0</c>). Execution properties for + sub-groups may be overridden as well. This feature makes it possible to + change properties of groups at the time of execution, + without even having to edit the test suite. The very same + feature is available for <c>group</c> elements in the <c>Suite:all/0</c> + list. Therefore, more detailed documentation, and examples, can be + found in the <seealso marker="write_test_chapter#test_case_groups"> + Test case groups</seealso> chapter.</p> + </section> - {release_shell, Bool}.</pre> + <section> + <title>Test specification syntax</title> + + <p>Below is the test specification syntax. Test specifications can + be used to run tests both in a single test host environment and + in a distributed Common Test environment (Large Scale + Testing). The node parameters in the <c>init</c> term are only + relevant in the latter (see the + <seealso marker="ct_master_chapter#test_specifications">Large + Scale Testing</seealso> chapter for information). For more information + about the various terms, please see the corresponding sections in the + User's Guide, such as e.g. the + <seealso marker="run_test_chapter#ct_run"><c>ct_run</c> + program</seealso> for an overview of available start flags + (since most flags have a corresponding configuration term), and + more detailed explanation of e.g. + <seealso marker="write_test_chapter#logging">Logging</seealso> + (for the <c>verbosity</c>, <c>stylesheet</c> and <c>basic_html</c> terms), + <seealso marker="config_file_chapter#top">External Configuration Data</seealso> + (for the <c>config</c> and <c>userconfig</c> terms), + <seealso marker="event_handler_chapter#event_handling">Event + Handling</seealso> (for the <c>event_handler</c> term), + <seealso marker="ct_hooks_chapter#installing">Common Test Hooks</seealso> + (for the <c>ct_hooks</c> term), etc.</p> + </section> + <p>Config terms:</p> + <pre> + {merge_tests, Bool}. + + {define, Constant, Value}. + + {specs, InclSpecsOption, TestSpecs}. + + {node, NodeAlias, Node}. + + {init, InitOptions}. + {init, [NodeAlias], InitOptions}. + + {label, Label}. + {label, NodeRefs, Label}. + + {verbosity, VerbosityLevels}. + {verbosity, NodeRefs, VerbosityLevels}. + + {stylesheet, CSSFile}. + {stylesheet, NodeRefs, CSSFile}. + + {silent_connections, ConnTypes}. + {silent_connections, NodeRefs, ConnTypes}. + + {multiply_timetraps, N}. + {multiply_timetraps, NodeRefs, N}. + + {scale_timetraps, Bool}. + {scale_timetraps, NodeRefs, Bool}. + + {cover, CoverSpecFile}. + {cover, NodeRefs, CoverSpecFile}. + + {cover_stop, Bool}. + {cover_stop, NodeRefs, Bool}. + + {include, IncludeDirs}. + {include, NodeRefs, IncludeDirs}. + + {auto_compile, Bool}, + {auto_compile, NodeRefs, Bool}, + + {config, ConfigFiles}. + {config, ConfigDir, ConfigBaseNames}. + {config, NodeRefs, ConfigFiles}. + {config, NodeRefs, ConfigDir, ConfigBaseNames}. + + {userconfig, {CallbackModule, ConfigStrings}}. + {userconfig, NodeRefs, {CallbackModule, ConfigStrings}}. + + {logdir, LogDir}. + {logdir, NodeRefs, LogDir}. + + {logopts, LogOpts}. + {logopts, NodeRefs, LogOpts}. + + {create_priv_dir, PrivDirOption}. + {create_priv_dir, NodeRefs, PrivDirOption}. + + {event_handler, EventHandlers}. + {event_handler, NodeRefs, EventHandlers}. + {event_handler, EventHandlers, InitArgs}. + {event_handler, NodeRefs, EventHandlers, InitArgs}. + + {ct_hooks, CTHModules}. + {ct_hooks, NodeRefs, CTHModules}. + + {enable_builtin_hooks, Bool}. + + {basic_html, Bool}. + {basic_html, NodeRefs, Bool}. + + {release_shell, Bool}.</pre> + <p>Test terms:</p> - <pre> - {suites, Dir, Suites}. - {suites, NodeRefs, Dir, Suites}. - - {groups, Dir, Suite, Groups}. - {groups, NodeRefs, Dir, Suite, Groups}. - - {groups, Dir, Suite, Groups, {cases,Cases}}. - {groups, NodeRefs, Dir, Suite, Groups, {cases,Cases}}. - - {cases, Dir, Suite, Cases}. - {cases, NodeRefs, Dir, Suite, Cases}. - - {skip_suites, Dir, Suites, Comment}. - {skip_suites, NodeRefs, Dir, Suites, Comment}. - - {skip_groups, Dir, Suite, GroupNames, Comment}. - {skip_groups, NodeRefs, Dir, Suite, GroupNames, Comment}. - - {skip_cases, Dir, Suite, Cases, Comment}. - {skip_cases, NodeRefs, Dir, Suite, Cases, Comment}.</pre> - + <pre> + {suites, Dir, Suites}. + {suites, NodeRefs, Dir, Suites}. + + {groups, Dir, Suite, Groups}. + {groups, NodeRefs, Dir, Suite, Groups}. + + {groups, Dir, Suite, Groups, {cases,Cases}}. + {groups, NodeRefs, Dir, Suite, Groups, {cases,Cases}}. + + {cases, Dir, Suite, Cases}. + {cases, NodeRefs, Dir, Suite, Cases}. + + {skip_suites, Dir, Suites, Comment}. + {skip_suites, NodeRefs, Dir, Suites, Comment}. + + {skip_groups, Dir, Suite, GroupNames, Comment}. + {skip_groups, NodeRefs, Dir, Suite, GroupNames, Comment}. + + {skip_cases, Dir, Suite, Cases, Comment}. + {skip_cases, NodeRefs, Dir, Suite, Cases, Comment}.</pre> + <p>Types:</p> - <pre> - Bool = true | false - Constant = atom() - Value = term() - NodeAlias = atom() - Node = node() - NodeRef = NodeAlias | Node | master - NodeRefs = all_nodes | [NodeRef] | NodeRef - InitOptions = term() - Label = atom() | string() - VerbosityLevels = integer() | [{Category,integer()}] - Category = atom() - CSSFile = string() - ConnTypes = all | [atom()] - N = integer() - CoverSpecFile = string() - IncludeDirs = string() | [string()] - ConfigFiles = string() | [string()] - ConfigDir = string() - ConfigBaseNames = string() | [string()] - CallbackModule = atom() - ConfigStrings = string() | [string()] - LogDir = string() - LogOpts = [term()] - PrivDirOption = auto_per_run | auto_per_tc | manual_per_tc - EventHandlers = atom() | [atom()] - InitArgs = [term()] - CTHModules = [CTHModule | - {CTHModule, CTHInitArgs} | - {CTHModule, CTHInitArgs, CTHPriority}] - CTHModule = atom() - CTHInitArgs = term() - Dir = string() - Suites = atom() | [atom()] | all - Suite = atom() - Groups = GroupPath | [GroupPath] | GroupSpec | [GroupSpec] | all - GroupPath = [GroupName] - GroupSpec = GroupName | {GroupName,Properties} | {GroupName,Properties,GroupSpec} - GroupName = atom() - GroupNames = GroupName | [GroupName] - Cases = atom() | [atom()] | all - Comment = string() | ""</pre> - - <p>The difference between the <c>config</c> terms above, is that with - <c>ConfigDir</c>, <c>ConfigBaseNames</c> is a list of base names, - i.e. without directory paths. <c>ConfigFiles</c> must be full names, - including paths. E.g, these two terms have the same meaning:</p> - <pre> - {config, ["/home/testuser/tests/config/nodeA.cfg", - "/home/testuser/tests/config/nodeB.cfg"]}. - - {config, "/home/testuser/tests/config", ["nodeA.cfg","nodeB.cfg"]}.</pre> - - <note><p>Any relative paths specified in the test specification, will be - relative to the directory which contains the test specification file, if - <c>ct_run -spec TestSpecFile ...</c> or - <c>ct:run:test([{spec,TestSpecFile},...])</c> - executes the test. The path will be relative to the top level log directory, if - <c>ct:run:testspec(TestSpec)</c> executes the test.</p></note> - - <p>The <c>define</c> term introduces a constant, which is used to - replace the name <c>Constant</c> with <c>Value</c>, wherever it's found in - the test specification. This replacement happens during an initial iteration - through the test specification. Constants may be used anywhere in the test - specification, e.g. in arbitrary lists and tuples, and even in strings - and inside the value part of other constant definitions! A constant can - also be part of a node name, but that is the only place where a constant - can be part of an atom.</p> - - <note><p>For the sake of readability, the name of the constant must always - begin with an upper case letter, or a <c>$</c>, <c>?</c>, or <c>_</c>. - This also means that it must always be single quoted (obviously, since - the constant name is actually an atom, not text).</p></note> - - <p>The main benefit of constants is that they can be used to reduce the size - (and avoid repetition) of long strings, such as file paths. Compare these - terms:</p> - - <pre> - %% 1a. no constant - {config, "/home/testuser/tests/config", ["nodeA.cfg","nodeB.cfg"]}. - {suites, "/home/testuser/tests/suites", all}. - - %% 1b. with constant - {define, 'TESTDIR', "/home/testuser/tests"}. - {config, "'TESTDIR'/config", ["nodeA.cfg","nodeB.cfg"]}. - {suites, "'TESTDIR'/suites", all}. - - %% 2a. no constants - {config, [testnode@host1, testnode@host2], "../config", ["nodeA.cfg","nodeB.cfg"]}. - {suites, [testnode@host1, testnode@host2], "../suites", [x_SUITE, y_SUITE]}. - - %% 2b. with constants - {define, 'NODE', testnode}. - {define, 'NODES', ['NODE'@host1, 'NODE'@host2]}. - {config, 'NODES', "../config", ["nodeA.cfg","nodeB.cfg"]}. - {suites, 'NODES', "../suites", [x_SUITE, y_SUITE]}.</pre> - - <p>Constants make the test specification term <c>alias</c>, in previous - versions of Common Test, redundant. This term has been deprecated but will - remain supported in upcoming Common Test releases. Replacing <c>alias</c> - terms with <c>define</c> is strongly recommended though! Here's an example - of such a replacement:</p> + <pre> + Bool = true | false + Constant = atom() + Value = term() + InclSpecsOption = join | separate + TestSpecs = string() | [string()] + NodeAlias = atom() + Node = node() + NodeRef = NodeAlias | Node | master + NodeRefs = all_nodes | [NodeRef] | NodeRef + InitOptions = term() + Label = atom() | string() + VerbosityLevels = integer() | [{Category,integer()}] + Category = atom() + CSSFile = string() + ConnTypes = all | [atom()] + N = integer() + CoverSpecFile = string() + IncludeDirs = string() | [string()] + ConfigFiles = string() | [string()] + ConfigDir = string() + ConfigBaseNames = string() | [string()] + CallbackModule = atom() + ConfigStrings = string() | [string()] + LogDir = string() + LogOpts = [term()] + PrivDirOption = auto_per_run | auto_per_tc | manual_per_tc + EventHandlers = atom() | [atom()] + InitArgs = [term()] + CTHModules = [CTHModule | + {CTHModule, CTHInitArgs} | + {CTHModule, CTHInitArgs, CTHPriority}] + CTHModule = atom() + CTHInitArgs = term() + Dir = string() + Suites = atom() | [atom()] | all + Suite = atom() + Groups = GroupPath | [GroupPath] | GroupSpec | [GroupSpec] | all + GroupPath = [GroupName] + GroupSpec = GroupName | {GroupName,Properties} | {GroupName,Properties,GroupSpec} + GroupName = atom() + GroupNames = GroupName | [GroupName] + Cases = atom() | [atom()] | all + Comment = string() | ""</pre> + + <section> + <p>The difference between the <c>config</c> terms above, is that with + <c>ConfigDir</c>, <c>ConfigBaseNames</c> is a list of base names, + i.e. without directory paths. <c>ConfigFiles</c> must be full names, + including paths. E.g, these two terms have the same meaning:</p> + <pre> + {config, ["/home/testuser/tests/config/nodeA.cfg", + "/home/testuser/tests/config/nodeB.cfg"]}. + + {config, "/home/testuser/tests/config", ["nodeA.cfg","nodeB.cfg"]}.</pre> + + <note><p>Any relative paths specified in the test specification, will be + relative to the directory which contains the test specification file, if + <c>ct_run -spec TestSpecFile ...</c> or + <c>ct:run:test([{spec,TestSpecFile},...])</c> + executes the test. The path will be relative to the top level log directory, if + <c>ct:run:testspec(TestSpec)</c> executes the test.</p></note> + </section> - <pre> - %% using the old alias term - {config, "/home/testuser/tests/config/nodeA.cfg"}. - {alias, suite_dir, "/home/testuser/tests/suites"}. - {groups, suite_dir, x_SUITE, group1}. - - %% replacing with constants - {define, 'TestDir', "/home/testuser/tests"}. - {define, 'CfgDir', "'TestDir'/config"}. - {define, 'SuiteDir', "'TestDir'/suites"}. - {config, 'CfgDir', "nodeA.cfg"}. - {groups, 'SuiteDir', x_SUITE, group1}.</pre> - - <p>Actually, constants could well replace the <c>node</c> term too, but - this still has declarative value, mainly when used in combination - with <c>NodeRefs == all_nodes</c> (see types above).</p> - - <p>Here follows a simple test specification example:</p> - <pre> - {define, 'Top', "/home/test"}. - {define, 'T1', "'Top'/t1"}. - {define, 'T2', "'Top'/t2"}. - {define, 'T3', "'Top'/t3"}. - {define, 'CfgFile', "config.cfg"}. + <section> + <title>Constants</title> + + <p>The <c>define</c> term introduces a constant, which is used to + replace the name <c>Constant</c> with <c>Value</c>, wherever it's found in + the test specification. This replacement happens during an initial iteration + through the test specification. Constants may be used anywhere in the test + specification, e.g. in arbitrary lists and tuples, and even in strings + and inside the value part of other constant definitions! A constant can + also be part of a node name, but that is the only place where a constant + can be part of an atom.</p> + + <note><p>For the sake of readability, the name of the constant must always + begin with an upper case letter, or a <c>$</c>, <c>?</c>, or <c>_</c>. + This also means that it must always be single quoted (obviously, since + the constant name is actually an atom, not text).</p></note> + + <p>The main benefit of constants is that they can be used to reduce the size + (and avoid repetition) of long strings, such as file paths. Compare these + terms:</p> + + <pre> + %% 1a. no constant + {config, "/home/testuser/tests/config", ["nodeA.cfg","nodeB.cfg"]}. + {suites, "/home/testuser/tests/suites", all}. + + %% 1b. with constant + {define, 'TESTDIR', "/home/testuser/tests"}. + {config, "'TESTDIR'/config", ["nodeA.cfg","nodeB.cfg"]}. + {suites, "'TESTDIR'/suites", all}. + + %% 2a. no constants + {config, [testnode@host1, testnode@host2], "../config", ["nodeA.cfg","nodeB.cfg"]}. + {suites, [testnode@host1, testnode@host2], "../suites", [x_SUITE, y_SUITE]}. + + %% 2b. with constants + {define, 'NODE', testnode}. + {define, 'NODES', ['NODE'@host1, 'NODE'@host2]}. + {config, 'NODES', "../config", ["nodeA.cfg","nodeB.cfg"]}. + {suites, 'NODES', "../suites", [x_SUITE, y_SUITE]}.</pre> + + <p>Constants make the test specification term <c>alias</c>, in previous + versions of Common Test, redundant. This term has been deprecated but will + remain supported in upcoming Common Test releases. Replacing <c>alias</c> + terms with <c>define</c> is strongly recommended though! Here's an example + of such a replacement:</p> + + <pre> + %% using the old alias term + {config, "/home/testuser/tests/config/nodeA.cfg"}. + {alias, suite_dir, "/home/testuser/tests/suites"}. + {groups, suite_dir, x_SUITE, group1}. + + %% replacing with constants + {define, 'TestDir', "/home/testuser/tests"}. + {define, 'CfgDir', "'TestDir'/config"}. + {define, 'SuiteDir', "'TestDir'/suites"}. + {config, 'CfgDir', "nodeA.cfg"}. + {groups, 'SuiteDir', x_SUITE, group1}.</pre> + + <p>Actually, constants could well replace the <c>node</c> term too, but + this still has declarative value, mainly when used in combination + with <c>NodeRefs == all_nodes</c> (see types above).</p> + </section> - {logdir, "'Top'/logs"}. - - {config, ["'T1'/'CfgFile'", "'T2'/'CfgFile'", "'T3'/'CfgFile'"]}. - - {suites, 'T1', all}. - {skip_suites, 'T1', [t1B_SUITE,t1D_SUITE], "Not implemented"}. - {skip_cases, 'T1', t1A_SUITE, [test3,test4], "Irrelevant"}. - {skip_cases, 'T1', t1C_SUITE, [test1], "Ignore"}. - - {suites, 'T2', [t2B_SUITE,t2C_SUITE]}. - {cases, 'T2', t2A_SUITE, [test4,test1,test7]}. - - {skip_suites, 'T3', all, "Not implemented"}.</pre> + <section> + <title>Example</title> + + <p>Here follows a simple test specification example:</p> + <pre> + {define, 'Top', "/home/test"}. + {define, 'T1', "'Top'/t1"}. + {define, 'T2', "'Top'/t2"}. + {define, 'T3', "'Top'/t3"}. + {define, 'CfgFile', "config.cfg"}. + + {logdir, "'Top'/logs"}. + + {config, ["'T1'/'CfgFile'", "'T2'/'CfgFile'", "'T3'/'CfgFile'"]}. + + {suites, 'T1', all}. + {skip_suites, 'T1', [t1B_SUITE,t1D_SUITE], "Not implemented"}. + {skip_cases, 'T1', t1A_SUITE, [test3,test4], "Irrelevant"}. + {skip_cases, 'T1', t1C_SUITE, [test1], "Ignore"}. + + {suites, 'T2', [t2B_SUITE,t2C_SUITE]}. + {cases, 'T2', t2A_SUITE, [test4,test1,test7]}. + + {skip_suites, 'T3', all, "Not implemented"}.</pre> + + <p>The example specifies the following:</p> + <list> + <item>The specified logdir directory will be used for storing + the HTML log files (in subdirectories tagged with node name, + date and time).</item> + <item>The variables in the specified test system config files will be + imported for the test.</item> + <item>The first test to run includes all suites for system t1. Excluded from + the test are however the t1B and t1D suites. Also test cases test3 and + test4 in t1A as well as the test1 case in t1C are excluded from + the test.</item> + <item>Secondly, the test for system t2 should run. The included suites are + t2B and t2C. Included are also test cases test4, test1 and test7 in suite + t2A. Note that the test cases will be executed in the specified order.</item> + <item>Lastly, all suites for systems t3 are to be completely skipped and this + should be explicitly noted in the log files.</item> + </list> + </section> - <p>The example specifies the following:</p> - <list> - <item>The specified logdir directory will be used for storing - the HTML log files (in subdirectories tagged with node name, - date and time).</item> - <item>The variables in the specified test system config files will be - imported for the test.</item> - <item>The first test to run includes all suites for system t1. Excluded from - the test are however the t1B and t1D suites. Also test cases test3 and - test4 in t1A as well as the test1 case in t1C are excluded from - the test.</item> - <item>Secondly, the test for system t2 should run. The included suites are - t2B and t2C. Included are also test cases test4, test1 and test7 in suite - t2A. Note that the test cases will be executed in the specified order.</item> - <item>Lastly, all suites for systems t3 are to be completely skipped and this - should be explicitly noted in the log files.</item> - </list> - <p>With the <c>init</c> term it's possible to specify initialization options - for nodes defined in the test specification. Currently, there are options - to start the node and/or to evaluate any function on the node. - See the <seealso marker="ct_master_chapter#ct_slave">Automatic startup of - the test target nodes</seealso> chapter for details.</p> - <p>It is possible for the user to provide a test specification that - includes (for Common Test) unrecognizable terms. If this is desired, - the <c>-allow_user_terms</c> flag should be used when starting tests with - <c>ct_run</c>. This forces Common Test to ignore unrecognizable terms. - Note that in this mode, Common Test is not able to check the specification - for errors as efficiently as if the scanner runs in default mode. - If <c><seealso marker="ct#run_test-1">ct:run_test/1</seealso></c> is used for starting the tests, the relaxed scanner - mode is enabled by means of the tuple: <c>{allow_user_terms,true}</c></p> + <section> + <title>The init term</title> + <p>With the <c>init</c> term it's possible to specify initialization options + for nodes defined in the test specification. Currently, there are options + to start the node and/or to evaluate any function on the node. + See the <seealso marker="ct_master_chapter#ct_slave">Automatic startup of + the test target nodes</seealso> chapter for details.</p> + </section> + <section> + <title>User specific terms</title> + <p>It is possible for the user to provide a test specification that + includes (for Common Test) unrecognizable terms. If this is desired, + the <c>-allow_user_terms</c> flag should be used when starting tests with + <c>ct_run</c>. This forces Common Test to ignore unrecognizable terms. + Note that in this mode, Common Test is not able to check the specification + for errors as efficiently as if the scanner runs in default mode. + If <c><seealso marker="ct#run_test-1">ct:run_test/1</seealso></c> is used + for starting the tests, the relaxed scanner + mode is enabled by means of the tuple: <c>{allow_user_terms,true}</c></p> + </section> </section> <section> diff --git a/lib/common_test/src/Makefile b/lib/common_test/src/Makefile index 31906b4568..4600c0ad78 100644 --- a/lib/common_test/src/Makefile +++ b/lib/common_test/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2003-2012. All Rights Reserved. +# Copyright Ericsson AB 2003-2013. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/common_test/src/ct.erl b/lib/common_test/src/ct.erl index 8eafdff29f..04a95a53fa 100644 --- a/lib/common_test/src/ct.erl +++ b/lib/common_test/src/ct.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2012. All Rights Reserved. +%% Copyright Ericsson AB 2003-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -144,8 +144,8 @@ run(TestDirs) -> %%% @spec run_test(Opts) -> Result %%% Opts = [OptTuples] %%% OptTuples = {dir,TestDirs} | {suite,Suites} | {group,Groups} | -%%% {testcase,Cases} | {spec,TestSpecs} | {label,Label} | -%%% {config,CfgFiles} | {userconfig, UserConfig} | +%%% {testcase,Cases} | {spec,TestSpecs} | {join_specs,Bool} | +%%% {label,Label} | {config,CfgFiles} | {userconfig, UserConfig} | %%% {allow_user_terms,Bool} | {logdir,LogDir} | %%% {silent_connections,Conns} | {stylesheet,CSSFile} | %%% {cover,CoverSpecFile} | {cover_stop,Bool} | {step,StepOpts} | @@ -735,7 +735,7 @@ fail(Format, Args) -> %%% overwrites the string set by this function.</p> comment(Comment) when is_list(Comment) -> Formatted = - case (catch io_lib:format("~s",[Comment])) of + case (catch io_lib:format("~ts",[Comment])) of {'EXIT',_} -> % it's a list not a string io_lib:format("~p",[Comment]); String -> diff --git a/lib/common_test/src/ct_config.erl b/lib/common_test/src/ct_config.erl index b1d709bc75..c35cbd3c08 100644 --- a/lib/common_test/src/ct_config.erl +++ b/lib/common_test/src/ct_config.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2012. All Rights Reserved. +%% Copyright Ericsson AB 2010-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -266,7 +266,10 @@ read_config_files_int([{Callback, File}|Files], FunToSave) -> read_config_files_int([], _FunToSave) -> ok. -store_config(Config, Callback, File) -> +store_config(Config, Callback, File) when is_tuple(Config) -> + store_config([Config], Callback, File); + +store_config(Config, Callback, File) when is_list(Config) -> [ets:insert(?attr_table, #ct_conf{key=Key, value=Val, @@ -607,7 +610,7 @@ encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) -> EncBin = crypto:des3_cbc_encrypt(K1, K2, K3, IVec, Bin2), case file:write_file(EncryptFileName, EncBin) of ok -> - io:format("~s --(encrypt)--> ~s~n", + io:format("~ts --(encrypt)--> ~ts~n", [SrcFileName,EncryptFileName]), ok; {error,Reason} -> @@ -649,7 +652,7 @@ decrypt_config_file(EncryptFileName, TargetFileName, {key,Key}) -> _ -> case file:write_file(TargetFileName, SrcBin) of ok -> - io:format("~s --(decrypt)--> ~s~n", + io:format("~ts --(decrypt)--> ~ts~n", [EncryptFileName,TargetFileName]), ok; {error,Reason} -> @@ -700,7 +703,7 @@ get_crypt_key_from_file() -> _ -> case catch string:tokens(binary_to_list(Result), [$\n,$\r]) of [Key] -> - io:format("~nCrypt key file: ~s~n", [FullName]), + io:format("~nCrypt key file: ~ts~n", [FullName]), Key; _ -> {error,{bad_crypt_file,FullName}} diff --git a/lib/common_test/src/ct_config_plain.erl b/lib/common_test/src/ct_config_plain.erl index 237df5c8f3..c6547f0a40 100644 --- a/lib/common_test/src/ct_config_plain.erl +++ b/lib/common_test/src/ct_config_plain.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -47,7 +47,7 @@ read_config(ConfigFile) -> {error,no_crypt_file} -> {error,{config_file_error, lists:flatten( - io_lib:format("~s",[file:format_error(Reason)]))}}; + io_lib:format("~ts",[file:format_error(Reason)]))}}; {error,CryptError} -> {error,{decrypt_file_error,CryptError}}; _ when is_list(Key) -> diff --git a/lib/common_test/src/ct_conn_log_h.erl b/lib/common_test/src/ct_conn_log_h.erl index d7bd18606b..ac08a3e0ad 100644 --- a/lib/common_test/src/ct_conn_log_h.erl +++ b/lib/common_test/src/ct_conn_log_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -52,7 +52,7 @@ open_files([],State) -> do_open_files([{Tag,File}|Logs],Acc) -> - case file:open(File, [write]) of + case file:open(File, [write,{encoding,utf8}]) of {ok,Fd} -> do_open_files(Logs,[{Tag,Fd}|Acc]); {error,Reason} -> @@ -98,9 +98,9 @@ terminate(_,#state{logs=Logs}) -> %%% Writing reports write_report(Time,#conn_log{module=ConnMod}=Info,Data,State) -> {LogType,Fd} = get_log(Info,State), - io:format(Fd,"~n~s~s~s",[format_head(ConnMod,LogType,Time), - format_title(LogType,Info), - format_data(ConnMod,LogType,Data)]). + io:format(Fd,"~n~ts~ts~ts",[format_head(ConnMod,LogType,Time), + format_title(LogType,Info), + format_data(ConnMod,LogType,Data)]). write_error(Time,#conn_log{module=ConnMod}=Info,Report,State) -> case get_log(Info,State) of @@ -109,9 +109,10 @@ write_error(Time,#conn_log{module=ConnMod}=Info,Report,State) -> %% sasl error handler, so don't write it again. ok; {LogType,Fd} -> - io:format(Fd,"~n~s~s~s",[format_head(ConnMod,LogType,Time," ERROR"), - format_title(LogType,Info), - format_error(LogType,Report)]) + io:format(Fd,"~n~ts~ts~ts", + [format_head(ConnMod,LogType,Time," ERROR"), + format_title(LogType,Info), + format_error(LogType,Report)]) end. get_log(Info,State) -> @@ -140,16 +141,16 @@ format_head(ConnMod,LogType,Time) -> format_head(ConnMod,LogType,Time,""). format_head(ConnMod,raw,Time,Text) -> - io_lib:format("~n~p, ~p~s, ",[now_to_time(Time),ConnMod,Text]); + io_lib:format("~n~w, ~w~ts, ",[now_to_time(Time),ConnMod,Text]); format_head(ConnMod,_,Time,Text) -> Head = pad_char_end(?WIDTH,pretty_head(now_to_time(Time),ConnMod,Text),$=), - io_lib:format("~n~s",[Head]). + io_lib:format("~n~ts",[Head]). format_title(raw,#conn_log{client=Client}=Info) -> - io_lib:format("Client ~p ~s ~s",[Client,actionstr(Info),serverstr(Info)]); + io_lib:format("Client ~w ~s ~ts",[Client,actionstr(Info),serverstr(Info)]); format_title(_,Info) -> Title = pad_char_end(?WIDTH,pretty_title(Info),$=), - io_lib:format("~n~s", [Title]). + io_lib:format("~n~ts", [Title]). format_data(_,_,NoData) when NoData == ""; NoData == <<>> -> ""; @@ -162,8 +163,6 @@ format_error(pretty,Report) -> [io_lib:format("~n ~p: ~p",[K,V]) || {K,V} <- Report]. - - %%%----------------------------------------------------------------- %%% Helpers conn_info(LoggingProc, #conn_log{client=undefined} = ConnInfo) -> @@ -187,12 +186,12 @@ now_to_time({_,_,MicroS}=Now) -> pretty_head({{{Y,Mo,D},{H,Mi,S}},MicroS},ConnMod,Text0) -> Text = string:to_upper(atom_to_list(ConnMod) ++ Text0), - io_lib:format("= ~s ==== ~s-~s-~p::~s:~s:~s,~s ", + io_lib:format("= ~s ==== ~s-~s-~w::~s:~s:~s,~s ", [Text,t(D),month(Mo),Y,t(H),t(Mi),t(S), micro2milli(MicroS)]). pretty_title(#conn_log{client=Client}=Info) -> - io_lib:format("= Client ~p ~s Server ~s ", + io_lib:format("= Client ~w ~s Server ~ts ", [Client,actionstr(Info),serverstr(Info)]). actionstr(#conn_log{action=send}) -> "----->"; @@ -204,7 +203,7 @@ actionstr(_) -> "<---->". serverstr(#conn_log{name=undefined,address=Address}) -> io_lib:format("~p",[Address]); serverstr(#conn_log{name=Alias,address=Address}) -> - io_lib:format("~p(~p)",[Alias,Address]). + io_lib:format("~w(~p)",[Alias,Address]). month(1) -> "Jan"; month(2) -> "Feb"; diff --git a/lib/common_test/src/ct_event.erl b/lib/common_test/src/ct_event.erl index 49e0635d79..c1c1d943b9 100644 --- a/lib/common_test/src/ct_event.erl +++ b/lib/common_test/src/ct_event.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2012. All Rights Reserved. +%% Copyright Ericsson AB 2006-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -150,7 +150,7 @@ init(RecvPids) -> %%-------------------------------------------------------------------- handle_event(Event,State=#state{receivers=RecvPids}) -> print("~n=== ~w ===~n", [?MODULE]), - print("~p: ~p~n", [Event#event.name,Event#event.data]), + print("~w: ~w~n", [Event#event.name,Event#event.data]), lists:foreach(fun(Recv) -> report_event(Recv,Event) end, RecvPids), {ok,State}. diff --git a/lib/common_test/src/ct_framework.erl b/lib/common_test/src/ct_framework.erl index c1abf27e9f..5fe4eaf511 100644 --- a/lib/common_test/src/ct_framework.erl +++ b/lib/common_test/src/ct_framework.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2012. All Rights Reserved. +%% Copyright Ericsson AB 2004-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -750,12 +750,12 @@ error_notification(Mod,Func,_Args,{Error,Loc}) -> Descr1 = lists:flatten(io_lib:format("~P",[Descr,10])), if length(Descr1) > 50 -> Descr2 = string:substr(Descr1,1,50), - io_lib:format("{badmatch,~s...}",[Descr2]); + io_lib:format("{badmatch,~ts...}",[Descr2]); true -> - io_lib:format("{badmatch,~s}",[Descr1]) + io_lib:format("{badmatch,~ts}",[Descr1]) end; {test_case_failed,Reason} -> - case (catch io_lib:format("{test_case_failed,~s}", [Reason])) of + case (catch io_lib:format("{test_case_failed,~ts}", [Reason])) of {'EXIT',_} -> io_lib:format("{test_case_failed,~p}", [Reason]); Result -> Result @@ -788,7 +788,7 @@ error_notification(Mod,Func,_Args,{Error,Loc}) -> "<font color=\"green\">" ++ "(" ++ "</font>" ++ Comment ++ "<font color=\"green\">" ++ ")" ++ "</font>", - Str = io_lib:format("~s ~s", [ErrorHtml,CommentHtml]), + Str = io_lib:format("~ts ~ts", [ErrorHtml,CommentHtml]), test_server:comment(Str) end end, @@ -803,24 +803,24 @@ error_notification(Mod,Func,_Args,{Error,Loc}) -> end, case Loc of [{?MODULE,error_in_suite}] -> - PrintErr("Error in suite detected: ~s", [ErrStr]); + PrintErr("Error in suite detected: ~ts", [ErrStr]); R when R == unknown; R == undefined -> - PrintErr("Error detected: ~s", [ErrStr]); + PrintErr("Error detected: ~ts", [ErrStr]); %% if a function specified by all/0 does not exist, we %% pick up undef here [{LastMod,LastFunc}|_] when ErrStr == "undef" -> - PrintErr("~w:~w could not be executed~nReason: ~s", + PrintErr("~w:~w could not be executed~nReason: ~ts", [LastMod,LastFunc,ErrStr]); [{LastMod,LastFunc}|_] -> - PrintErr("~w:~w failed~nReason: ~s", [LastMod,LastFunc,ErrStr]); + PrintErr("~w:~w failed~nReason: ~ts", [LastMod,LastFunc,ErrStr]); [{LastMod,LastFunc,LastLine}|_] -> %% print error to console, we are only %% interested in the last executed expression - PrintErr("~w:~w failed on line ~w~nReason: ~s", + PrintErr("~w:~w failed on line ~w~nReason: ~ts", [LastMod,LastFunc,LastLine,ErrStr]), case ct_util:read_suite_data({seq,Mod,Func}) of @@ -1184,7 +1184,7 @@ report(What,Data) -> ok; {error,Reason} -> ct_logs:log("COVER INFO", - "Importing cover data from: ~s fails! " + "Importing cover data from: ~ts fails! " "Reason: ~p", [Imp,Reason]) end end, Imps) @@ -1349,4 +1349,7 @@ format_comment(Comment) -> %%%----------------------------------------------------------------- %%% @spec get_html_wrapper(TestName, PrintLabel, Cwd) -> Header get_html_wrapper(TestName, PrintLabel, Cwd, TableCols) -> - ct_logs:get_ts_html_wrapper(TestName, PrintLabel, Cwd, TableCols). + get_html_wrapper(TestName, PrintLabel, Cwd, TableCols, utf8). + +get_html_wrapper(TestName, PrintLabel, Cwd, TableCols, Encoding) -> + ct_logs:get_ts_html_wrapper(TestName, PrintLabel, Cwd, TableCols, Encoding). diff --git a/lib/common_test/src/ct_gen_conn.erl b/lib/common_test/src/ct_gen_conn.erl index 1f01d84601..2d4b1d1f52 100644 --- a/lib/common_test/src/ct_gen_conn.erl +++ b/lib/common_test/src/ct_gen_conn.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2012. All Rights Reserved. +%% Copyright Ericsson AB 2003-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -272,7 +272,7 @@ call(Pid, Msg, Timeout) -> after Timeout -> erlang:demonitor(MRef, [flush]), log("ct_gen_conn", - "Connection process ~p not responding. Killing now!", + "Connection process ~w not responding. Killing now!", [Pid]), exit(Pid, kill), {error,{process_down,Pid,forced_termination}} diff --git a/lib/common_test/src/ct_groups.erl b/lib/common_test/src/ct_groups.erl index 74ab5e5439..14a8aab881 100644 --- a/lib/common_test/src/ct_groups.erl +++ b/lib/common_test/src/ct_groups.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2012. All Rights Reserved. +%% Copyright Ericsson AB 2004-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -462,7 +462,7 @@ make_conf(Mod, Name, Props, TestSpec) -> false -> ct_logs:log("TEST INFO", "init_per_group/2 and " "end_per_group/2 missing for group " - "~p in ~p, using default.", + "~w in ~w, using default.", [Name,Mod]), {{ct_framework,init_per_group}, {ct_framework,end_per_group}, diff --git a/lib/common_test/src/ct_hooks.erl b/lib/common_test/src/ct_hooks.erl index 1bcc63738e..3d87a82e24 100644 --- a/lib/common_test/src/ct_hooks.erl +++ b/lib/common_test/src/ct_hooks.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2012. All Rights Reserved. +%% Copyright Ericsson AB 2004-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -362,11 +362,11 @@ catch_apply(M,F,A, Default) -> [{M,F,A,_}|_] when Reason == undef -> Default; Trace -> - ct_logs:log("Suite Hook","Call to CTH failed: ~p:~p", + ct_logs:log("Suite Hook","Call to CTH failed: ~w:~p", [error,{Reason,Trace}]), throw({error_in_cth_call, lists:flatten( - io_lib:format("~p:~p/~p CTH call failed", + io_lib:format("~w:~w/~w CTH call failed", [M,F,length(A)]))}) end end. diff --git a/lib/common_test/src/ct_logs.erl b/lib/common_test/src/ct_logs.erl index 0b7a8bb075..5924930072 100644 --- a/lib/common_test/src/ct_logs.erl +++ b/lib/common_test/src/ct_logs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2012. All Rights Reserved. +%% Copyright Ericsson AB 2003-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -35,9 +35,10 @@ -export([add_external_logs/1, add_link/3]). -export([make_last_run_index/0]). -export([make_all_suites_index/1,make_all_runs_index/1]). --export([get_ts_html_wrapper/4]). +-export([get_ts_html_wrapper/5]). -export([xhtml/2, locate_priv_file/1, make_relative/1]). -export([insert_javascript/1]). +-export([uri/1]). %% Logging stuff directly from testcase -export([tc_log/3, tc_log/4, tc_log_async/3, tc_print/3, tc_print/4, @@ -307,8 +308,8 @@ end_log() -> %%% calling test suite.</p> add_external_logs(Logs) -> start_log("External Logs"), - [cont_log("<a href=~p>~s</a>\n", - [filename:join("log_private",Log),Log]) || Log <- Logs], + [cont_log("<a href=\"~ts\">~ts</a>\n", + [uri(filename:join("log_private",Log)),Log]) || Log <- Logs], end_log(). %%%----------------------------------------------------------------- @@ -320,8 +321,8 @@ add_external_logs(Logs) -> %%% @doc Print a link to a given file stored in the priv_dir of the %%% calling test suite. add_link(Heading,File,Type) -> - log(Heading,"<a href=~p type=~p>~s</a>\n", - [filename:join("log_private",File),Type,File]). + log(Heading,"<a href=\"~ts\" type=~p>~ts</a>\n", + [uri(filename:join("log_private",File)),Type,File]). %%%----------------------------------------------------------------- @@ -469,7 +470,7 @@ ct_log(Category,Format,Args) -> %%%================================================================= %%% Internal functions int_header() -> - "<div class=\"ct_internal\"><b>*** CT ~s *** ~s</b>". + "<div class=\"ct_internal\"><b>*** CT ~s *** ~ts</b>". int_footer() -> "</div>". @@ -692,7 +693,7 @@ logger_loop(State) -> logger_loop(State); {set_stylesheet,TC,SSFile} -> Fd = State#logger_state.ct_log_fd, - io:format(Fd, "~p loading external style sheet: ~s~n", + io:format(Fd, "~p loading external style sheet: ~ts~n", [TC,SSFile]), logger_loop(State#logger_state{stylesheet = SSFile}); {clear_stylesheet,_} when State#logger_state.stylesheet == undefined -> @@ -752,7 +753,7 @@ print_to_log(sync, FromPid, TCGL, List, State) -> IoProc = if FromPid /= TCGL -> TCGL; true -> State#logger_state.ct_log_fd end, - io:format(IoProc, "~s", [lists:foldl(IoFun, [], List)]), + io:format(IoProc, "~ts", [lists:foldl(IoFun, [], List)]), State; print_to_log(async, FromPid, TCGL, List, State) -> @@ -764,7 +765,7 @@ print_to_log(async, FromPid, TCGL, List, State) -> end, Printer = fun() -> test_server:permit_io(IoProc, self()), - io:format(IoProc, "~s", [lists:foldl(IoFun, [], List)]) + io:format(IoProc, "~ts", [lists:foldl(IoFun, [], List)]) end, case State#logger_state.async_print_jobs of [] -> @@ -868,7 +869,7 @@ set_evmgr_gl(GL) -> end. open_ctlog() -> - {ok,Fd} = file:open(?ct_log_name,[write]), + {ok,Fd} = file:open(?ct_log_name,[write,{encoding,utf8}]), io:format(Fd, header("Common Test Framework Log", {[],[1,2],[]}), []), case file:consult(ct_run:variables_file_name("../")) of {ok,Vars} -> @@ -878,7 +879,7 @@ open_ctlog() -> Dir = filename:dirname(Cwd), Variables = ct_run:variables_file_name(Dir), io:format(Fd, - "Can not read the file \'~s\' Reason: ~w\n" + "Can not read the file \'~ts\' Reason: ~w\n" "No configuration found for test!!\n", [Variables,Reason]) end, @@ -904,7 +905,7 @@ print_style(Fd,undefined) -> print_style(Fd,StyleSheet) -> case file:read_file(StyleSheet) of {ok,Bin} -> - Str = binary_to_list(Bin), + Str = b2s(Bin,encoding(StyleSheet)), Pos0 = case string:str(Str,"<style>") of 0 -> string:str(Str,"<STYLE>"); N0 -> N0 @@ -919,9 +920,9 @@ print_style(Fd,StyleSheet) -> print_style_error(Fd,StyleSheet,missing_style_end_tag); Pos0 /= 0 -> Style = string:sub_string(Str,Pos0,Pos1+7), - io:format(Fd,"~s\n",[Style]); + io:format(Fd,"~ts\n",[Style]); Pos0 == 0 -> - io:format(Fd,"<style>~s</style>\n",[Str]) + io:format(Fd,"<style>~ts</style>\n",[Str]) end; {error,Reason} -> print_style_error(Fd,StyleSheet,Reason) @@ -934,7 +935,7 @@ print_style(Fd,StyleSheet) -> %% [StyleSheet]). print_style_error(Fd,StyleSheet,Reason) -> - io:format(Fd,"\n<!-- Failed to load stylesheet ~s: ~p -->\n", + io:format(Fd,"\n<!-- Failed to load stylesheet ~ts: ~p -->\n", [StyleSheet,Reason]), print_style(Fd,undefined). @@ -963,7 +964,7 @@ make_last_run_index(StartTime) -> % io:put_chars("done\n"), ok; Err -> - io:format("Unknown internal error while updating ~s. " + io:format("Unknown internal error while updating ~ts. " "Please report.\n(Err: ~p, ID: 1)", [AbsIndexName,Err]), {error, Err} @@ -1001,7 +1002,7 @@ make_last_run_index1(StartTime,IndexName) -> %% write current Totals to file, later to be used in all_runs log write_totals_file(?totals_name,Label,Logs1,Totals), Index = [Index0|index_footer()], - case force_write_file(IndexName, Index) of + case force_write_file(IndexName, unicode:characters_to_binary(Index)) of ok -> ok; {error, Reason} -> @@ -1085,7 +1086,7 @@ make_one_index_entry1(SuiteName, Link, Label, Success, Fail, UserSkip, AutoSkip, CrashDumpName = SuiteName ++ "_erl_crash.dump", case filelib:is_file(CrashDumpName) of true -> - [" <a href=\"", CrashDumpName, + [" <a href=\"", uri(CrashDumpName), "\">(CrashDump)</a>"]; false -> "" @@ -1115,10 +1116,10 @@ make_one_index_entry1(SuiteName, Link, Label, Success, Fail, UserSkip, AutoSkip, [] -> "none"; _ -> "<a href=\""++?all_runs_name++"\">Old Runs</a>" end, - A = xhtml(["<td><font size=\"-1\"><a href=\"",CtLogFile, + A = xhtml(["<td><font size=\"-1\"><a href=\"",uri(CtLogFile), "\">CT Log</a></font></td>\n", "<td><font size=\"-1\">",OldRunsLink,"</font></td>\n"], - ["<td><a href=\"",CtLogFile,"\">CT Log</a></td>\n", + ["<td><a href=\"",uri(CtLogFile),"\">CT Log</a></td>\n", "<td>",OldRunsLink,"</td>\n"]), {L,T,N,A}; false -> @@ -1128,7 +1129,8 @@ make_one_index_entry1(SuiteName, Link, Label, Success, Fail, UserSkip, AutoSkip, if NotBuilt == 0 -> ["<td align=right>",integer_to_list(NotBuilt),"</td>\n"]; true -> - ["<td align=right><a href=\"",filename:join(CtRunDir,?ct_log_name),"\">", + ["<td align=right><a href=\"", + uri(filename:join(CtRunDir,?ct_log_name)),"\">", integer_to_list(NotBuilt),"</a></td>\n"] end, FailStr = @@ -1151,7 +1153,7 @@ make_one_index_entry1(SuiteName, Link, Label, Success, Fail, UserSkip, AutoSkip, [xhtml("<tr valign=top>\n", ["<tr class=\"",odd_or_even(),"\">\n"]), xhtml("<td><font size=\"-1\"><a href=\"", "<td><a href=\""), - LogFile,"\">",SuiteName,"</a>", CrashDumpLink, + uri(LogFile),"\">",SuiteName,"</a>", CrashDumpLink, xhtml("</font></td>\n", "</td>\n"), Lbl, Timestamp, "<td align=right>",integer_to_list(Success),"</td>\n", @@ -1364,8 +1366,9 @@ header1(Title, SubTitle, TableCols) -> "<head>\n", "<title>" ++ Title ++ " " ++ SubTitle ++ "</title>\n", "<meta http-equiv=\"cache-control\" content=\"no-cache\">\n", + "<meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\">\n", xhtml("", - ["<link rel=\"stylesheet\" href=\"",CSSFile,"\" type=\"text/css\">\n"]), + ["<link rel=\"stylesheet\" href=\"",uri(CSSFile),"\" type=\"text/css\">\n"]), xhtml("", ["<script type=\"text/javascript\" src=\"",JQueryFile, "\"></script>\n"]), @@ -1462,7 +1465,7 @@ count_cases(Dir) -> LogFile = filename:join(Dir, ?suitelog_name), case file:read_file(LogFile) of {ok, Bin} -> - case count_cases1(binary_to_list(Bin), + case count_cases1(b2s(Bin), {undefined,undefined,undefined,undefined}) of {error,not_complete} -> %% The test is not complete - dont write summary @@ -1557,7 +1560,7 @@ config_table1([{Key,Value}|Vars]) -> "<td><pre>",io_lib:format("~p",[Value]),"</pre></td></tr>\n"], ["<tr class=\"", odd_or_even(), "\">\n", "<td>", atom_to_list(Key), "</td>\n", - "<td>", io_lib:format("~p",[Value]), "</td>\n</tr>\n"]) | + "<td>", io_lib:format("~p",[Value]), "</td>\n</tr>\n"]) | config_table1(Vars)]; config_table1([]) -> ["</tbody>\n</table>\n"]. @@ -1574,8 +1577,9 @@ make_all_runs_index(When) -> DirsSorted = (catch sort_all_runs(Dirs)), Header = all_runs_header(), Index = [runentry(Dir) || Dir <- DirsSorted], - Result = file:write_file(AbsName,Header++Index++ - all_runs_index_footer()), + Result = file:write_file(AbsName, + unicode:characters_to_binary( + Header++Index++all_runs_index_footer())), if When == start -> ok; true -> io:put_chars("done\n") end, @@ -1602,10 +1606,27 @@ sort_all_runs(Dirs) -> interactive_link() -> [Dir|_] = lists:reverse(filelib:wildcard(logdir_prefix()++"*.*")), CtLog = filename:join(Dir,"ctlog.html"), - Body = ["Log from last interactive run: <a href=\"",CtLog,"\">", - timestamp(Dir),"</a>"], - file:write_file("last_interactive.html",Body), - io:format("~n~nUpdated ~s\n" + Body = + [xhtml( + ["<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n", + "<html>\n"], + ["<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\"\n", + "\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n", + "<html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"en\" lang=\"en\">\n"]), + "<!-- autogenerated by '"++atom_to_list(?MODULE)++"' -->\n", + "<head>\n", + "<title>Last interactive run</title>\n", + "<meta http-equiv=\"cache-control\" content=\"no-cache\">\n", + "<meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\">\n", + "</head>\n", + "<body>\n", + "Log from last interactive run: <a href=\"",uri(CtLog),"\">", + timestamp(Dir),"</a>", + "</body>\n", + "</html>\n" + ], + file:write_file("last_interactive.html",unicode:characters_to_binary(Body)), + io:format("~n~nUpdated ~ts\n" "Any CT activities will be logged here\n", [?abs("last_interactive.html")]). @@ -1655,7 +1676,7 @@ runentry(Dir) -> TestNames; true -> Trunc = Polish(string:substr(TestNames,1,?testname_width-3)), - lists:flatten(io_lib:format("~s...",[Trunc])) + lists:flatten(io_lib:format("~ts...",[Trunc])) end, Total = TotSucc+TotFail+AllSkip, A = xhtml(["<td align=center><font size=\"-1\">",Node, @@ -1695,7 +1716,7 @@ runentry(Dir) -> "<td align=right>?</td>\n"], A++B++C end, - Index = filename:join(Dir,?index_name), + Index = uri(filename:join(Dir,?index_name)), [xhtml("<tr>\n", ["<tr class=\"",odd_or_even(),"\">\n"]), xhtml(["<td><font size=\"-1\"><a href=\"",Index,"\">",timestamp(Dir),"</a>", TotalsStr,"</font></td>\n"], @@ -1836,7 +1857,7 @@ make_all_suites_index(NewTestData = {_TestName,DirName}) -> ok -> ok; Err -> - io:format("Unknown internal error while updating ~s. " + io:format("Unknown internal error while updating ~ts. " "Please report.\n(Err: ~p, ID: 1)", [AbsIndexName,Err]), {error, Err} @@ -1900,7 +1921,7 @@ make_all_suites_index1(When, AbsIndexName, AllLogDirs) -> ok end; Err -> - io:format("Unknown internal error while updating ~s. " + io:format("Unknown internal error while updating ~ts. " "Please report.\n(Err: ~p, ID: 1)", [AbsIndexName,Err]), {error, Err} @@ -1912,7 +1933,7 @@ make_all_suites_index2(IndexName, AllTestLogDirs) -> all_suites_index_header(), 0, 0, 0, 0, 0, [], []), Index = [Index0|index_footer()], - case force_write_file(IndexName, Index) of + case force_write_file(IndexName, unicode:characters_to_binary(Index)) of ok -> {ok,CacheData}; {error, Reason} -> @@ -1970,7 +1991,7 @@ make_all_suites_ix_cached(AbsIndexName, NewTestData, Label, AllTestLogDirs) -> all_suites_index_header(IndexDir), 0, 0, 0, 0, 0), Index = [Index0|index_footer()], - case force_write_file(AbsIndexName, Index) of + case force_write_file(AbsIndexName, unicode:characters_to_binary(Index)) of ok -> ok; {error, Reason} -> @@ -2116,7 +2137,7 @@ simulate_logger_loop() -> receive {log,_,_,_,_,_,List} -> S = [[io_lib:format(Str,Args),io_lib:nl()] || {Str,Args} <- List], - io:format("~s",[S]), + io:format("~ts",[S]), simulate_logger_loop(); stop -> ok @@ -2273,11 +2294,12 @@ make_relative1(DirTs, CwdTs) -> Ups ++ DirTs. %%%----------------------------------------------------------------- -%%% @spec get_ts_html_wrapper(TestName, PrintLabel, Cwd) -> {Mode,Header,Footer} +%%% @spec get_ts_html_wrapper(TestName, PrintLabel, Cwd, TableCols, Encoding) +%%% -> {Mode,Header,Footer} %%% %%% @doc %%% -get_ts_html_wrapper(TestName, PrintLabel, Cwd, TableCols) -> +get_ts_html_wrapper(TestName, PrintLabel, Cwd, TableCols, Encoding) -> TestName1 = if is_list(TestName) -> lists:flatten(TestName); true -> @@ -2319,14 +2341,16 @@ get_ts_html_wrapper(TestName, PrintLabel, Cwd, TableCols) -> "<html>\n", "<head><title>", TestName1, "</title>\n", "<meta http-equiv=\"cache-control\" content=\"no-cache\">\n", + "<meta http-equiv=\"content-type\" content=\"text/html; charset=", + html_encoding(Encoding),"\">\n", "</head>\n", "<body", Bgr, " bgcolor=\"white\" text=\"black\" ", "link=\"blue\" vlink=\"purple\" alink=\"red\">\n", LabelStr, "\n"], ["<center>\n<br><hr><p>\n", - "<a href=\"", AllRuns, + "<a href=\"", uri(AllRuns), "\">Test run history\n</a> | ", - "<a href=\"", TestIndex, + "<a href=\"", uri(TestIndex), "\">Top level test index\n</a>\n</p>\n", Copyright,"</center>\n</body>\n</html>\n"]}; _ -> @@ -2363,14 +2387,15 @@ get_ts_html_wrapper(TestName, PrintLabel, Cwd, TableCols) -> "<html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"en\" lang=\"en\">\n", "<head>\n<title>", TestName1, "</title>\n", "<meta http-equiv=\"cache-control\" content=\"no-cache\">\n", - "<link rel=\"stylesheet\" href=\"", CSSFile, "\" type=\"text/css\">\n", + "<meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\">\n", + "<link rel=\"stylesheet\" href=\"", uri(CSSFile), "\" type=\"text/css\">\n", "<script type=\"text/javascript\" src=\"", JQueryFile, "\"></script>\n", "<script type=\"text/javascript\" src=\"", TableSorterFile, "\"></script>\n"] ++ TableSorterScript ++ ["</head>\n","<body>\n", LabelStr, "\n"], ["<center>\n<br /><hr /><p>\n", - "<a href=\"", AllRuns, + "<a href=\"", uri(AllRuns), "\">Test run history\n</a> | ", - "<a href=\"", TestIndex, + "<a href=\"", uri(TestIndex), "\">Top level test index\n</a>\n</p>\n", Copyright,"</center>\n</body>\n</html>\n"]} end. @@ -2460,3 +2485,31 @@ insert_javascript({tablesorter,TableName, " $(\"#",TableName,"\").trigger(\"update\");\n", " $(\"#",TableName,"\").trigger(\"appendCache\");\n", "});\n</script>\n"]. + +uri("") -> + ""; +uri(Href) -> + test_server_ctrl:uri_encode(Href). + +%% Read magic comment to get encoding of text file. +%% If no magic comment exists, assume default encoding +encoding(File) -> + case epp:read_encoding(File) of + none -> + epp:default_encoding(); + E -> + E + end. + +%% Convert binary to string using default encoding +b2s(Bin) -> + b2s(Bin,epp:default_encoding()). + +%% Convert binary to string using given encoding +b2s(Bin,Encoding) -> + unicode:characters_to_list(Bin,Encoding). + +html_encoding(latin1) -> + "iso-8859-1"; +html_encoding(utf8) -> + "utf-8". diff --git a/lib/common_test/src/ct_make.erl b/lib/common_test/src/ct_make.erl index 8ddb91d355..d4bd81e78d 100644 --- a/lib/common_test/src/ct_make.erl +++ b/lib/common_test/src/ct_make.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2011. All Rights Reserved. +%% Copyright Ericsson AB 2009-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -280,13 +280,13 @@ recompile(File, NoExec, Load, Opts) -> do_recompile(_File, true, _Load, _Opts) -> out_of_date; do_recompile(File, false, noload, Opts) -> - io:format("Recompile: ~s\n",[File]), + io:format("Recompile: ~ts\n",[File]), compile:file(File, [report_errors, report_warnings, error_summary |Opts]); do_recompile(File, false, load, Opts) -> - io:format("Recompile: ~s\n",[File]), + io:format("Recompile: ~ts\n",[File]), c:c(File, Opts); do_recompile(File, false, netload, Opts) -> - io:format("Recompile: ~s\n",[File]), + io:format("Recompile: ~ts\n",[File]), c:nc(File, Opts). exists(File) -> diff --git a/lib/common_test/src/ct_master.erl b/lib/common_test/src/ct_master.erl index f29eba605c..e516f635d2 100644 --- a/lib/common_test/src/ct_master.erl +++ b/lib/common_test/src/ct_master.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2012. All Rights Reserved. +%% Copyright Ericsson AB 2006-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -82,39 +82,48 @@ run_test(NodeOptsList) when is_list(NodeOptsList) -> %%% ExclNodes = [atom()] %%% %%% @doc Tests are spawned on the nodes as specified in <code>TestSpecs</code>. -%%% Each specification in TestSpec will be handled separately. It is however possible -%%% to also specify a list of specifications that should be merged into one before -%%% the tests are executed. Any test without a particular node specification will -%%% also be executed on the nodes in <code>InclNodes</code>. Nodes in the -%%% <code>ExclNodes</code> list will be excluded from the test. +%%% Each specification in TestSpec will be handled separately. It is however +%%% possible to also specify a list of specifications that should be merged +%%% into one before the tests are executed. Any test without a particular node +%%% specification will also be executed on the nodes in <code>InclNodes</code>. +%%% Nodes in the <code>ExclNodes</code> list will be excluded from the test. run([TS|TestSpecs],AllowUserTerms,InclNodes,ExclNodes) when is_list(TS), is_list(InclNodes), is_list(ExclNodes) -> - TS1 = - case TS of - List=[S|_] when is_list(S) -> List; - Spec -> [Spec] - end, - Result = - case catch ct_testspec:collect_tests_from_file(TS1,InclNodes,AllowUserTerms) of - {error,Reason} -> - {error,Reason}; - TSRec=#testspec{logdir=AllLogDirs, - config=StdCfgFiles, - userconfig=UserCfgFiles, - include=AllIncludes, - init=AllInitOpts, - event_handler=AllEvHs} -> - AllCfgFiles = {StdCfgFiles, UserCfgFiles}, - RunSkipPerNode = ct_testspec:prepare_tests(TSRec), - RunSkipPerNode2 = exclude_nodes(ExclNodes,RunSkipPerNode), - run_all(RunSkipPerNode2,AllLogDirs,AllCfgFiles,AllEvHs, - AllIncludes,[],[],AllInitOpts,TS1) - end, - [{TS,Result} | run(TestSpecs,AllowUserTerms,InclNodes,ExclNodes)]; + %% Note: [Spec] means run one test with Spec + %% [Spec1,Spec2] means run two tests separately + %% [[Spec1,Spec2]] means run one test, with the two specs merged + case catch ct_testspec:collect_tests_from_file([TS],InclNodes, + AllowUserTerms) of + {error,Reason} -> + [{error,Reason} | run(TestSpecs,AllowUserTerms,InclNodes,ExclNodes)]; + Tests -> + RunResult = + lists:map( + fun({Specs,TSRec=#testspec{logdir=AllLogDirs, + config=StdCfgFiles, + userconfig=UserCfgFiles, + include=AllIncludes, + init=AllInitOpts, + event_handler=AllEvHs}}) -> + AllCfgFiles = + {StdCfgFiles,UserCfgFiles}, + RunSkipPerNode = + ct_testspec:prepare_tests(TSRec), + RunSkipPerNode2 = + exclude_nodes(ExclNodes,RunSkipPerNode), + TSList = if is_integer(hd(TS)) -> [TS]; + true -> TS end, + {Specs,run_all(RunSkipPerNode2,AllLogDirs, + AllCfgFiles,AllEvHs, + AllIncludes,[],[],AllInitOpts,TSList)} + end, Tests), + RunResult ++ run(TestSpecs,AllowUserTerms,InclNodes,ExclNodes) + end; run([],_,_,_) -> []; -run(TS,AllowUserTerms,InclNodes,ExclNodes) when is_list(InclNodes), is_list(ExclNodes) -> +run(TS,AllowUserTerms,InclNodes,ExclNodes) when is_list(InclNodes), + is_list(ExclNodes) -> run([TS],AllowUserTerms,InclNodes,ExclNodes). %%%----------------------------------------------------------------- @@ -152,29 +161,32 @@ exclude_nodes([],RunSkipPerNode) -> %%% AllowUserTerms = bool() %%% Node = atom() %%% -%%% @doc Tests are spawned on <code>Node</code> according to <code>TestSpecs</code>. +%%% @doc Tests are spawned on <code>Node</code> according to +%%% <code>TestSpecs</code>. run_on_node([TS|TestSpecs],AllowUserTerms,Node) when is_list(TS),is_atom(Node) -> - TS1 = - case TS of - [List|_] when is_list(List) -> List; - Spec -> [Spec] - end, - Result = - case catch ct_testspec:collect_tests_from_file(TS1,[Node],AllowUserTerms) of - {error,Reason} -> - {error,Reason}; - TSRec=#testspec{logdir=AllLogDirs, - config=StdCfgFiles, - init=AllInitOpts, - include=AllIncludes, - userconfig=UserCfgFiles, - event_handler=AllEvHs} -> - AllCfgFiles = {StdCfgFiles, UserCfgFiles}, - {Run,Skip} = ct_testspec:prepare_tests(TSRec,Node), - run_all([{Node,Run,Skip}],AllLogDirs,AllCfgFiles,AllEvHs, - AllIncludes, [],[],AllInitOpts,TS1) - end, - [{TS,Result} | run_on_node(TestSpecs,AllowUserTerms,Node)]; + case catch ct_testspec:collect_tests_from_file([TS],[Node], + AllowUserTerms) of + {error,Reason} -> + [{error,Reason} | run_on_node(TestSpecs,AllowUserTerms,Node)]; + Tests -> + RunResult = + lists:map( + fun({Specs,TSRec=#testspec{logdir=AllLogDirs, + config=StdCfgFiles, + init=AllInitOpts, + include=AllIncludes, + userconfig=UserCfgFiles, + event_handler=AllEvHs}}) -> + AllCfgFiles = {StdCfgFiles,UserCfgFiles}, + {Run,Skip} = ct_testspec:prepare_tests(TSRec,Node), + TSList = if is_integer(hd(TS)) -> [TS]; + true -> TS end, + {Specs,run_all([{Node,Run,Skip}],AllLogDirs, + AllCfgFiles,AllEvHs, + AllIncludes, [],[],AllInitOpts,TSList)} + end, Tests), + RunResult ++ run_on_node(TestSpecs,AllowUserTerms,Node) + end; run_on_node([],_,_) -> []; run_on_node(TS,AllowUserTerms,Node) when is_atom(Node) -> @@ -244,8 +256,9 @@ run_all([],AllLogDirs,_,AllEvHs,_AllIncludes, {value,{_,Dir}} -> Dir; false -> "." end, - log(tty,"Master Logdir","~s",[MasterLogDir]), - start_master(lists:reverse(NodeOpts),Handlers,MasterLogDir,LogDirs,InitOptions,Specs), + log(tty,"Master Logdir","~ts",[MasterLogDir]), + start_master(lists:reverse(NodeOpts),Handlers,MasterLogDir, + LogDirs,InitOptions,Specs), ok. @@ -297,13 +310,15 @@ start_master(NodeOptsList) -> start_master(NodeOptsList,EvHandlers,MasterLogDir,LogDirs,InitOptions,Specs) -> Master = spawn_link(?MODULE,init_master,[self(),NodeOptsList,EvHandlers, - MasterLogDir,LogDirs,InitOptions,Specs]), + MasterLogDir,LogDirs, + InitOptions,Specs]), receive {Master,Result} -> Result end. %%% @hidden -init_master(Parent,NodeOptsList,EvHandlers,MasterLogDir,LogDirs,InitOptions,Specs) -> +init_master(Parent,NodeOptsList,EvHandlers,MasterLogDir,LogDirs, + InitOptions,Specs) -> case whereis(ct_master) of undefined -> register(ct_master,self()), @@ -325,13 +340,14 @@ init_master(Parent,NodeOptsList,EvHandlers,MasterLogDir,LogDirs,InitOptions,Spec {MLPid,_} = ct_master_logs:start(MasterLogDir, [N || {N,_} <- NodeOptsList]), log(all,"Master Logger process started","~w",[MLPid]), + case Specs of [] -> ok; _ -> SpecsStr = lists:map(fun(Name) -> Name ++ " " end,Specs), - ct_master_logs:log("Test Specification file(s)","~s", + ct_master_logs:log("Test Specification file(s)","~ts", [lists:flatten(SpecsStr)]) end, @@ -340,7 +356,7 @@ init_master(Parent,NodeOptsList,EvHandlers,MasterLogDir,LogDirs,InitOptions,Spec ct_master_event:add_handler(), %% add user handlers for master event manager Add = fun({H,Args}) -> - log(all,"Adding Event Handler","~p",[H]), + log(all,"Adding Event Handler","~w",[H]), case gen_event:add_handler(?CT_MEVMGR_REF,H,Args) of ok -> ok; {'EXIT',Why} -> exit(Why); @@ -359,7 +375,8 @@ init_master(Parent,NodeOptsList,EvHandlers,MasterLogDir,LogDirs,InitOptions,Spec init_master1(Parent,NodeOptsList,InitOptions,LogDirs). init_master1(Parent,NodeOptsList,InitOptions,LogDirs) -> - {Inaccessible,NodeOptsList1,InitOptions1} = init_nodes(NodeOptsList,InitOptions), + {Inaccessible,NodeOptsList1,InitOptions1} = init_nodes(NodeOptsList, + InitOptions), case Inaccessible of [] -> init_master2(Parent,NodeOptsList,LogDirs); @@ -391,8 +408,9 @@ init_master2(Parent,NodeOptsList,LogDirs) -> SpawnAndMon = fun({Node,Opts}) -> monitor_node(Node,true), - log(all,"Test Info","Starting test(s) on ~p...",[Node]), - {spawn_link(Node,?MODULE,init_node_ctrl,[self(),Cookie,Opts]),Node} + log(all,"Test Info","Starting test(s) on ~w...",[Node]), + {spawn_link(Node,?MODULE,init_node_ctrl,[self(),Cookie,Opts]), + Node} end, NodeCtrlPids = lists:map(SpawnAndMon,NodeOptsList), Result = master_loop(#state{node_ctrl_pids=NodeCtrlPids, @@ -404,7 +422,8 @@ master_loop(#state{node_ctrl_pids=[], results=Finished}) -> Str = lists:map(fun({Node,Result}) -> - io_lib:format("~-40.40.*s~p\n",[$_,atom_to_list(Node),Result]) + io_lib:format("~-40.40.*ts~p\n", + [$_,atom_to_list(Node),Result]) end,lists:reverse(Finished)), log(all,"TEST RESULTS",Str,[]), log(all,"Info","Updating log files",[]), @@ -437,18 +456,20 @@ master_loop(State=#state{node_ctrl_pids=NodeCtrlPids, Bad end, log(all,"Test Info", - "Test on node ~w failed! Reason: ~p",[Node,Error]), + "Test on node ~w failed! Reason: ~p", + [Node,Error]), {Locks1,Blocked1} = update_queue(exit,Node,Locks,Blocked), master_loop(State#state{node_ctrl_pids=NodeCtrlPids1, - results=[{Node,Error}|Results], + results=[{Node, + Error}|Results], locks=Locks1, blocked=Blocked1}) end; undefined -> %% ignore (but report) exit from master_logger etc log(all,"Test Info", - "Warning! Process ~p has terminated. Reason: ~p", + "Warning! Process ~w has terminated. Reason: ~p", [Pid,Reason]), master_loop(State) end; @@ -531,7 +552,7 @@ update_queue(take,Node,From,Lock={Op,Resource},Locks,Blocked) -> %% Blocked: [{{Operation,Resource},Node,WaitingPid},...] case lists:keysearch(Lock,1,Locks) of {value,{_Lock,Owner}} -> % other node has lock - log(html,"Lock Info","Node ~p blocked on ~w by ~w. Resource: ~p", + log(html,"Lock Info","Node ~w blocked on ~w by ~w. Resource: ~p", [Node,Op,Owner,Resource]), Blocked1 = Blocked ++ [{Lock,Node,From}], {Locks,Blocked1}; @@ -546,7 +567,7 @@ update_queue(release,Node,_From,Lock={Op,Resource},Locks,Blocked) -> case lists:keysearch(Lock,1,Blocked) of {value,E={Lock,SomeNode,WaitingPid}} -> Blocked1 = lists:delete(E,Blocked), - log(html,"Lock Info","Node ~p proceeds with ~w. Resource: ~p", + log(html,"Lock Info","Node ~w proceeds with ~w. Resource: ~p", [SomeNode,Op,Resource]), reply(ok,WaitingPid), % waiting process may start {Locks1,Blocked1}; @@ -625,7 +646,8 @@ refresh_logs([D|Dirs],Refreshed) -> refresh_logs([],Refreshed) -> Str = lists:map(fun({D,Result}) -> - io_lib:format("Refreshing logs in ~p... ~p",[D,Result]) + io_lib:format("Refreshing logs in ~p... ~p", + [D,Result]) end,Refreshed), log(all,"Info",Str,[]). @@ -638,7 +660,7 @@ init_node_ctrl(MasterPid,Cookie,Opts) -> process_flag(trap_exit, true), MasterNode = node(MasterPid), group_leader(whereis(user),self()), - io:format("~n********** node_ctrl process ~p started on ~p **********~n", + io:format("~n********** node_ctrl process ~w started on ~w **********~n", [self(),node()]), %% initially this node must have the same cookie as the master node %% but now we set it explicitly for the connection so that test suites @@ -671,7 +693,7 @@ init_node_ctrl(MasterPid,Cookie,Opts) -> pong -> MasterPid ! {self(),{result,Result}}; pang -> - io:format("Warning! Connection to master node ~p is lost. " + io:format("Warning! Connection to master node ~w is lost. " "Can't report result!~n~n", [MasterNode]) end. @@ -752,21 +774,25 @@ start_nodes(InitOptions)-> IsAlive = lists:member(NodeName, nodes()), case {HasNodeStart, IsAlive} of {false, false}-> - io:format("WARNING: Node ~p is not alive but has no node_start option~n", [NodeName]); + io:format("WARNING: Node ~w is not alive but has no " + "node_start option~n", [NodeName]); {false, true}-> - io:format("Node ~p is alive~n", [NodeName]); + io:format("Node ~w is alive~n", [NodeName]); {true, false}-> {node_start, NodeStart} = lists:keyfind(node_start, 1, Options), {value, {callback_module, Callback}, NodeStart2}= lists:keytake(callback_module, 1, NodeStart), case Callback:start(Host, Node, NodeStart2) of {ok, NodeName} -> - io:format("Node ~p started successfully with callback ~p~n", [NodeName,Callback]); + io:format("Node ~w started successfully " + "with callback ~w~n", [NodeName,Callback]); {error, Reason, _NodeName} -> - io:format("Failed to start node ~p with callback ~p! Reason: ~p~n", [NodeName, Callback, Reason]) + io:format("Failed to start node ~w with callback ~w! " + "Reason: ~p~n", [NodeName, Callback, Reason]) end; {true, true}-> - io:format("WARNING: Node ~p is alive but has node_start option~n", [NodeName]) + io:format("WARNING: Node ~w is alive but has node_start " + "option~n", [NodeName]) end end, InitOptions). @@ -779,7 +805,8 @@ eval_on_nodes(InitOptions)-> {false,_}-> ok; {true,false}-> - io:format("WARNING: Node ~p is not alive but has eval option ~n", [NodeName]); + io:format("WARNING: Node ~w is not alive but has eval " + "option~n", [NodeName]); {true,true}-> {eval, MFAs} = lists:keyfind(eval, 1, Options), evaluate(NodeName, MFAs) @@ -790,9 +817,11 @@ eval_on_nodes(InitOptions)-> evaluate(Node, [{M,F,A}|MFAs])-> case rpc:call(Node, M, F, A) of {badrpc,Reason}-> - io:format("WARNING: Failed to call ~p:~p/~p on node ~p due to ~p~n", [M,F,length(A),Node,Reason]); + io:format("WARNING: Failed to call ~w:~w/~w on node ~w " + "due to ~p~n", [M,F,length(A),Node,Reason]); Result-> - io:format("Called ~p:~p/~p on node ~p, result: ~p~n", [M,F,length(A),Node,Result]) + io:format("Called ~w:~w/~w on node ~w, result: ~p~n", + [M,F,length(A),Node,Result]) end, evaluate(Node, MFAs); evaluate(_Node, [])-> diff --git a/lib/common_test/src/ct_master_event.erl b/lib/common_test/src/ct_master_event.erl index a70baefaaf..5877b7c6f2 100644 --- a/lib/common_test/src/ct_master_event.erl +++ b/lib/common_test/src/ct_master_event.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2009. All Rights Reserved. +%% Copyright Ericsson AB 2006-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -114,13 +114,13 @@ init(_) -> %% each installed event handler to handle the event. %%-------------------------------------------------------------------- handle_event(#event{name=start_logging,node=Node,data=RunDir},State) -> - ct_master_logs:log("CT Master Event Handler","Got ~s from ~p",[RunDir,Node]), + ct_master_logs:log("CT Master Event Handler","Got ~ts from ~w",[RunDir,Node]), ct_master_logs:nodedir(Node,RunDir), {ok,State}; handle_event(#event{name=Name,node=Node,data=Data},State) -> print("~n=== ~w ===~n", [?MODULE]), - print("~p on ~p: ~p~n", [Name,Node,Data]), + print("~w on ~w: ~p~n", [Name,Node,Data]), {ok,State}. %%-------------------------------------------------------------------- diff --git a/lib/common_test/src/ct_master_logs.erl b/lib/common_test/src/ct_master_logs.erl index 84f175c0a9..5393097f57 100644 --- a/lib/common_test/src/ct_master_logs.erl +++ b/lib/common_test/src/ct_master_logs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2012. All Rights Reserved. +%% Copyright Ericsson AB 2006-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -133,7 +133,7 @@ init(Parent,LogDir,Nodes) -> end,Nodes)), io:format(CtLogFd,int_header(),[log_timestamp(now()),"Test Nodes\n"]), - io:format(CtLogFd,"~s\n",[NodeStr]), + io:format(CtLogFd,"~ts\n",[NodeStr]), io:put_chars(CtLogFd,[int_footer(),"\n"]), NodeDirIxFd = open_nodedir_index(RunDirAbs,Time), @@ -201,7 +201,7 @@ loop(State) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%% open_ct_master_log(Dir) -> FullName = filename:join(Dir,?ct_master_log_name), - {ok,Fd} = file:open(FullName,[write]), + {ok,Fd} = file:open(FullName,[write,{encoding,utf8}]), io:put_chars(Fd,header("Common Test Master Log", {[],[1,2],[]})), %% maybe add config info here later io:put_chars(Fd,config_table([])), @@ -235,7 +235,7 @@ config_table1([]) -> ["</tbody>\n</table>\n"]. int_header() -> - "<div class=\"ct_internal\"><b>*** CT MASTER ~s *** ~s</b>". + "<div class=\"ct_internal\"><b>*** CT MASTER ~s *** ~ts</b>". int_footer() -> "</div>". @@ -244,7 +244,7 @@ int_footer() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% open_nodedir_index(Dir,StartTime) -> FullName = filename:join(Dir,?nodedir_index_name), - {ok,Fd} = file:open(FullName,[write]), + {ok,Fd} = file:open(FullName,[write,{encoding,utf8}]), io:put_chars(Fd,nodedir_index_header(StartTime)), Fd. @@ -253,7 +253,8 @@ print_nodedir(Node,RunDir,Fd) -> io:put_chars(Fd, ["<tr>\n" "<td align=center>",atom_to_list(Node),"</td>\n", - "<td align=left><a href=\"",Index,"\">",Index,"</a></td>\n", + "<td align=left><a href=\"",ct_logs:uri(Index),"\">",Index, + "</a></td>\n", "</tr>\n"]), ok. @@ -283,7 +284,9 @@ make_all_runs_index(LogDir) -> DirsSorted = (catch sort_all_runs(Dirs)), Header = all_runs_header(), Index = [runentry(Dir) || Dir <- DirsSorted], - Result = file:write_file(FullName,Header++Index++index_footer()), + Result = file:write_file(FullName, + unicode:characters_to_binary( + Header++Index++index_footer())), Result. sort_all_runs(Dirs) -> @@ -323,7 +326,8 @@ runentry(Dir) -> end, Index = filename:join(Dir,?nodedir_index_name), ["<tr>\n" - "<td align=center><a href=\"",Index,"\">",timestamp(Dir),"</a></td>\n", + "<td align=center><a href=\"",ct_logs:uri(Index),"\">", + timestamp(Dir),"</a></td>\n", "<td align=center>",MasterStr,"</td>\n", "<td align=center>",NodesStr,"</td>\n", "</tr>\n"]. @@ -381,8 +385,10 @@ header(Title, TableCols) -> "<head>\n", "<title>" ++ Title ++ "</title>\n", "<meta http-equiv=\"cache-control\" content=\"no-cache\">\n", + "<meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\">\n", xhtml("", - ["<link rel=\"stylesheet\" href=\"",CSSFile,"\" type=\"text/css\">"]), + ["<link rel=\"stylesheet\" href=\"",ct_logs:uri(CSSFile), + "\" type=\"text/css\">"]), xhtml("", ["<script type=\"text/javascript\" src=\"",JQueryFile, "\"></script>\n"]), diff --git a/lib/common_test/src/ct_master_status.erl b/lib/common_test/src/ct_master_status.erl index 76060fb7bb..f9f511ecca 100644 --- a/lib/common_test/src/ct_master_status.erl +++ b/lib/common_test/src/ct_master_status.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2009. All Rights Reserved. +%% Copyright Ericsson AB 2006-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -70,7 +70,7 @@ init(_) -> %% handle_event(#event{name=Name,node=Node,data=Data},State) -> print("~n=== ~w ===~n", [?MODULE]), - print("~p on ~p: ~p~n", [Name,Node,Data]), + print("~w on ~w: ~p~n", [Name,Node,Data]), {ok,State}. %%-------------------------------------------------------------------- diff --git a/lib/common_test/src/ct_netconfc.erl b/lib/common_test/src/ct_netconfc.erl index 1ccbc86d8f..1339e53780 100644 --- a/lib/common_test/src/ct_netconfc.erl +++ b/lib/common_test/src/ct_netconfc.erl @@ -1,7 +1,7 @@ %%---------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1281,10 +1281,11 @@ do_send(Connection, SimpleXml) -> to_xml_doc(Simple) -> Prolog = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>", - Xml = list_to_binary(xmerl:export_simple([Simple], - xmerl_xml, - [#xmlAttribute{name=prolog, - value=Prolog}])), + Xml = unicode:characters_to_binary( + xmerl:export_simple([Simple], + xmerl_xml, + [#xmlAttribute{name=prolog, + value=Prolog}])), <<Xml/binary,?END_TAG/binary>>. %%%----------------------------------------------------------------- @@ -1688,18 +1689,27 @@ log(#connection{host=Host,port=Port,name=Name},Action,Data) -> %% Log callback - called from the error handler process -format_data(raw,Data) -> - io_lib:format("~n~s~n",[hide_password(Data)]); -format_data(pretty,Data) -> - io_lib:format("~n~s~n",[indent(Data)]); -format_data(html,Data) -> - io_lib:format("~n~s~n",[html_format(Data)]). +format_data(How,Data) -> + %% Assuming that the data is encoded as UTF-8. If it is not, then + %% the printout might be wrong, but the format function will not + %% crash! + %% FIXME: should probably read encoding from the data and do + %% unicode:characters_to_binary(Data,InEncoding,utf8) when calling + %% log/3 instead of assuming utf8 in as done here! + do_format_data(How,unicode:characters_to_binary(Data)). + +do_format_data(raw,Data) -> + io_lib:format("~n~ts~n",[hide_password(Data)]); +do_format_data(pretty,Data) -> + io_lib:format("~n~ts~n",[indent(Data)]); +do_format_data(html,Data) -> + io_lib:format("~n~ts~n",[html_format(Data)]). %%%----------------------------------------------------------------- %%% Hide password elements from XML data hide_password(Bin) -> re:replace(Bin,<<"(<password[^>]*>)[^<]*(</password>)">>,<<"\\1*****\\2">>, - [global,{return,binary}]). + [global,{return,binary},unicode]). %%%----------------------------------------------------------------- %%% HTML formatting @@ -1717,13 +1727,13 @@ indent(Bin) -> Part -> indent1(lists:reverse(Part)++String,erase(indent)) end, - list_to_binary(IndentedString). + unicode:characters_to_binary(IndentedString). %% Normalizes the XML document by removing all space and newline %% between two XML tags. %% Returns a list, no matter if the input was a list or a binary. -normalize(Str) -> - re:replace(Str,<<">[ \r\n\t]+<">>,<<"><">>,[global,{return,list}]). +normalize(Bin) -> + re:replace(Bin,<<">[ \r\n\t]+<">>,<<"><">>,[global,{return,list},unicode]). indent1("<?"++Rest1,Indent1) -> diff --git a/lib/common_test/src/ct_run.erl b/lib/common_test/src/ct_run.erl index eb05c90ba8..c0bdbb2a09 100644 --- a/lib/common_test/src/ct_run.erl +++ b/lib/common_test/src/ct_run.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2012. All Rights Reserved. +%% Copyright Ericsson AB 2004-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -148,7 +148,7 @@ script_start(Args) -> _ -> "" end end, - io:format("~nCommon Test~s starting (cwd is ~s)~n~n", + io:format("~nCommon Test~s starting (cwd is ~ts)~n~n", [CTVsn,Cwd]), Self = self(), Pid = spawn_link(fun() -> script_start1(Self, Args) end), @@ -330,33 +330,33 @@ script_start1(Parent, Args) -> true end, - StartOpts = #opts{label = Label, profile = Profile, - vts = Vts, shell = Shell, - cover = Cover, cover_stop = CoverStop, - logdir = LogDir, logopts = LogOpts, - basic_html = BasicHtml, - verbosity = Verbosity, - event_handlers = EvHandlers, - ct_hooks = CTHooks, - enable_builtin_hooks = EnableBuiltinHooks, - auto_compile = AutoCompile, - include = IncludeDirs, - silent_connections = SilentConns, - stylesheet = Stylesheet, - multiply_timetraps = MultTT, - scale_timetraps = ScaleTT, - create_priv_dir = CreatePrivDir, - starter = script}, - + Opts = #opts{label = Label, profile = Profile, + vts = Vts, shell = Shell, + cover = Cover, cover_stop = CoverStop, + logdir = LogDir, logopts = LogOpts, + basic_html = BasicHtml, + verbosity = Verbosity, + event_handlers = EvHandlers, + ct_hooks = CTHooks, + enable_builtin_hooks = EnableBuiltinHooks, + auto_compile = AutoCompile, + include = IncludeDirs, + silent_connections = SilentConns, + stylesheet = Stylesheet, + multiply_timetraps = MultTT, + scale_timetraps = ScaleTT, + create_priv_dir = CreatePrivDir, + starter = script}, + %% check if log files should be refreshed or go on to run tests... - Result = run_or_refresh(StartOpts, Args), + Result = run_or_refresh(Opts, Args), %% send final results to starting process waiting in script_start/0 Parent ! {self(), Result}. -run_or_refresh(StartOpts = #opts{logdir = LogDir}, Args) -> +run_or_refresh(Opts = #opts{logdir = LogDir}, Args) -> case proplists:get_value(refresh_logs, Args) of undefined -> - script_start2(StartOpts, Args); + script_start2(Opts, Args); Refresh -> LogDir1 = case Refresh of [] -> which(logdir,LogDir); @@ -378,171 +378,203 @@ run_or_refresh(StartOpts = #opts{logdir = LogDir}, Args) -> {error,{all_suites_index,ASReason}}; _ -> file:set_cwd(Cwd), - io:format("Logs in ~s refreshed!~n~n", [LogDir1]), + io:format("Logs in ~ts refreshed!~n~n", + [LogDir1]), timer:sleep(500), % time to flush io before quitting ok end end end. -script_start2(StartOpts = #opts{vts = undefined, - shell = undefined}, Args) -> - TestSpec = proplists:get_value(spec, Args), - {Terms,Opts} = - case TestSpec of - Specs when Specs =/= [], Specs =/= undefined -> - %% using testspec as input for test - Relaxed = get_start_opt(allow_user_terms, true, false, Args), - case catch ct_testspec:collect_tests_from_file(Specs, Relaxed) of - {E,Reason} when E == error ; E == 'EXIT' -> - {{error,Reason},StartOpts}; - TS -> - SpecStartOpts = get_data_for_node(TS, node()), - - Label = choose_val(StartOpts#opts.label, - SpecStartOpts#opts.label), - - Profile = choose_val(StartOpts#opts.profile, - SpecStartOpts#opts.profile), - - LogDir = choose_val(StartOpts#opts.logdir, - SpecStartOpts#opts.logdir), - - AllLogOpts = merge_vals([StartOpts#opts.logopts, - SpecStartOpts#opts.logopts]), - AllVerbosity = - merge_keyvals([StartOpts#opts.verbosity, - SpecStartOpts#opts.verbosity]), - AllSilentConns = - merge_vals([StartOpts#opts.silent_connections, - SpecStartOpts#opts.silent_connections]), - Cover = - choose_val(StartOpts#opts.cover, - SpecStartOpts#opts.cover), - CoverStop = - choose_val(StartOpts#opts.cover_stop, - SpecStartOpts#opts.cover_stop), - MultTT = - choose_val(StartOpts#opts.multiply_timetraps, - SpecStartOpts#opts.multiply_timetraps), - ScaleTT = - choose_val(StartOpts#opts.scale_timetraps, - SpecStartOpts#opts.scale_timetraps), - - CreatePrivDir = - choose_val(StartOpts#opts.create_priv_dir, - SpecStartOpts#opts.create_priv_dir), - - AllEvHs = - merge_vals([StartOpts#opts.event_handlers, - SpecStartOpts#opts.event_handlers]), - - AllCTHooks = merge_vals( - [StartOpts#opts.ct_hooks, - SpecStartOpts#opts.ct_hooks]), - - EnableBuiltinHooks = - choose_val( - StartOpts#opts.enable_builtin_hooks, - SpecStartOpts#opts.enable_builtin_hooks), - - Stylesheet = - choose_val(StartOpts#opts.stylesheet, - SpecStartOpts#opts.stylesheet), - - AllInclude = merge_vals([StartOpts#opts.include, - SpecStartOpts#opts.include]), - application:set_env(common_test, include, AllInclude), - - AutoCompile = - case choose_val(StartOpts#opts.auto_compile, - SpecStartOpts#opts.auto_compile) of - undefined -> - true; - ACBool -> - application:set_env(common_test, - auto_compile, - ACBool), - ACBool - end, - - BasicHtml = - case choose_val(StartOpts#opts.basic_html, - SpecStartOpts#opts.basic_html) of - undefined -> - false; - BHBool -> - application:set_env(common_test, basic_html, - BHBool), - BHBool - end, - - {TS,StartOpts#opts{label = Label, - profile = Profile, - testspecs = Specs, - cover = Cover, - cover_stop = CoverStop, - logdir = LogDir, - logopts = AllLogOpts, - basic_html = BasicHtml, - verbosity = AllVerbosity, - silent_connections = AllSilentConns, - config = SpecStartOpts#opts.config, - event_handlers = AllEvHs, - ct_hooks = AllCTHooks, - enable_builtin_hooks = - EnableBuiltinHooks, - stylesheet = Stylesheet, - auto_compile = AutoCompile, - include = AllInclude, - multiply_timetraps = MultTT, - scale_timetraps = ScaleTT, - create_priv_dir = CreatePrivDir}} - end; - _ -> - {undefined,StartOpts} - end, - %% read config/userconfig from start flags - InitConfig = ct_config:prepare_config_list(Args), - TheLogDir = which(logdir, Opts#opts.logdir), - case {TestSpec,Terms} of - {_,{error,_}=Error} -> - Error; - {[],_} -> +script_start2(Opts = #opts{vts = undefined, + shell = undefined}, Args) -> + case proplists:get_value(spec, Args) of + Specs when Specs =/= [], Specs =/= undefined -> + Specs1 = get_start_opt(join_specs, [Specs], Specs, Args), + %% using testspec as input for test + Relaxed = get_start_opt(allow_user_terms, true, false, Args), + case catch ct_testspec:collect_tests_from_file(Specs1, Relaxed) of + {E,Reason} when E == error ; E == 'EXIT' -> + {error,Reason}; + TestSpecData -> + execute_all_specs(TestSpecData, Opts, Args, []) + end; + [] -> {error,no_testspec_specified}; - {undefined,_} -> % no testspec used - case check_and_install_configfiles(InitConfig, TheLogDir, Opts) of + _ -> % no testspec used + %% read config/userconfig from start flags + InitConfig = ct_config:prepare_config_list(Args), + TheLogDir = which(logdir, Opts#opts.logdir), + case check_and_install_configfiles(InitConfig, + TheLogDir, + Opts) of ok -> % go on read tests from start flags script_start3(Opts#opts{config=InitConfig, logdir=TheLogDir}, Args); Error -> Error - end; - {_,_} -> % testspec used - %% merge config from start flags with config from testspec - AllConfig = merge_vals([InitConfig, Opts#opts.config]), - case check_and_install_configfiles(AllConfig, TheLogDir, Opts) of - ok -> % read tests from spec - {Run,Skip} = ct_testspec:prepare_tests(Terms, node()), - do_run(Run, Skip, Opts#opts{config=AllConfig, - logdir=TheLogDir}, Args); - Error -> - Error end end; -script_start2(StartOpts, Args) -> +script_start2(Opts, Args) -> %% read config/userconfig from start flags InitConfig = ct_config:prepare_config_list(Args), - LogDir = which(logdir, StartOpts#opts.logdir), - case check_and_install_configfiles(InitConfig, LogDir, StartOpts) of + LogDir = which(logdir, Opts#opts.logdir), + case check_and_install_configfiles(InitConfig, LogDir, Opts) of ok -> % go on read tests from start flags - script_start3(StartOpts#opts{config=InitConfig, - logdir=LogDir}, Args); + script_start3(Opts#opts{config=InitConfig, + logdir=LogDir}, Args); + Error -> + Error + end. + +execute_all_specs([], _, _, Result) -> + Result1 = lists:reverse(Result), + case lists:keysearch('EXIT', 1, Result1) of + {value,{_,_,ExitReason}} -> + exit(ExitReason); + false -> + case lists:keysearch(error, 1, Result1) of + {value,Error} -> + Error; + false -> + lists:foldl(fun({Ok,Fail,{UserSkip,AutoSkip}}, + {Ok1,Fail1,{UserSkip1,AutoSkip1}}) -> + {Ok1+Ok,Fail1+Fail, + {UserSkip1+UserSkip, + AutoSkip1+AutoSkip}} + end, {0,0,{0,0}}, Result1) + end + end; + +execute_all_specs([{Specs,TS} | TSs], Opts, Args, Result) -> + CombinedOpts = combine_test_opts(TS, Specs, Opts), + try execute_one_spec(TS, CombinedOpts, Args) of + ExecResult -> + execute_all_specs(TSs, Opts, Args, [ExecResult|Result]) + catch + _ : ExitReason -> + execute_all_specs(TSs, Opts, Args, + [{'EXIT',self(),ExitReason}|Result]) + end. + +execute_one_spec(TS, Opts, Args) -> + %% read config/userconfig from start flags + InitConfig = ct_config:prepare_config_list(Args), + TheLogDir = which(logdir, Opts#opts.logdir), + %% merge config from start flags with config from testspec + AllConfig = merge_vals([InitConfig, Opts#opts.config]), + case check_and_install_configfiles(AllConfig, TheLogDir, Opts) of + ok -> % read tests from spec + {Run,Skip} = ct_testspec:prepare_tests(TS, node()), + do_run(Run, Skip, Opts#opts{config=AllConfig, + logdir=TheLogDir}, Args); Error -> Error end. +combine_test_opts(TS, Specs, Opts) -> + TSOpts = get_data_for_node(TS, node()), + + Label = choose_val(Opts#opts.label, + TSOpts#opts.label), + + Profile = choose_val(Opts#opts.profile, + TSOpts#opts.profile), + + LogDir = choose_val(Opts#opts.logdir, + TSOpts#opts.logdir), + + AllLogOpts = merge_vals([Opts#opts.logopts, + TSOpts#opts.logopts]), + AllVerbosity = + merge_keyvals([Opts#opts.verbosity, + TSOpts#opts.verbosity]), + AllSilentConns = + merge_vals([Opts#opts.silent_connections, + TSOpts#opts.silent_connections]), + Cover = + choose_val(Opts#opts.cover, + TSOpts#opts.cover), + CoverStop = + choose_val(Opts#opts.cover_stop, + TSOpts#opts.cover_stop), + MultTT = + choose_val(Opts#opts.multiply_timetraps, + TSOpts#opts.multiply_timetraps), + ScaleTT = + choose_val(Opts#opts.scale_timetraps, + TSOpts#opts.scale_timetraps), + + CreatePrivDir = + choose_val(Opts#opts.create_priv_dir, + TSOpts#opts.create_priv_dir), + + AllEvHs = + merge_vals([Opts#opts.event_handlers, + TSOpts#opts.event_handlers]), + + AllCTHooks = merge_vals( + [Opts#opts.ct_hooks, + TSOpts#opts.ct_hooks]), + + EnableBuiltinHooks = + choose_val( + Opts#opts.enable_builtin_hooks, + TSOpts#opts.enable_builtin_hooks), + + Stylesheet = + choose_val(Opts#opts.stylesheet, + TSOpts#opts.stylesheet), + + AllInclude = merge_vals([Opts#opts.include, + TSOpts#opts.include]), + application:set_env(common_test, include, AllInclude), + + AutoCompile = + case choose_val(Opts#opts.auto_compile, + TSOpts#opts.auto_compile) of + undefined -> + true; + ACBool -> + application:set_env(common_test, + auto_compile, + ACBool), + ACBool + end, + + BasicHtml = + case choose_val(Opts#opts.basic_html, + TSOpts#opts.basic_html) of + undefined -> + false; + BHBool -> + application:set_env(common_test, basic_html, + BHBool), + BHBool + end, + + Opts#opts{label = Label, + profile = Profile, + testspecs = Specs, + cover = Cover, + cover_stop = CoverStop, + logdir = which(logdir, LogDir), + logopts = AllLogOpts, + basic_html = BasicHtml, + verbosity = AllVerbosity, + silent_connections = AllSilentConns, + config = TSOpts#opts.config, + event_handlers = AllEvHs, + ct_hooks = AllCTHooks, + enable_builtin_hooks = EnableBuiltinHooks, + stylesheet = Stylesheet, + auto_compile = AutoCompile, + include = AllInclude, + multiply_timetraps = MultTT, + scale_timetraps = ScaleTT, + create_priv_dir = CreatePrivDir}. + check_and_install_configfiles( Configs, LogDir, #opts{ event_handlers = EvHandlers, @@ -562,12 +594,12 @@ check_and_install_configfiles( {error,{cant_load_callback_module,Info}} end. -script_start3(StartOpts, Args) -> - StartOpts1 = get_start_opt(step, - fun(Step) -> - StartOpts#opts{step = Step, - cover = undefined} - end, StartOpts, Args), +script_start3(Opts, Args) -> + Opts1 = get_start_opt(step, + fun(Step) -> + Opts#opts{step = Step, + cover = undefined} + end, Opts, Args), case {proplists:get_value(dir, Args), proplists:get_value(suite, Args), groups_and_cases(proplists:get_value(group, Args), @@ -581,17 +613,17 @@ script_start3(StartOpts, Args) -> {error,no_dir_specified}; {Dirs,undefined,[]} when is_list(Dirs) -> - script_start4(StartOpts#opts{tests = tests(Dirs)}, Args); + script_start4(Opts#opts{tests = tests(Dirs)}, Args); {undefined,Suites,[]} when is_list(Suites) -> Ts = tests([suite_to_test(S) || S <- Suites]), - script_start4(StartOpts1#opts{tests = Ts}, Args); + script_start4(Opts1#opts{tests = Ts}, Args); {undefined,Suite,GsAndCs} when is_list(Suite) -> case [suite_to_test(S) || S <- Suite] of DirMods = [_] -> Ts = tests(DirMods, GsAndCs), - script_start4(StartOpts1#opts{tests = Ts}, Args); + script_start4(Opts1#opts{tests = Ts}, Args); [_,_|_] -> {error,multiple_suites_and_cases}; _ -> @@ -605,10 +637,10 @@ script_start3(StartOpts, Args) -> case [suite_to_test(Dir,S) || S <- Suite] of DirMods when GsAndCs == [] -> Ts = tests(DirMods), - script_start4(StartOpts1#opts{tests = Ts}, Args); + script_start4(Opts1#opts{tests = Ts}, Args); DirMods = [_] when GsAndCs /= [] -> Ts = tests(DirMods, GsAndCs), - script_start4(StartOpts1#opts{tests = Ts}, Args); + script_start4(Opts1#opts{tests = Ts}, Args); [_,_|_] when GsAndCs /= [] -> {error,multiple_suites_and_cases}; _ -> @@ -619,8 +651,8 @@ script_start3(StartOpts, Args) -> {error,incorrect_start_options}; {undefined,undefined,_} -> - if StartOpts#opts.vts ; StartOpts#opts.shell -> - script_start4(StartOpts#opts{tests = []}, Args); + if Opts#opts.vts ; Opts#opts.shell -> + script_start4(Opts#opts{tests = []}, Args); true -> script_usage(), {error,missing_start_options} @@ -750,6 +782,7 @@ script_usage() -> "\n\t[-logopts LogOpt1 LogOpt2 .. LogOptN]" "\n\t[-verbosity GenVLvl | [CategoryVLvl1 .. CategoryVLvlN]]" "\n\t[-allow_user_terms]" + "\n\t[-join_specs]" "\n\t[-silent_connections [ConnType1 ConnType2 .. ConnTypeN]]" "\n\t[-stylesheet CSSFile]" "\n\t[-cover CoverCfgFile]" @@ -848,7 +881,7 @@ run_test1(StartOpts) when is_list(StartOpts) -> undefined -> Tracing = start_trace(StartOpts), {ok,Cwd} = file:get_cwd(), - io:format("~nCommon Test starting (cwd is ~s)~n~n", [Cwd]), + io:format("~nCommon Test starting (cwd is ~ts)~n~n", [Cwd]), Res = case ct_repeat:loop_test(func, StartOpts) of false -> @@ -1044,105 +1077,60 @@ run_test2(StartOpts) -> end. run_spec_file(Relaxed, - Opts = #opts{testspecs = Specs, config = CfgFiles}, + Opts = #opts{testspecs = Specs}, StartOpts) -> Specs1 = case Specs of [X|_] when is_integer(X) -> [Specs]; _ -> Specs end, AbsSpecs = lists:map(fun(SF) -> ?abs(SF) end, Specs1), - log_ts_names(AbsSpecs), - case catch ct_testspec:collect_tests_from_file(AbsSpecs, Relaxed) of + AbsSpecs1 = get_start_opt(join_specs, [AbsSpecs], AbsSpecs, StartOpts), + case catch ct_testspec:collect_tests_from_file(AbsSpecs1, Relaxed) of {Error,CTReason} when Error == error ; Error == 'EXIT' -> exit({error,CTReason}); - TS -> - SpecOpts = get_data_for_node(TS, node()), - Label = choose_val(Opts#opts.label, - SpecOpts#opts.label), - Profile = choose_val(Opts#opts.profile, - SpecOpts#opts.profile), - LogDir = choose_val(Opts#opts.logdir, - SpecOpts#opts.logdir), - AllLogOpts = merge_vals([Opts#opts.logopts, - SpecOpts#opts.logopts]), - Stylesheet = choose_val(Opts#opts.stylesheet, - SpecOpts#opts.stylesheet), - AllVerbosity = merge_keyvals([Opts#opts.verbosity, - SpecOpts#opts.verbosity]), - AllSilentConns = merge_vals([Opts#opts.silent_connections, - SpecOpts#opts.silent_connections]), - AllConfig = merge_vals([CfgFiles, SpecOpts#opts.config]), - Cover = choose_val(Opts#opts.cover, - SpecOpts#opts.cover), - CoverStop = choose_val(Opts#opts.cover_stop, - SpecOpts#opts.cover_stop), - MultTT = choose_val(Opts#opts.multiply_timetraps, - SpecOpts#opts.multiply_timetraps), - ScaleTT = choose_val(Opts#opts.scale_timetraps, - SpecOpts#opts.scale_timetraps), - CreatePrivDir = choose_val(Opts#opts.create_priv_dir, - SpecOpts#opts.create_priv_dir), - AllEvHs = merge_vals([Opts#opts.event_handlers, - SpecOpts#opts.event_handlers]), - AllInclude = merge_vals([Opts#opts.include, - SpecOpts#opts.include]), - AllCTHooks = merge_vals([Opts#opts.ct_hooks, - SpecOpts#opts.ct_hooks]), - EnableBuiltinHooks = choose_val(Opts#opts.enable_builtin_hooks, - SpecOpts#opts.enable_builtin_hooks), - - application:set_env(common_test, include, AllInclude), - - AutoCompile = case choose_val(Opts#opts.auto_compile, - SpecOpts#opts.auto_compile) of - undefined -> - true; - ACBool -> - application:set_env(common_test, auto_compile, - ACBool), - ACBool - end, + TestSpecData -> + run_all_specs(TestSpecData, Opts, StartOpts, []) + end. - BasicHtml = case choose_val(Opts#opts.basic_html, - SpecOpts#opts.basic_html) of - undefined -> - false; - BHBool -> - application:set_env(common_test, basic_html, - BHBool), - BHBool - end, - - Opts1 = Opts#opts{label = Label, - profile = Profile, - cover = Cover, - cover_stop = CoverStop, - logdir = which(logdir, LogDir), - logopts = AllLogOpts, - stylesheet = Stylesheet, - basic_html = BasicHtml, - verbosity = AllVerbosity, - silent_connections = AllSilentConns, - config = AllConfig, - event_handlers = AllEvHs, - auto_compile = AutoCompile, - include = AllInclude, - testspecs = AbsSpecs, - multiply_timetraps = MultTT, - scale_timetraps = ScaleTT, - create_priv_dir = CreatePrivDir, - ct_hooks = AllCTHooks, - enable_builtin_hooks = EnableBuiltinHooks - }, - - case check_and_install_configfiles(AllConfig,Opts1#opts.logdir, - Opts1) of - ok -> - {Run,Skip} = ct_testspec:prepare_tests(TS, node()), - reformat_result(catch do_run(Run, Skip, Opts1, StartOpts)); - {error,GCFReason} -> - exit({error,GCFReason}) +run_all_specs([], _, _, TotResult) -> + TotResult1 = lists:reverse(TotResult), + case lists:keysearch('EXIT', 1, TotResult1) of + {value,{_,_,ExitReason}} -> + exit(ExitReason); + false -> + case lists:keysearch(error, 1, TotResult1) of + {value,Error} -> + Error; + false -> + lists:foldl(fun({Ok,Fail,{UserSkip,AutoSkip}}, + {Ok1,Fail1,{UserSkip1,AutoSkip1}}) -> + {Ok1+Ok,Fail1+Fail, + {UserSkip1+UserSkip, + AutoSkip1+AutoSkip}} + end, {0,0,{0,0}}, TotResult1) end + end; + +run_all_specs([{Specs,TS} | TSs], Opts, StartOpts, TotResult) -> + log_ts_names(Specs), + Combined = #opts{config = TSConfig} = combine_test_opts(TS, Specs, Opts), + AllConfig = merge_vals([Opts#opts.config, TSConfig]), + try run_one_spec(TS, Combined#opts{config = AllConfig}, StartOpts) of + Result -> + run_all_specs(TSs, Opts, StartOpts, [Result | TotResult]) + catch + _ : Reason -> + run_all_specs(TSs, Opts, StartOpts, [{error,Reason} | TotResult]) + end. + +run_one_spec(TS, CombinedOpts, StartOpts) -> + #opts{logdir = Logdir, config = Config} = CombinedOpts, + case check_and_install_configfiles(Config, Logdir, CombinedOpts) of + ok -> + {Run,Skip} = ct_testspec:prepare_tests(TS, node()), + reformat_result(catch do_run(Run, Skip, CombinedOpts, StartOpts)); + Error -> + Error end. run_prepared(Run, Skip, Opts = #opts{logdir = LogDir, @@ -1332,7 +1320,7 @@ run_testspec(TestSpec) -> run_testspec1(TestSpec) -> {ok,Cwd} = file:get_cwd(), - io:format("~nCommon Test starting (cwd is ~s)~n~n", [Cwd]), + io:format("~nCommon Test starting (cwd is ~ts)~n~n", [Cwd]), case catch run_testspec2(TestSpec) of {'EXIT',Reason} -> file:set_cwd(Cwd), @@ -1469,7 +1457,7 @@ refresh_logs(LogDir) -> {error,{all_runs_index,ARReason}}; _ -> file:set_cwd(Cwd), - io:format("Logs in ~s refreshed!~n",[LogDir]), + io:format("Logs in ~ts refreshed!~n",[LogDir]), ok end end @@ -1792,7 +1780,7 @@ possibly_spawn(true, Tests, Skip, Opts) -> end, unlink(CTUtilSrv), SupPid = spawn(Supervisor), - io:format(user, "~nTest control handed over to process ~p~n~n", + io:format(user, "~nTest control handed over to process ~w~n~n", [SupPid]), SupPid. @@ -1880,7 +1868,7 @@ verify_suites(TestSuites) -> Suite)), io:format(user, "Suite ~w not found" - "in directory ~s~n", + "in directory ~ts~n", [Suite,TestDir]), {Found,[{DS,[Name]}|NotFound]} end @@ -1895,7 +1883,7 @@ verify_suites(TestSuites) -> ActualDir = filename:dirname(SuiteFile), {[{ActualDir,Suite}|Found],NotFound}; false -> - io:format(user, "Directory ~s is " + io:format(user, "Directory ~ts is " "invalid~n", [Dir]), Name = filename:join(Dir, atom_to_list(Suite)), {Found,[{DS,[Name]}|NotFound]} @@ -2135,7 +2123,7 @@ do_run_test(Tests, Skip, Opts) -> cross = CovCross, src = _CovSrc}} -> ct_logs:log("COVER INFO", - "Using cover specification file: ~s~n" + "Using cover specification file: ~ts~n" "App: ~w~n" "Cross cover: ~w~n" "Including ~w modules~n" @@ -2150,7 +2138,7 @@ do_run_test(Tests, Skip, Opts) -> DelResult = file:delete(CovExport), ct_logs:log("COVER INFO", "Warning! " - "Export file ~s already exists. " + "Export file ~ts already exists. " "Deleting with result: ~p", [CovExport,DelResult]); false -> @@ -2632,7 +2620,7 @@ log_ts_names(Specs) -> List = lists:map(fun(Name) -> Name ++ " " end, Specs), - ct_logs:log("Test Specification file(s)", "~s", + ct_logs:log("Test Specification file(s)", "~ts", [lists:flatten(List)]). merge_arguments(Args) -> @@ -2743,10 +2731,10 @@ event_handler_args2opts(Default, Args) -> event_handler_init_args2opts(EHs) end. event_handler_init_args2opts([EH, Arg, "and" | EHs]) -> - [{list_to_atom(EH),lists:flatten(io_lib:format("~s",[Arg]))} | + [{list_to_atom(EH),lists:flatten(io_lib:format("~ts",[Arg]))} | event_handler_init_args2opts(EHs)]; event_handler_init_args2opts([EH, Arg]) -> - [{list_to_atom(EH),lists:flatten(io_lib:format("~s",[Arg]))}]; + [{list_to_atom(EH),lists:flatten(io_lib:format("~ts",[Arg]))}]; event_handler_init_args2opts([]) -> []. @@ -2896,6 +2884,10 @@ opts2args(EnvStartOpts) -> [{allow_user_terms,[]}]; ({allow_user_terms,false}) -> []; + ({join_specs,true}) -> + [{join_specs,[]}]; + ({join_specs,false}) -> + []; ({auto_compile,false}) -> [{no_auto_compile,[]}]; ({auto_compile,true}) -> @@ -3064,7 +3056,7 @@ start_trace(Args) -> false end; {_,Error} -> - io:format("Warning! Tracing not started. Reason: ~s~n~n", + io:format("Warning! Tracing not started. Reason: ~ts~n~n", [file:format_error(Error)]), false end; diff --git a/lib/common_test/src/ct_telnet.erl b/lib/common_test/src/ct_telnet.erl index b13c050e32..02186864a5 100644 --- a/lib/common_test/src/ct_telnet.erl +++ b/lib/common_test/src/ct_telnet.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2012. All Rights Reserved. +%% Copyright Ericsson AB 2003-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -201,7 +201,7 @@ open(KeyOrName,ConnType,TargetMod,Extra) -> close(Connection) -> case get_handle(Connection) of {ok,Pid} -> - log("ct_telnet:close","Handle: ~p",[Pid]), + log("ct_telnet:close","Handle: ~w",[Pid]), case ct_gen_conn:stop(Pid) of {error,{process_down,Pid,noproc}} -> {error,already_closed}; @@ -558,7 +558,7 @@ reconnect(Ip,Port,N,State=#state{target_mod=TargetMod, Error when N==0 -> Error; _Error -> - log("Reconnect failed!","Retries left: ~p",[N]), + log("Reconnect failed!","Retries left: ~w",[N]), timer:sleep(ReconnInt), reconnect(Ip,Port,N-1,State) end. @@ -567,7 +567,7 @@ reconnect(Ip,Port,N,State=#state{target_mod=TargetMod, %% @hidden terminate(TelnPid,State) -> log(heading(terminate,State#state.name), - "Closing telnet connection.\nId: ~p", + "Closing telnet connection.\nId: ~w", [TelnPid]), ct_telnet_client:close(TelnPid). @@ -899,7 +899,7 @@ one_expect(Data,Pattern,EO) -> [Prompt] when Prompt==prompt; Prompt=={prompt,PromptType} -> %% Only searching for prompt log_lines(UptoPrompt), - try_cont_log("<b>PROMPT:</b> ~s", [PromptType]), + try_cont_log("<b>PROMPT:</b> ~ts", [PromptType]), {match,{prompt,PromptType},Rest}; [{prompt,_OtherPromptType}] -> %% Only searching for one specific prompt, not thisone @@ -969,7 +969,7 @@ seq_expect1(Data,[prompt|Patterns],Acc,Rest,EO) -> {continue,[prompt|Patterns],Acc,LastLine}; PromptType -> log_lines(Data), - try_cont_log("<b>PROMPT:</b> ~s", [PromptType]), + try_cont_log("<b>PROMPT:</b> ~ts", [PromptType]), seq_expect(Rest,Patterns,[{prompt,PromptType}|Acc],EO) end; seq_expect1(Data,[{prompt,PromptType}|Patterns],Acc,Rest,EO) -> @@ -980,7 +980,7 @@ seq_expect1(Data,[{prompt,PromptType}|Patterns],Acc,Rest,EO) -> {continue,[{prompt,PromptType}|Patterns],Acc,LastLine}; PromptType -> log_lines(Data), - try_cont_log("<b>PROMPT:</b> ~s", [PromptType]), + try_cont_log("<b>PROMPT:</b> ~ts", [PromptType]), seq_expect(Rest,Patterns,[{prompt,PromptType}|Acc],EO); _OtherPromptType -> log_lines(Data), @@ -1032,13 +1032,13 @@ match_line(Line,Patterns,FoundPrompt,EO) -> match_line(Line,[prompt|Patterns],false,EO,RetTag) -> match_line(Line,Patterns,false,EO,RetTag); match_line(Line,[prompt|_Patterns],FoundPrompt,_EO,RetTag) -> - try_cont_log(" ~s", [Line]), - try_cont_log("<b>PROMPT:</b> ~s", [FoundPrompt]), + try_cont_log(" ~ts", [Line]), + try_cont_log("<b>PROMPT:</b> ~ts", [FoundPrompt]), {RetTag,{prompt,FoundPrompt}}; match_line(Line,[{prompt,PromptType}|_Patterns],FoundPrompt,_EO,RetTag) when PromptType==FoundPrompt -> - try_cont_log(" ~s", [Line]), - try_cont_log("<b>PROMPT:</b> ~s", [FoundPrompt]), + try_cont_log(" ~ts", [Line]), + try_cont_log("<b>PROMPT:</b> ~ts", [FoundPrompt]), {RetTag,{prompt,FoundPrompt}}; match_line(Line,[{prompt,PromptType}|Patterns],FoundPrompt,EO,RetTag) when PromptType=/=FoundPrompt -> @@ -1048,7 +1048,7 @@ match_line(Line,[{Tag,Pattern}|Patterns],FoundPrompt,EO,RetTag) -> nomatch -> match_line(Line,Patterns,FoundPrompt,EO,RetTag); {match,Match} -> - try_cont_log("<b>MATCH:</b> ~s", [Line]), + try_cont_log("<b>MATCH:</b> ~ts", [Line]), {RetTag,{Tag,Match}} end; match_line(Line,[Pattern|Patterns],FoundPrompt,EO,RetTag) -> @@ -1056,13 +1056,13 @@ match_line(Line,[Pattern|Patterns],FoundPrompt,EO,RetTag) -> nomatch -> match_line(Line,Patterns,FoundPrompt,EO,RetTag); {match,Match} -> - try_cont_log("<b>MATCH:</b> ~s", [Line]), + try_cont_log("<b>MATCH:</b> ~ts", [Line]), {RetTag,Match} end; match_line(Line,[],FoundPrompt,EO,match) -> match_line(Line,EO#eo.haltpatterns,FoundPrompt,EO,halt); match_line(Line,[],_FoundPrompt,_EO,halt) -> - try_cont_log(" ~s", [Line]), + try_cont_log(" ~ts", [Line]), nomatch. one_line([$\n|Rest],Line) -> @@ -1086,7 +1086,7 @@ log_lines(String) -> [] -> ok; LastLine -> - try_cont_log(" ~s", [LastLine]) + try_cont_log(" ~ts", [LastLine]) end. log_lines_not_last(String) -> @@ -1094,7 +1094,7 @@ log_lines_not_last(String) -> {[],LastLine} -> LastLine; {String1,LastLine} -> - try_cont_log("~s",[String1]), + try_cont_log("~ts",[String1]), LastLine end. diff --git a/lib/common_test/src/ct_telnet_client.erl b/lib/common_test/src/ct_telnet_client.erl index d703b39ac5..7329498ed6 100644 --- a/lib/common_test/src/ct_telnet_client.erl +++ b/lib/common_test/src/ct_telnet_client.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% Copyright Ericsson AB 2003-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -308,7 +308,7 @@ cmd_dbg(_Cmd) -> [Opt] -> Opt; _ -> Opts end, - io:format("~s(~w): ~w\n", [CtrlStr,Ctrl,Opts1]); + io:format("~ts(~w): ~w\n", [CtrlStr,Ctrl,Opts1]); Any -> io:format("Unexpected in cmd_dbg:~n~w~n",[Any]) end. diff --git a/lib/common_test/src/ct_testspec.erl b/lib/common_test/src/ct_testspec.erl index 202d8f9373..e341391a91 100644 --- a/lib/common_test/src/ct_testspec.erl +++ b/lib/common_test/src/ct_testspec.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2012. All Rights Reserved. +%% Copyright Ericsson AB 2006-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -28,7 +28,6 @@ collect_tests_from_file/2, collect_tests_from_file/3]). -include("ct_util.hrl"). - -define(testspec_fields, record_info(fields, testspec)). %%%------------------------------------------------------------------ @@ -93,7 +92,7 @@ prepare_tests(TestSpec) when is_record(TestSpec,testspec) -> %% run_per_node/2 takes the Run list as input and returns a list %% of {Node,RunPerNode,[]} tuples where the tests have been sorted %% on a per node basis. -run_per_node([{{Node,Dir},Test}|Ts],Result, MergeTests) -> +run_per_node([{{Node,Dir},Test}|Ts],Result,MergeTests) -> {value,{Node,{Run,Skip}}} = lists:keysearch(Node,1,Result), Run1 = case MergeTests of false -> @@ -190,7 +189,7 @@ prepare_suites(_Node,_Dir,[],Run,Skip) -> prepare_cases(Node,Dir,Suite,Cases) -> case get_skipped_cases(Node,Dir,Suite,Cases) of - SkipAll=[{{Node,Dir},{Suite,_Cmt}}] -> % all cases to be skipped + SkipAll=[{{Node,Dir},{Suite,_Cmt}}] -> % all cases to be skipped %% note: this adds an 'all' test even if only skip is specified {[{{Node,Dir},{Suite,all}}],SkipAll}; Skipped -> @@ -241,34 +240,147 @@ get_skipped_cases1(_,_,_,[]) -> %%% collect_tests_from_file reads a testspec file and returns a record %%% containing the data found. -collect_tests_from_file(Specs, Relaxed) -> +collect_tests_from_file(Specs,Relaxed) -> collect_tests_from_file(Specs,[node()],Relaxed). collect_tests_from_file(Specs,Nodes,Relaxed) when is_list(Nodes) -> NodeRefs = lists:map(fun(N) -> {undefined,N} end, Nodes), - catch collect_tests_from_file1(Specs,#testspec{nodes=NodeRefs},Relaxed). + %% [Spec1,Spec2,...] means create one testpec record per Spec file + %% [[Spec1,Spec2,...]] means merge all specs into one testspec record + {Join,Specs1} = if is_list(hd(hd(Specs))) -> {true,hd(Specs)}; + true -> {false,Specs} + end, + Specs2 = [filename:absname(S) || S <- Specs1], + TS0 = #testspec{nodes=NodeRefs}, + + try create_specs(Specs2,TS0,Relaxed,Join) of + {{[],_},SeparateTestSpecs} -> + filter_and_convert(SeparateTestSpecs); + {{_,#testspec{tests=[]}},SeparateTestSpecs} -> + filter_and_convert(SeparateTestSpecs); + {Joined,SeparateTestSpecs} -> + [filter_and_convert(Joined) | + filter_and_convert(SeparateTestSpecs)] + catch + _:Error -> + Error + end. + +filter_and_convert(Joined) when is_tuple(Joined) -> + hd(filter_and_convert([Joined])); +filter_and_convert([{_,#testspec{tests=[]}}|TSs]) -> + filter_and_convert(TSs); +filter_and_convert([{[{SpecFile,MergeTests}|SMs],TestSpec}|TSs]) -> + #testspec{config = CfgFiles} = TestSpec, + TestSpec1 = TestSpec#testspec{config = delete_dups(CfgFiles), + merge_tests = MergeTests}, + %% set the merge_tests value for the testspec to the value + %% of the first test spec in the set + [{[SpecFile | [SF || {SF,_} <- SMs]], TestSpec1} | filter_and_convert(TSs)]; +filter_and_convert([]) -> + []. + +delete_dups(Elems) -> + delete_dups1(lists:reverse(Elems),[]). + +delete_dups1([E|Es],Keep) -> + case lists:member(E,Es) of + true -> + delete_dups1(Es,Keep); + false -> + delete_dups1(Es,[E|Keep]) + end; +delete_dups1([],Keep) -> + Keep. + +create_specs(Specs,TestSpec,Relaxed,Join) -> + SpecsTree = create_spec_tree(Specs,TestSpec,Join,[]), + create_specs(SpecsTree,TestSpec,Relaxed). -collect_tests_from_file1([Spec|Specs],TestSpec,Relaxed) -> +create_spec_tree([Spec|Specs],TS,JoinWithNext,Known) -> SpecDir = filename:dirname(filename:absname(Spec)), - case file:consult(Spec) of - {ok,Terms} -> - case collect_tests(Terms, - TestSpec#testspec{spec_dir=SpecDir}, - Relaxed) of - TS = #testspec{tests=Tests, logdir=LogDirs} when Specs == [] -> - LogDirs1 = lists:delete(".",LogDirs) ++ ["."], - TS#testspec{tests=lists:flatten(Tests), logdir=LogDirs1}; - TS = #testspec{alias = As, nodes = Ns} -> - TS1 = TS#testspec{alias = lists:reverse(As), - nodes = lists:reverse(Ns)}, - collect_tests_from_file1(Specs,TS1,Relaxed) - end; - {error,Reason} -> - ReasonStr = - lists:flatten(io_lib:format("~s", - [file:format_error(Reason)])), - throw({error,{Spec,ReasonStr}}) - end. + TS1 = TS#testspec{spec_dir=SpecDir}, + SpecAbsName = get_absfile(Spec,TS1), + case lists:member(SpecAbsName,Known) of + true -> + throw({error,{cyclic_reference,SpecAbsName}}); + false -> + case file:consult(SpecAbsName) of + {ok,Terms} -> + Terms1 = replace_names(Terms), + {InclJoin,InclSep} = get_included_specs(Terms1,TS1), + {SpecAbsName,Terms1, + create_spec_tree(InclJoin,TS,true,[SpecAbsName|Known]), + create_spec_tree(InclSep,TS,false,[SpecAbsName|Known]), + JoinWithNext, + create_spec_tree(Specs,TS,JoinWithNext,Known)}; + {error,Reason} -> + ReasonStr = + lists:flatten(io_lib:format("~s", + [file:format_error(Reason)])), + throw({error,{SpecAbsName,ReasonStr}}) + end + end; +create_spec_tree([],_TS,_JoinWithNext,_Known) -> + []. + +create_specs({Spec,Terms,InclJoin,InclSep,JoinWithNext,NextSpec}, + TestSpec,Relaxed) -> + SpecDir = filename:dirname(filename:absname(Spec)), + TestSpec1 = create_spec(Terms,TestSpec#testspec{spec_dir=SpecDir}, + JoinWithNext,Relaxed), + + {{JoinSpecs1,JoinTS1},Separate1} = create_specs(InclJoin,TestSpec1,Relaxed), + + {{JoinSpecs2,JoinTS2},Separate2} = + case JoinWithNext of + true -> + create_specs(NextSpec,JoinTS1,Relaxed); + false -> + {{[],JoinTS1},[]} + end, + {SepJoinSpecs,Separate3} = create_specs(InclSep,TestSpec,Relaxed), + {SepJoinSpecs1,Separate4} = + case JoinWithNext of + true -> + {{[],TestSpec},[]}; + false -> + create_specs(NextSpec,TestSpec,Relaxed) + end, + + SpecInfo = {Spec,TestSpec1#testspec.merge_tests}, + AllSeparate = + [TSData || TSData = {Ss,_TS} <- Separate3++Separate1++ + [SepJoinSpecs]++Separate2++ + [SepJoinSpecs1]++Separate4, + Ss /= []], + + case {JoinWithNext,JoinSpecs1} of + {true,_} -> + {{[SpecInfo|(JoinSpecs1++JoinSpecs2)],JoinTS2}, + AllSeparate}; + {false,[]} -> + {{[],TestSpec}, + [{[SpecInfo],TestSpec1}|AllSeparate]}; + {false,_} -> + {{[SpecInfo|(JoinSpecs1++JoinSpecs2)],JoinTS2}, + AllSeparate} + end; +create_specs([],TestSpec,_Relaxed) -> + {{[],TestSpec},[]}. + +create_spec(Terms,TestSpec,JoinedByPrev,Relaxed) -> + %% it's the "includer" that decides the value of merge_tests + Terms1 = if not JoinedByPrev -> + [{set_merge_tests,true}|Terms]; + true -> + [{set_merge_tests,false}|Terms] + end, + TS = #testspec{tests=Tests, logdir=LogDirs} = + collect_tests({false,Terms1},TestSpec,Relaxed), + LogDirs1 = lists:delete(".",LogDirs) ++ ["."], + TS#testspec{tests=lists:flatten(Tests), + logdir=LogDirs1}. collect_tests_from_list(Terms,Relaxed) -> collect_tests_from_list(Terms,[node()],Relaxed). @@ -276,8 +388,8 @@ collect_tests_from_list(Terms,Relaxed) -> collect_tests_from_list(Terms,Nodes,Relaxed) when is_list(Nodes) -> {ok,Cwd} = file:get_cwd(), NodeRefs = lists:map(fun(N) -> {undefined,N} end, Nodes), - case catch collect_tests(Terms,#testspec{nodes=NodeRefs, - spec_dir=Cwd}, + case catch collect_tests({true,Terms},#testspec{nodes=NodeRefs, + spec_dir=Cwd}, Relaxed) of E = {error,_} -> E; @@ -287,13 +399,28 @@ collect_tests_from_list(Terms,Nodes,Relaxed) when is_list(Nodes) -> TS#testspec{tests=lists:flatten(Tests), logdir=LogDirs1} end. -collect_tests(Terms,TestSpec,Relaxed) -> +collect_tests({Replace,Terms},TestSpec=#testspec{alias=As,nodes=Ns},Relaxed) -> put(relaxed,Relaxed), - Terms1 = replace_names(Terms), - TestSpec1 = get_global(Terms1,TestSpec), - TestSpec2 = get_all_nodes(Terms1,TestSpec1), - {Terms2, TestSpec3} = filter_init_terms(Terms1, [], TestSpec2), - add_tests(Terms2,TestSpec3). + Terms1 = if Replace -> replace_names(Terms); + true -> Terms + end, + {MergeTestsDef,Terms2} = + case proplists:get_value(set_merge_tests,Terms1,true) of + false -> + %% disable merge_tests + {TestSpec#testspec.merge_tests, + proplists:delete(merge_tests,Terms1)}; + true -> + {true,Terms1} + end, + %% reverse nodes and aliases initially to get the order of them right + %% in case this spec is being joined with a previous one + TestSpec1 = get_global(Terms2,TestSpec#testspec{alias = lists:reverse(As), + nodes = lists:reverse(Ns), + merge_tests = MergeTestsDef}), + TestSpec2 = get_all_nodes(Terms2,TestSpec1), + {Terms3, TestSpec3} = filter_init_terms(Terms2, [], TestSpec2), + add_tests(Terms3,TestSpec3). %% replace names (atoms) in the testspec matching those in 'define' terms by %% searching recursively through tuples and lists @@ -420,9 +547,30 @@ replace_names_in_node1(NodeStr,Defs=[{Name,Replacement}|Ds]) -> replace_names_in_node1(NodeStr,[]) -> NodeStr. +%% look for other specification files, either to join with the +%% current spec, or execute as separate test runs +get_included_specs(Terms,TestSpec) -> + get_included_specs(Terms,TestSpec,[],[]). + +get_included_specs([{specs,How,SpecOrSpecs}|Ts],TestSpec,Join,Sep) -> + Specs = case SpecOrSpecs of + [File|_] when is_list(File) -> + [get_absfile(Spec,TestSpec) || Spec <- SpecOrSpecs]; + [Ch|_] when is_integer(Ch) -> + [get_absfile(SpecOrSpecs,TestSpec)] + end, + if How == join -> + get_included_specs(Ts,TestSpec,Join++Specs,Sep); + true -> + get_included_specs(Ts,TestSpec,Join,Sep++Specs) + end; +get_included_specs([_|Ts],TestSpec,Join,Sep) -> + get_included_specs(Ts,TestSpec,Join,Sep); +get_included_specs([],_,Join,Sep) -> + {Join,Sep}. %% global terms that will be used for analysing all other terms in the spec -get_global([{merge_tests,Bool} | Ts], Spec) -> +get_global([{merge_tests,Bool}|Ts],Spec) -> get_global(Ts,Spec#testspec{merge_tests=Bool}); %% the 'define' term replaces the 'alias' and 'node' terms, but we need to keep @@ -588,7 +736,7 @@ add_option({Key,Value},Node,List,WarnIfExists) when is_list(Value) -> NewOption = case lists:keyfind(Key,1,OldOptions) of {Key,OldOption} when WarnIfExists,OldOption/=[]-> io:format("There is an option ~w=~w already " - "defined for node ~p, skipping new ~w~n", + "defined for node ~w, skipping new ~w~n", [Key,OldOption,Node,Value]), OldOption; {Key,OldOption}-> @@ -637,7 +785,7 @@ add_tests([{suites,all_nodes,Dir,Ss}|Ts],Spec) -> add_tests([{suites,Dir,Ss}|Ts],Spec) -> add_tests([{suites,all_nodes,Dir,Ss}|Ts],Spec); add_tests([{suites,Nodes,Dir,Ss}|Ts],Spec) when is_list(Nodes) -> - Ts1 = separate(Nodes,suites,[Dir,Ss],Ts,Spec#testspec.nodes), + Ts1 = per_node(Nodes,suites,[Dir,Ss],Ts,Spec#testspec.nodes), add_tests(Ts1,Spec); add_tests([{suites,Node,Dir,Ss}|Ts],Spec) -> Tests = Spec#testspec.tests, @@ -660,11 +808,11 @@ add_tests([{groups,Dir,Suite,Gs}|Ts],Spec) -> add_tests([{groups,Dir,Suite,Gs,{cases,TCs}}|Ts],Spec) -> add_tests([{groups,all_nodes,Dir,Suite,Gs,{cases,TCs}}|Ts],Spec); add_tests([{groups,Nodes,Dir,Suite,Gs}|Ts],Spec) when is_list(Nodes) -> - Ts1 = separate(Nodes,groups,[Dir,Suite,Gs],Ts,Spec#testspec.nodes), + Ts1 = per_node(Nodes,groups,[Dir,Suite,Gs],Ts,Spec#testspec.nodes), add_tests(Ts1,Spec); add_tests([{groups,Nodes,Dir,Suite,Gs,{cases,TCs}}|Ts], Spec) when is_list(Nodes) -> - Ts1 = separate(Nodes,groups,[Dir,Suite,Gs,{cases,TCs}],Ts, + Ts1 = per_node(Nodes,groups,[Dir,Suite,Gs,{cases,TCs}],Ts, Spec#testspec.nodes), add_tests(Ts1,Spec); add_tests([{groups,Node,Dir,Suite,Gs}|Ts],Spec) -> @@ -688,7 +836,7 @@ add_tests([{cases,all_nodes,Dir,Suite,Cs}|Ts],Spec) -> add_tests([{cases,Dir,Suite,Cs}|Ts],Spec) -> add_tests([{cases,all_nodes,Dir,Suite,Cs}|Ts],Spec); add_tests([{cases,Nodes,Dir,Suite,Cs}|Ts],Spec) when is_list(Nodes) -> - Ts1 = separate(Nodes,cases,[Dir,Suite,Cs],Ts,Spec#testspec.nodes), + Ts1 = per_node(Nodes,cases,[Dir,Suite,Cs],Ts,Spec#testspec.nodes), add_tests(Ts1,Spec); add_tests([{cases,Node,Dir,Suite,Cs}|Ts],Spec) -> Tests = Spec#testspec.tests, @@ -703,7 +851,7 @@ add_tests([{skip_suites,all_nodes,Dir,Ss,Cmt}|Ts],Spec) -> add_tests([{skip_suites,Dir,Ss,Cmt}|Ts],Spec) -> add_tests([{skip_suites,all_nodes,Dir,Ss,Cmt}|Ts],Spec); add_tests([{skip_suites,Nodes,Dir,Ss,Cmt}|Ts],Spec) when is_list(Nodes) -> - Ts1 = separate(Nodes,skip_suites,[Dir,Ss,Cmt],Ts,Spec#testspec.nodes), + Ts1 = per_node(Nodes,skip_suites,[Dir,Ss,Cmt],Ts,Spec#testspec.nodes), add_tests(Ts1,Spec); add_tests([{skip_suites,Node,Dir,Ss,Cmt}|Ts],Spec) -> Tests = Spec#testspec.tests, @@ -724,11 +872,11 @@ add_tests([{skip_groups,Dir,Suite,Gs,Cmt}|Ts],Spec) -> add_tests([{skip_groups,Dir,Suite,Gs,{cases,TCs},Cmt}|Ts],Spec) -> add_tests([{skip_groups,all_nodes,Dir,Suite,Gs,{cases,TCs},Cmt}|Ts],Spec); add_tests([{skip_groups,Nodes,Dir,Suite,Gs,Cmt}|Ts],Spec) when is_list(Nodes) -> - Ts1 = separate(Nodes,skip_groups,[Dir,Suite,Gs,Cmt],Ts,Spec#testspec.nodes), + Ts1 = per_node(Nodes,skip_groups,[Dir,Suite,Gs,Cmt],Ts,Spec#testspec.nodes), add_tests(Ts1,Spec); add_tests([{skip_groups,Nodes,Dir,Suite,Gs,{cases,TCs},Cmt}|Ts], Spec) when is_list(Nodes) -> - Ts1 = separate(Nodes,skip_groups,[Dir,Suite,Gs,{cases,TCs},Cmt],Ts, + Ts1 = per_node(Nodes,skip_groups,[Dir,Suite,Gs,{cases,TCs},Cmt],Ts, Spec#testspec.nodes), add_tests(Ts1,Spec); add_tests([{skip_groups,Node,Dir,Suite,Gs,Cmt}|Ts],Spec) -> @@ -752,7 +900,7 @@ add_tests([{skip_cases,all_nodes,Dir,Suite,Cs,Cmt}|Ts],Spec) -> add_tests([{skip_cases,Dir,Suite,Cs,Cmt}|Ts],Spec) -> add_tests([{skip_cases,all_nodes,Dir,Suite,Cs,Cmt}|Ts],Spec); add_tests([{skip_cases,Nodes,Dir,Suite,Cs,Cmt}|Ts],Spec) when is_list(Nodes) -> - Ts1 = separate(Nodes,skip_cases,[Dir,Suite,Cs,Cmt],Ts,Spec#testspec.nodes), + Ts1 = per_node(Nodes,skip_cases,[Dir,Suite,Cs,Cmt],Ts,Spec#testspec.nodes), add_tests(Ts1,Spec); add_tests([{skip_cases,Node,Dir,Suite,Cs,Cmt}|Ts],Spec) -> Tests = Spec#testspec.tests, @@ -783,6 +931,9 @@ add_tests([{release_shell,Bool}|Ts],Spec) -> add_tests(Ts, Spec#testspec{release_shell = Bool}); %% --- handled/errors --- +add_tests([{set_merge_tests,_}|Ts],Spec) -> % internal + add_tests(Ts,Spec); + add_tests([{define,_,_}|Ts],Spec) -> % handled add_tests(Ts,Spec); @@ -792,7 +943,10 @@ add_tests([{alias,_,_}|Ts],Spec) -> % handled add_tests([{node,_,_}|Ts],Spec) -> % handled add_tests(Ts,Spec); -add_tests([{merge_tests, _} | Ts], Spec) -> % handled +add_tests([{merge_tests,_} | Ts], Spec) -> % handled + add_tests(Ts,Spec); + +add_tests([{specs,_,_} | Ts], Spec) -> % handled add_tests(Ts,Spec); %% -------------------------------------------------- @@ -821,7 +975,7 @@ add_tests([{Tag,NodesOrOther,Data}|Ts],Spec) when is_list(NodesOrOther) -> case lists:all(fun(Test) -> is_node(Test,Spec#testspec.nodes) end, NodesOrOther) of true -> - Ts1 = separate(NodesOrOther,Tag,[Data],Ts,Spec#testspec.nodes), + Ts1 = per_node(NodesOrOther,Tag,[Data],Ts,Spec#testspec.nodes), add_tests(Ts1,Spec); false -> add_tests([{Tag,all_nodes,{NodesOrOther,Data}}|Ts],Spec) @@ -866,7 +1020,18 @@ add_tests([],Spec) -> % done %% check if it's a CT term that has bad format or if the user seems to %% have added something of his/her own, which we'll let pass if relaxed %% mode is enabled. -check_term(Term) -> +check_term(Atom) when is_atom(Atom) -> + Valid = valid_terms(), + case lists:member(Atom,Valid) of + true -> + valid; + false -> % ignore + case get(relaxed) of + true -> invalid; + false -> throw({error,{undefined_term_in_spec,Atom}}) + end + end; +check_term(Term) when is_tuple(Term) -> Size = size(Term), [Name|_] = tuple_to_list(Term), Valid = valid_terms(), @@ -894,7 +1059,9 @@ check_term(Term) -> throw({error,{undefined_term_in_spec,Term}}) end end - end. + end; +check_term(Other) -> + throw({error,{undefined_term_in_spec,Other}}). %% specific data handling before saving in testspec record, e.g. %% converting relative paths to absolute for directories and files @@ -977,12 +1144,12 @@ update_recorded(Tag,Node,Spec) -> end. %% create one test term per node -separate(Nodes,Tag,Data,Tests,Refs) -> - Separated = separate(Nodes,Tag,Data,Refs), +per_node(Nodes,Tag,Data,Tests,Refs) -> + Separated = per_node(Nodes,Tag,Data,Refs), Separated ++ Tests. -separate([N|Ns],Tag,Data,Refs) -> - [list_to_tuple([Tag,ref2node(N,Refs)|Data])|separate(Ns,Tag,Data,Refs)]; -separate([],_,_,_) -> +per_node([N|Ns],Tag,Data,Refs) -> + [list_to_tuple([Tag,ref2node(N,Refs)|Data])|per_node(Ns,Tag,Data,Refs)]; +per_node([],_,_,_) -> []. %% read the value for FieldName in record Rec#testspec @@ -1039,14 +1206,21 @@ insert_groups(Node,Dir,Suite,Groups,Cases,Tests,true) when {[Gr],Cases}; true -> {Gr,Cases} end || Gr <- Groups], - case lists:keysearch({Node,Dir},1,Tests) of - {value,{{Node,Dir},[{all,_}]}} -> - Tests; - {value,{{Node,Dir},Suites0}} -> - Suites1 = insert_groups1(Suite,Groups1,Suites0), - insert_in_order({{Node,Dir},Suites1},Tests); - false -> - insert_in_order({{Node,Dir},[{Suite,Groups1}]},Tests) + {Tests1,Done} = + lists:foldr(fun(All={{N,D},[{all,_}]},{Replaced,_}) when N == Node, + D == Dir -> + {[All|Replaced],true}; + ({{N,D},Suites0},{Replaced,_}) when N == Node, + D == Dir -> + Suites1 = insert_groups1(Suite,Groups1,Suites0), + {[{{N,D},Suites1}|Replaced],true}; + (T,{Replaced,Match}) -> + {[T|Replaced],Match} + end, {[],false}, Tests), + if not Done -> + Tests ++ [{{Node,Dir},[{Suite,Groups1}]}]; + true -> + Tests1 end; insert_groups(Node,Dir,Suite,Groups,Case,Tests, MergeTests) when is_atom(Case) -> @@ -1084,14 +1258,21 @@ insert_groups2([],GrAndCases) -> insert_cases(Node,Dir,Suite,Cases,Tests,false) when is_list(Cases) -> append({{Node,Dir},[{Suite,Cases}]},Tests); insert_cases(Node,Dir,Suite,Cases,Tests,true) when is_list(Cases) -> - case lists:keysearch({Node,Dir},1,Tests) of - {value,{{Node,Dir},[{all,_}]}} -> - Tests; - {value,{{Node,Dir},Suites0}} -> - Suites1 = insert_cases1(Suite,Cases,Suites0), - insert_in_order({{Node,Dir},Suites1},Tests); - false -> - insert_in_order({{Node,Dir},[{Suite,Cases}]},Tests) + {Tests1,Done} = + lists:foldr(fun(All={{N,D},[{all,_}]},{Replaced,_}) when N == Node, + D == Dir -> + {[All|Replaced],true}; + ({{N,D},Suites0},{Replaced,_}) when N == Node, + D == Dir -> + Suites1 = insert_cases1(Suite,Cases,Suites0), + {[{{N,D},Suites1}|Replaced],true}; + (T,{Replaced,Match}) -> + {[T|Replaced],Match} + end, {[],false}, Tests), + if not Done -> + Tests ++ [{{Node,Dir},[{Suite,Cases}]}]; + true -> + Tests1 end; insert_cases(Node,Dir,Suite,Case,Tests,MergeTests) when is_atom(Case) -> insert_cases(Node,Dir,Suite,[Case],Tests,MergeTests). @@ -1132,15 +1313,23 @@ skip_groups(Node,Dir,Suite,Groups,Cases,Cmt,Tests,false) when append({{Node,Dir},Suites1},Tests); skip_groups(Node,Dir,Suite,Groups,Cases,Cmt,Tests,true) when ((Cases == all) or is_list(Cases)) and is_list(Groups) -> - Suites = - case lists:keysearch({Node,Dir},1,Tests) of - {value,{{Node,Dir},Suites0}} -> - Suites0; - false -> - [] - end, - Suites1 = skip_groups1(Suite,[{Gr,Cases} || Gr <- Groups],Cmt,Suites), - insert_in_order({{Node,Dir},Suites1},Tests); + {Tests1,Done} = + lists:foldr(fun({{N,D},Suites0},{Replaced,_}) when N == Node, + D == Dir -> + Suites1 = skip_groups1(Suite, + [{Gr,Cases} || Gr <- Groups], + Cmt,Suites0), + {[{{N,D},Suites1}|Replaced],true}; + (T,{Replaced,Match}) -> + {[T|Replaced],Match} + end, {[],false}, Tests), + if not Done -> + Tests ++ [{{Node,Dir},skip_groups1(Suite, + [{Gr,Cases} || Gr <- Groups], + Cmt,[])}]; + true -> + Tests1 + end; skip_groups(Node,Dir,Suite,Groups,Case,Cmt,Tests,MergeTests) when is_atom(Case) -> Cases = if Case == all -> all; true -> [Case] end, @@ -1162,15 +1351,19 @@ skip_cases(Node,Dir,Suite,Cases,Cmt,Tests,false) when is_list(Cases) -> Suites1 = skip_cases1(Suite,Cases,Cmt,[]), append({{Node,Dir},Suites1},Tests); skip_cases(Node,Dir,Suite,Cases,Cmt,Tests,true) when is_list(Cases) -> - Suites = - case lists:keysearch({Node,Dir},1,Tests) of - {value,{{Node,Dir},Suites0}} -> - Suites0; - false -> - [] - end, - Suites1 = skip_cases1(Suite,Cases,Cmt,Suites), - insert_in_order({{Node,Dir},Suites1},Tests); + {Tests1,Done} = + lists:foldr(fun({{N,D},Suites0},{Replaced,_}) when N == Node, + D == Dir -> + Suites1 = skip_cases1(Suite,Cases,Cmt,Suites0), + {[{{N,D},Suites1}|Replaced],true}; + (T,{Replaced,Match}) -> + {[T|Replaced],Match} + end, {[],false}, Tests), + if not Done -> + Tests ++ [{{Node,Dir},skip_cases1(Suite,Cases,Cmt,[])}]; + true -> + Tests1 + end; skip_cases(Node,Dir,Suite,Case,Cmt,Tests,MergeTests) when is_atom(Case) -> skip_cases(Node,Dir,Suite,[Case],Cmt,Tests,MergeTests). @@ -1260,7 +1453,9 @@ is_node([],_) -> valid_terms() -> [ + {set_merge_tests,2}, {define,3}, + {specs,3}, {node,3}, {cover,2}, {cover,3}, diff --git a/lib/common_test/src/ct_util.erl b/lib/common_test/src/ct_util.erl index cf891ed043..0f2b2081d9 100644 --- a/lib/common_test/src/ct_util.erl +++ b/lib/common_test/src/ct_util.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2012. All Rights Reserved. +%% Copyright Ericsson AB 2003-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -396,14 +396,14 @@ loop(Mode,TestData,StartDir) -> %% A connection crashed - remove the connection but don't die ct_logs:tc_log_async(ct_error_notify, "Connection process died: " - "Pid: ~p, Address: ~p, Callback: ~p\n" + "Pid: ~w, Address: ~p, Callback: ~w\n" "Reason: ~p\n\n", [Pid,A,CB,Reason]), catch CB:close(Pid), loop(Mode,TestData,StartDir); _ -> %% Let process crash in case of error, this shouldn't happen! - io:format("\n\nct_util_server got EXIT from ~p: ~p\n\n", + io:format("\n\nct_util_server got EXIT from ~w: ~p\n\n", [Pid,Reason]), file:set_cwd(StartDir), exit(Reason) @@ -956,7 +956,7 @@ open_url(iexplore, Args, URL) -> Path = proplists:get_value(default, Paths), [Cmd | _] = string:tokens(Path, "%"), Cmd1 = Cmd ++ " " ++ Args ++ " " ++ URL, - io:format(user, "~nOpening ~s with command:~n ~s~n", [URL,Cmd1]), + io:format(user, "~nOpening ~ts with command:~n ~ts~n", [URL,Cmd1]), open_port({spawn,Cmd1}, []); _ -> io:format("~nNo path to iexplore.exe~n",[]) @@ -969,6 +969,6 @@ open_url(Prog, Args, URL) -> is_list(Prog) -> Prog end, Cmd = ProgStr ++ " " ++ Args ++ " " ++ URL, - io:format(user, "~nOpening ~s with command:~n ~s~n", [URL,Cmd]), + io:format(user, "~nOpening ~ts with command:~n ~ts~n", [URL,Cmd]), open_port({spawn,Cmd},[]), ok. diff --git a/lib/common_test/src/cth_conn_log.erl b/lib/common_test/src/cth_conn_log.erl index 255f3ec78a..644594e34d 100644 --- a/lib/common_test/src/cth_conn_log.erl +++ b/lib/common_test/src/cth_conn_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -105,8 +105,9 @@ pre_init_per_testcase(TestCase,Config,CthState) -> "<table borders=1>" "<b>" ++ ConnModStr ++ " logs:</b>\n" ++ [io_lib:format( - "<tr><td>~p</td><td><a href=~p>~s</a></td></tr>", - [S,L,filename:basename(L)]) + "<tr><td>~p</td><td><a href=\"~ts\">~ts</a>" + "</td></tr>", + [S,ct_logs:uri(L),filename:basename(L)]) || {S,L} <- Ls] ++ "</table>", io:format(Str,[]), diff --git a/lib/common_test/src/cth_surefire.erl b/lib/common_test/src/cth_surefire.erl index e6eaad8d48..1a38b6584b 100644 --- a/lib/common_test/src/cth_surefire.erl +++ b/lib/common_test/src/cth_surefire.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -297,7 +297,7 @@ sanitize([]) -> now_to_string(Now) -> {{YY,MM,DD},{HH,Mi,SS}} = calendar:now_to_local_time(Now), - io_lib:format("~p-~2..0B-~2..0BT~2..0B:~2..0B:~2..0B",[YY,MM,DD,HH,Mi,SS]). + io_lib:format("~w-~2..0B-~2..0BT~2..0B:~2..0B:~2..0B",[YY,MM,DD,HH,Mi,SS]). make_url(undefined,_) -> undefined; diff --git a/lib/common_test/src/unix_telnet.erl b/lib/common_test/src/unix_telnet.erl index 25b9d4d5d2..99ce92e9f1 100644 --- a/lib/common_test/src/unix_telnet.erl +++ b/lib/common_test/src/unix_telnet.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2010. All Rights Reserved. +%% Copyright Ericsson AB 2004-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -110,7 +110,7 @@ connect1(Ip,Port,Timeout,KeepAlive,Username,Password) -> case ct_telnet:silent_teln_expect(Pid,[],[prompt],?prx,[]) of {ok,{prompt,?username},_} -> ok = ct_telnet_client:send_data(Pid,Username), - cont_log("Username: ~s",[Username]), + cont_log("Username: ~ts",[Username]), case ct_telnet:silent_teln_expect(Pid,[],prompt,?prx,[]) of {ok,{prompt,?password},_} -> ok = ct_telnet_client:send_data(Pid,Password), diff --git a/lib/common_test/test/Makefile b/lib/common_test/test/Makefile index d469d03e04..760cc20410 100644 --- a/lib/common_test/test/Makefile +++ b/lib/common_test/test/Makefile @@ -40,6 +40,7 @@ MODULES= \ ct_repeat_1_SUITE \ ct_testspec_1_SUITE \ ct_testspec_2_SUITE \ + ct_testspec_3_SUITE \ ct_skip_SUITE \ ct_error_SUITE \ ct_test_server_if_1_SUITE \ diff --git a/lib/common_test/test/ct_error_SUITE.erl b/lib/common_test/test/ct_error_SUITE.erl index 6d90b29f41..86ec71c98e 100644 --- a/lib/common_test/test/ct_error_SUITE.erl +++ b/lib/common_test/test/ct_error_SUITE.erl @@ -44,8 +44,12 @@ %% there will be clashes with logging processes etc). %%-------------------------------------------------------------------- init_per_suite(Config) -> - Config1 = ct_test_support:init_per_suite(Config), - Config1. + DataDir = ?config(data_dir, Config), + TestDir = filename:join(DataDir, "error/test/"), + CTH = filename:join(TestDir, "verify_config.erl"), + ct:pal("Compiling ~p: ~p", + [CTH,compile:file(CTH,[{outdir,TestDir},debug_info])]), + ct_test_support:init_per_suite([{path_dirs,[TestDir]} | Config]). end_per_suite(Config) -> ct_test_support:end_per_suite(Config). @@ -61,7 +65,8 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [cfg_error, lib_error, no_compile, timetrap_end_conf, timetrap_normal, timetrap_extended, timetrap_parallel, - timetrap_fun, timetrap_fun_group, misc_errors]. + timetrap_fun, timetrap_fun_group, misc_errors, + config_restored]. groups() -> []. @@ -285,6 +290,24 @@ misc_errors(Config) when is_list(Config) -> TestEvents = events_to_check(misc_errors), ok = ct_test_support:verify_events(TestEvents, Events, Config). +%%%----------------------------------------------------------------- +%%% +config_restored(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + Suite = filename:join(DataDir, "error/test/config_restored_SUITE"), + {Opts,ERPid} = setup([{suite,Suite}, + {ct_hooks,[verify_config]}], + Config), + ok = ct_test_support:run(Opts, Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(config_restored, + reformat(Events, ?eh), + ?config(priv_dir, Config), + Opts), + + TestEvents = events_to_check(config_restored), + ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- %%% HELP FUNCTIONS @@ -1439,4 +1462,40 @@ test_events(misc_errors) -> {?eh,test_stats,{2,7,{0,0}}}, {?eh,test_done,{'DEF','STOP_TIME'}}, {?eh,stop_logging,[]} + ]; + +test_events(config_restored) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,4}}, + {?eh,tc_start,{config_restored_SUITE,init_per_suite}}, + {?eh,tc_done,{config_restored_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{config_restored_SUITE,to_tc}}, + {?eh,cth,{verify_config,post_end_per_testcase,{to_tc,diff_ok}}}, + {?eh,tc_done, + {config_restored_SUITE,to_tc,{failed,{timetrap_timeout,1000}}}}, + {?eh,test_stats,{0,1,{0,0}}}, + {?eh,tc_start,{config_restored_SUITE,exit_tc}}, + {?eh,cth,{verify_config,post_end_per_testcase,{exit_tc,diff_ok}}}, + {?eh,tc_done,{config_restored_SUITE,exit_tc, + {failed,{error,{test_case_failed,"Goodbye!"}}}}}, + {?eh,test_stats,{0,2,{0,0}}}, + [{?eh,tc_start,{config_restored_SUITE,{init_per_group,g1,[]}}}, + {?eh,tc_start,{config_restored_SUITE,to_tc}}, + {?eh,cth,{verify_config,post_end_per_testcase,{to_tc,diff_ok}}}, + {?eh,tc_done, + {config_restored_SUITE,to_tc,{failed,{timetrap_timeout,1000}}}}, + {?eh,test_stats,{0,3,{0,0}}}, + {?eh,tc_start,{config_restored_SUITE,exit_tc}}, + {?eh,cth,{verify_config,post_end_per_testcase,{exit_tc,diff_ok}}}, + {?eh,tc_done,{config_restored_SUITE,exit_tc, + {failed,{error,{test_case_failed,"Goodbye!"}}}}}, + {?eh,test_stats,{0,4,{0,0}}}, + {?eh,tc_start,{config_restored_SUITE,{end_per_group,g1,[]}}}, + {?eh,tc_done,{config_restored_SUITE,{end_per_group,g1,[]},ok}}], + {?eh,tc_start,{config_restored_SUITE,end_per_suite}}, + {?eh,tc_done,{config_restored_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} ]. diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/config_restored_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/config_restored_SUITE.erl new file mode 100644 index 0000000000..fb47f2bcb9 --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/config_restored_SUITE.erl @@ -0,0 +1,175 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(config_restored_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% Info = [tuple()] +%%-------------------------------------------------------------------- +suite() -> + [{timetrap,1000}]. + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config1 = [{init_per_suite,?MODULE} | Config], + TabPid = spawn(fun() -> + ets:new(?MODULE, [named_table, set, public]), + receive _ -> ok end + end), + [{tab,TabPid} | Config1]. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_suite(Config) -> + ets:delete(?MODULE), + exit(?config(tab, Config), kill), + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_group(GroupName, Config) -> + [{init_per_group,GroupName} | Config]. + +%%-------------------------------------------------------------------- +%% Function: end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_testcase(TC, Config) -> + Config1 = [{init_per_testcase,TC} | Config], + ets:insert(?MODULE, {config,Config}), + %% ct:pal("Config after init_per_testcase(~w) = ~p", [TC,Config1]), + Config1. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_testcase(TC, Config) -> + ct:pal("Config in end_per_testcase(~w) = ~p", [TC,Config]), + [{_,MemConfig}] = ets:lookup(?MODULE, config), + diff_config(Config, MemConfig, [tc_status]), + ?MODULE = proplists:get_value(init_per_suite, Config), + TC = proplists:get_value(init_per_testcase, Config), + case ?config(tc_group_properties, Config) of + undefined -> + ok; + Props -> + GName = proplists:get_value(name, Props), + GName = proplists:get_value(init_per_group, Config) + end, + ok. + +%%-------------------------------------------------------------------- +%% Function: groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%%-------------------------------------------------------------------- +groups() -> + [{g1,[],[to_tc, exit_tc]}]. + +%%-------------------------------------------------------------------- +%% Function: all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%%-------------------------------------------------------------------- +all() -> + [to_tc, exit_tc, + {group,g1}]. + + +to_tc(Config) -> + %% ct:pal("Config for to_tc = ~p", [Config]), + [{_,MemConfig}] = ets:lookup(?MODULE, config), + diff_config(Config, MemConfig, []), + ct:sleep(2000). + +exit_tc(Config) -> + %% ct:pal("Config for exit_tc = ~p", [Config]), + [{_,MemConfig}] = ets:lookup(?MODULE, config), + diff_config(Config, MemConfig, []), + ct:fail("Goodbye!"). + + +%%%----------------------------------------------------------------- +%%% HELP FUNCTIONS + +diff_config(Cfg1, Cfg2, DiffKeys) -> + diff_config(Cfg1, Cfg2, DiffKeys, []). + +diff_config([{K,V} | Cfg1], Cfg2, DiffKeys, RemKeys) -> + case proplists:get_value(K, Cfg2) of + undefined -> + diff_config(Cfg1, Cfg2, proplists:delete(K, DiffKeys), RemKeys); + V -> + diff_config(Cfg1, proplists:delete(K, Cfg2), DiffKeys, RemKeys); + _Other -> + case proplists:is_defined(K, DiffKeys) of + true -> + diff_config(Cfg1, Cfg2, proplists:delete(K, DiffKeys), RemKeys); + false -> + diff_config(Cfg1, Cfg2, DiffKeys, [K | RemKeys]) + end + end; +diff_config([], [], [], []) -> + ct:pal("Diff ok!", []), + ok; +diff_config([], Cfg2, DiffKeys, RemKeys) -> + Result = {diff_failed, {cfg2,Cfg2}, {diffkeys,DiffKeys}, {remkeys,RemKeys}}, + ct:pal("Diff failed! Result = ~p", [Result]), + exit(Result). + diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/verify_config.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/verify_config.erl new file mode 100644 index 0000000000..40a54b9f99 --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/verify_config.erl @@ -0,0 +1,239 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%% @doc Common Test Example Suite Callback module. +%%% +%%% <p>This module gives an example of a common test CTH (Common Test Hook). +%%% There are many ways to add a CTH to a test run, you can do it either in +%%% the command line using -ct_hook, in a test spec using +%%% {ct_hook,M} or in the suite it self by returning ct_hook +%%% from either suite/0, init_per_suite/1, init_per_group/2 and +%%% init_per_testcase/2. The scope of the CTH is determined by where is it +%%% started. If it is started in the command line or test spec then it will +%%% be stopped at the end of all tests. If it is started in init_per_suite, +%%% it will be stopped after end_per_suite and so on. See terminate +%%% documentation for a table describing the scoping machanics. +%%% +%%% All of callbacks except init/1 in a CTH are optional.</p> + +-module(verify_config). + +%% CT Hooks +-export([id/1]). +-export([init/2]). + +-export([pre_init_per_suite/3]). +-export([post_init_per_suite/4]). +-export([pre_end_per_suite/3]). +-export([post_end_per_suite/4]). + +-export([pre_init_per_group/3]). +-export([post_init_per_group/4]). +-export([pre_end_per_group/3]). +-export([post_end_per_group/4]). + +-export([pre_init_per_testcase/3]). +-export([post_end_per_testcase/4]). + +-export([on_tc_fail/3]). +-export([on_tc_skip/3]). + +-export([terminate/1]). + +-include_lib("common_test/src/ct_util.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + +-type config() :: proplists:proplist(). +-type reason() :: term(). +-type skip_or_fail() :: {skip, reason()} | + {auto_skip, reason()} | + {fail, reason()} | + {'EXIT',reason()}. + +-record(state, { id = ?MODULE :: term()}). + +%% @doc Always called before any other callback function. Use this to initiate +%% any common state. It should return an state for this CTH. +-spec init(Id :: term(), Opts :: proplists:proplist()) -> + {ok, State :: #state{}}. +init(Id, Opts) -> + {ok,Opts}. + +%% @doc The ID is used to uniquly identify an CTH instance, if two CTH's +%% return the same ID the seconds CTH is ignored. This function should NOT +%% have any side effects as it might be called multiple times by common test. +-spec id(Opts :: proplists:proplist()) -> + Id :: term(). +id(Opts) -> + now(). + +%% @doc Called before init_per_suite is called. Note that this callback is +%% only called if the CTH is added before init_per_suite is run (eg. in a test +%% specification, suite/0 function etc). +%% You can change the config in the this function. +-spec pre_init_per_suite(Suite :: atom(), + Config :: config(), + State :: #state{}) -> + {config() | skip_or_fail(), NewState :: #state{}}. +pre_init_per_suite(Suite,Config,State) -> + {Config, State}. + +%% @doc Called after init_per_suite. +%% you can change the return value in this function. +-spec post_init_per_suite(Suite :: atom(), + Config :: config(), + Return :: config() | skip_or_fail(), + State :: #state{}) -> + {config() | skip_or_fail(), NewState :: #state{}}. +post_init_per_suite(Suite,Config,Return,State) -> + {Return, State}. + +%% @doc Called before end_per_suite. The config/state can be changed here, +%% though it will only affect the *end_per_suite function. +-spec pre_end_per_suite(Suite :: atom(), + Config :: config() | skip_or_fail(), + State :: #state{}) -> + {ok | skip_or_fail(), NewState :: #state{}}. +pre_end_per_suite(Suite,Config,State) -> + {Config, State}. + +%% @doc Called after end_per_suite. Note that the config cannot be +%% changed here, only the status of the suite. +-spec post_end_per_suite(Suite :: atom(), + Config :: config(), + Return :: term(), + State :: #state{}) -> + {ok | skip_or_fail(), NewState :: #state{}}. +post_end_per_suite(Suite,Config,Return,State) -> + {Return, State}. + +%% @doc Called before each init_per_group. +%% You can change the config in this function. +-spec pre_init_per_group(Group :: atom(), + Config :: config(), + State :: #state{}) -> + {config() | skip_or_fail(), NewState :: #state{}}. +pre_init_per_group(Group,Config,State) -> + {Config, State}. + +%% @doc Called after each init_per_group. +%% You can change the return value in this function. +-spec post_init_per_group(Group :: atom(), + Config :: config(), + Return :: config() | skip_or_fail(), + State :: #state{}) -> + {config() | skip_or_fail(), NewState :: #state{}}. +post_init_per_group(Group,Config,Return,State) -> + {Return, State}. + +%% @doc Called after each end_per_group. The config/state can be changed here, +%% though it will only affect the *end_per_group functions. +-spec pre_end_per_group(Group :: atom(), + Config :: config() | skip_or_fail(), + State :: #state{}) -> + {ok | skip_or_fail(), NewState :: #state{}}. +pre_end_per_group(Group,Config,State) -> + {Config, State}. + +%% @doc Called after each end_per_group. Note that the config cannot be +%% changed here, only the status of the group. +-spec post_end_per_group(Group :: atom(), + Config :: config(), + Return :: term(), + State :: #state{}) -> + {ok | skip_or_fail(), NewState :: #state{}}. +post_end_per_group(Group,Config,Return,State) -> + {Return, State}. + +%% @doc Called before each test case. +%% You can change the config in this function. +-spec pre_init_per_testcase(TC :: atom(), + Config :: config(), + State :: #state{}) -> + {config() | skip_or_fail(), NewState :: #state{}}. +pre_init_per_testcase(TC,Config,State) -> + {Config, State}. + +%% @doc Called after each test case. Note that the config cannot be +%% changed here, only the status of the test case. +-spec post_end_per_testcase(TC :: atom(), + Config :: config(), + Return :: term(), + State :: #state{}) -> + {ok | skip_or_fail(), NewState :: #state{}}. +post_end_per_testcase(TC,Config,Return,State) -> + %% check that config has been restored + ct:pal("Config in verify_config:post_end_per_testcase(~w) = ~p", + [TC,Config]), + [{_,MemConfig}] = ets:lookup(config_restored_SUITE, config), + try config_restored_SUITE:diff_config(Config, MemConfig, [tc_status]) of + ok -> + gen_event:notify( + ?CT_EVMGR_REF, #event{ name = cth, node = node(), + data = {?MODULE, post_end_per_testcase, + {TC,diff_ok}}}) + catch + _:_ -> + gen_event:notify( + ?CT_EVMGR_REF, #event{ name = cth, node = node(), + data = {?MODULE, post_end_per_testcase, + {TC,diff_failed}}}) + end, + {Return, State}. + +%% @doc Called after post_init_per_suite, post_end_per_suite, post_init_per_group, +%% post_end_per_group and post_end_per_tc if the suite, group or test case failed. +%% This function should be used for extra cleanup which might be needed. +%% It is not possible to modify the config or the status of the test run. +-spec on_tc_fail(TC :: init_per_suite | end_per_suite | + init_per_group | end_per_group | atom(), + Reason :: term(), State :: #state{}) -> + NewState :: #state{}. +on_tc_fail(TC, Reason, State) -> + State. + +%% @doc Called when a test case is skipped by either user action +%% or due to an init function failing. Test case can be +%% end_per_suite, init_per_group, end_per_group and the actual test cases. +-spec on_tc_skip(TC :: end_per_suite | + init_per_group | end_per_group | atom(), + {tc_auto_skip, {failed, {Mod :: atom(), Function :: atom(), Reason :: term()}}} | + {tc_user_skip, {skipped, Reason :: term()}}, + State :: #state{}) -> + NewState :: #state{}. +on_tc_skip(TC, Reason, State) -> + State. + +%% @doc Called when the scope of the CTH is done, this depends on +%% when the CTH was specified. This translation table describes when this +%% function is called. +%% +%% | Started in | terminate called | +%% |---------------------|-------------------------| +%% | command_line | after all tests are run | +%% | test spec | after all tests are run | +%% | suite/0 | after SUITE is done | +%% | init_per_suite/1 | after SUITE is done | +%% | init_per_group/2 | after group is done | +%% |-----------------------------------------------| +%% +-spec terminate(State :: #state{}) -> + term(). +terminate(State) -> + ok. diff --git a/lib/common_test/test/ct_master_SUITE.erl b/lib/common_test/test/ct_master_SUITE.erl index b89d7d4dc5..0f336d2d79 100644 --- a/lib/common_test/test/ct_master_SUITE.erl +++ b/lib/common_test/test/ct_master_SUITE.erl @@ -109,7 +109,7 @@ ct_master_test(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - [{TSFile,ok}] = run_test(ct_master_test, FileName, Config), + [{[TSFile],ok}] = run_test(ct_master_test, FileName, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -195,12 +195,12 @@ get_log_dir(_,PrivDir,NodeName) -> run_test(_Name, FileName, Config) -> %% run the test twice, using different html versions - [{FileName,ok}] = ct_test_support:run({ct_master,run,[FileName]}, - [{ct_master,basic_html,[true]}], - Config), - [{FileName,ok}] = ct_test_support:run({ct_master,run,[FileName]}, - [{ct_master,basic_html,[false]}], - Config). + [{[FileName],ok}] = ct_test_support:run({ct_master,run,[FileName]}, + [{ct_master,basic_html,[true]}], + Config), + [{[FileName],ok}] = ct_test_support:run({ct_master,run,[FileName]}, + [{ct_master,basic_html,[false]}], + Config). reformat(Events, EH) -> ct_test_support:reformat(Events, EH). diff --git a/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl b/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl index 8a5009ad62..df54c4419c 100644 --- a/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl +++ b/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010. All Rights Reserved. +%% Copyright Ericsson AB 2010-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl b/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl index 2427f37f52..09217f60a3 100644 --- a/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl +++ b/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -426,7 +426,7 @@ reply(ConnRef,Reply) -> send(ConnRef, make_msg(Reply)). from_simple(Simple) -> - list_to_binary(xmerl:export_simple_element(Simple,xmerl_xml)). + unicode_c2b(xmerl:export_simple_element(Simple,xmerl_xml)). xml(Content) -> <<"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n", @@ -435,30 +435,30 @@ xml(Content) -> rpc_reply(Content) when is_binary(Content) -> MsgId = case erase(msg_id) of undefined -> <<>>; - Id -> list_to_binary([" message-id=\"",Id,"\""]) + Id -> unicode_c2b([" message-id=\"",Id,"\""]) end, <<"<rpc-reply xmlns=\"",?NETCONF_NAMESPACE,"\"",MsgId/binary,">\n", Content/binary,"\n</rpc-reply>">>; rpc_reply(Content) -> - rpc_reply(list_to_binary(Content)). + rpc_reply(unicode_c2b(Content)). session_id(no_session_id) -> <<>>; session_id(SessionId0) -> - SessionId = list_to_binary(integer_to_list(SessionId0)), + SessionId = unicode_c2b(integer_to_list(SessionId0)), <<"<session-id>",SessionId/binary,"</session-id>\n">>. capabilities(undefined) -> - CapsXml = list_to_binary([["<capability>",C,"</capability>\n"] + CapsXml = unicode_c2b([["<capability>",C,"</capability>\n"] || C <- ?CAPABILITIES]), <<"<capabilities>\n",CapsXml/binary,"</capabilities>\n">>; capabilities({base,Vsn}) -> - CapsXml = list_to_binary([["<capability>",C,"</capability>\n"] + CapsXml = unicode_c2b([["<capability>",C,"</capability>\n"] || C <- ?CAPABILITIES_VSN(Vsn)]), <<"<capabilities>\n",CapsXml/binary,"</capabilities>\n">>; capabilities(no_base) -> [_|Caps] = ?CAPABILITIES, - CapsXml = list_to_binary([["<capability>",C,"</capability>\n"] || C <- Caps]), + CapsXml = unicode_c2b([["<capability>",C,"</capability>\n"] || C <- Caps]), <<"<capabilities>\n",CapsXml/binary,"</capabilities>\n">>; capabilities(no_caps) -> <<>>. @@ -553,3 +553,8 @@ make_msg(Xml) when is_binary(Xml) -> xml(Xml); make_msg(Simple) when is_tuple(Simple) -> xml(from_simple(Simple)). + +%%%----------------------------------------------------------------- +%%% Convert to unicode binary, since we use UTF-8 encoding in XML +unicode_c2b(Characters) -> + unicode:characters_to_binary(Characters). diff --git a/lib/common_test/test/ct_surefire_SUITE.erl b/lib/common_test/test/ct_surefire_SUITE.erl index 69e98cef48..b86b47f0a2 100644 --- a/lib/common_test/test/ct_surefire_SUITE.erl +++ b/lib/common_test/test/ct_surefire_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -32,6 +32,7 @@ -include_lib("common_test/include/ct_event.hrl"). -include_lib("xmerl/include/xmerl.hrl"). +-include_lib("kernel/include/file.hrl"). -define(eh, ct_test_support_eh). @@ -77,53 +78,52 @@ all() -> %%%----------------------------------------------------------------- %%% default(Config) when is_list(Config) -> - run(default,[cth_surefire],Config), - PrivDir = ?config(priv_dir,Config), - XmlRe = filename:join([PrivDir,"*","junit_report.xml"]), - check_xml(default,XmlRe). + run(default,[cth_surefire],"junit_report.xml",Config). absolute_path(Config) when is_list(Config) -> PrivDir = ?config(priv_dir,Config), Path = filename:join(PrivDir,"abspath.xml"), - run(absolute_path,[{cth_surefire,[{path,Path}]}],Config), - check_xml(absolute_path,Path). + run(absolute_path,[{cth_surefire,[{path,Path}]}],Path,Config). relative_path(Config) when is_list(Config) -> Path = "relpath.xml", - run(relative_path,[{cth_surefire,[{path,Path}]}],Config), - PrivDir = ?config(priv_dir,Config), - XmlRe = filename:join([PrivDir,"*",Path]), - check_xml(relative_path,XmlRe). + run(relative_path,[{cth_surefire,[{path,Path}]}],Path,Config). url(Config) when is_list(Config) -> Path = "url.xml", - run(url,[{cth_surefire,[{url_base,?url_base}, - {path,Path}]}],Config), - PrivDir = ?config(priv_dir,Config), - XmlRe = filename:join([PrivDir,"*",Path]), - check_xml(url,XmlRe). + run(url,[{cth_surefire,[{url_base,?url_base},{path,Path}]}], + Path,Config). logdir(Config) when is_list(Config) -> - PrivDir = ?config(priv_dir,Config), - LogDir = filename:join(PrivDir,"specific_logdir"), - file:make_dir(LogDir), + Opts = ct_test_support:get_opts(Config), + LogDir = + case lists:keyfind(logdir,1,Opts) of + {logdir,LD} -> LD; + false -> ?config(priv_dir,Config) + end, + MyLogDir = filename:join(LogDir,"specific_logdir"), + ensure_exists_empty(MyLogDir), Path = "logdir.xml", - run(logdir,[{cth_surefire,[{path,Path}]}],Config,[{logdir,LogDir}]), - PrivDir = ?config(priv_dir,Config), - XmlRe = filename:join([LogDir,"*",Path]), - check_xml(logdir,XmlRe). + run(logdir,[{cth_surefire,[{path,Path}]}],Path,Config,[{logdir,MyLogDir}]). %%%----------------------------------------------------------------- %%% HELP FUNCTIONS %%%----------------------------------------------------------------- -run(Case,CTHs,Config) -> - run(Case,CTHs,Config,[]). -run(Case,CTHs,Config,ExtraOpts) -> +run(Case,CTHs,Report,Config) -> + run(Case,CTHs,Report,Config,[]). +run(Case,CTHs,Report,Config,ExtraOpts) -> DataDir = ?config(data_dir, Config), Suite = filename:join(DataDir, "surefire_SUITE"), {Opts,ERPid} = setup([{suite,Suite},{ct_hooks,CTHs},{label,Case}|ExtraOpts], Config), - ok = execute(Case, Opts, ERPid, Config). + ok = execute(Case, Opts, ERPid, Config), + LogDir = + case lists:keyfind(logdir,1,Opts) of + {logdir,LD} -> LD; + false -> ?config(priv_dir,Config) + end, + Re = filename:join([LogDir,"*",Report]), + check_xml(Case,Re). setup(Test, Config) -> Opts0 = ct_test_support:get_opts(Config), @@ -349,3 +349,26 @@ get_numbers_from_attrs([_|A],T,E,F,S) -> get_numbers_from_attrs(A,T,E,F,S); get_numbers_from_attrs([],T,E,F,S) -> {T,E,F,S}. + +ensure_exists_empty(Dir) -> + case file:list_dir(Dir) of + {error,enoent} -> + file:make_dir(Dir); + {ok,Files} -> + del_files(Dir,Files) + end. + +del_files(Dir,[F0|Fs] ) -> + F = filename:join(Dir,F0), + case file:read_file_info(F) of + {ok,#file_info{type=directory}} -> + {ok,Files} = file:list_dir(F), + del_files(F,Files), + file:del_dir(F), + del_files(Dir,Fs); + _ -> + file:delete(F), + del_files(Dir,Fs) + end; +del_files(_,[]) -> + ok. diff --git a/lib/common_test/test/ct_test_support.erl b/lib/common_test/test/ct_test_support.erl index 8814570e99..7c33fd404d 100644 --- a/lib/common_test/test/ct_test_support.erl +++ b/lib/common_test/test/ct_test_support.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2012. All Rights Reserved. +%% Copyright Ericsson AB 2008-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1132,6 +1132,8 @@ reformat([{_EH,#event{name=test_start,data=_}} | Events], EH) -> [{EH,test_start,{'DEF',{'START_TIME','LOGDIR'}}} | reformat(Events, EH)]; reformat([{_EH,#event{name=test_done,data=_}} | Events], EH) -> [{EH,test_done,{'DEF','STOP_TIME'}} | reformat(Events, EH)]; +reformat([{_EH,#event{name=tc_logfile,data=_}} | Events], EH) -> + reformat(Events, EH); reformat([{_EH,#event{name=test_stats,data=Data}} | Events], EH) -> [{EH,test_stats,Data} | reformat(Events, EH)]; %% use this to only print the last test_stats event: diff --git a/lib/common_test/test/ct_testspec_2_SUITE.erl b/lib/common_test/test/ct_testspec_2_SUITE.erl index 9d2dc84ad3..ea22312a7d 100644 --- a/lib/common_test/test/ct_testspec_2_SUITE.erl +++ b/lib/common_test/test/ct_testspec_2_SUITE.erl @@ -479,7 +479,7 @@ multiple_specs(_Config) -> "multiple_specs.1.spec"), SpecFile2 = ct_test_support:write_testspec(Spec2,SpecDir, "multiple_specs.2.spec"), - FileResult = ct_testspec:collect_tests_from_file([SpecFile1,SpecFile2], + FileResult = ct_testspec:collect_tests_from_file([[SpecFile1,SpecFile2]], false), ct:pal("TESTSPEC RECORD FROM FILE:~n~p~n", [rec2proplist(FileResult)]), @@ -490,7 +490,7 @@ multiple_specs(_Config) -> [{Node2,get_absdir(filename:join(SpecDir,CfgDir))} || CfgDir <- CfgDir2]], LogDirV = get_absdir(filename:join(SpecDir,"../logs")), - Verify = #testspec{merge_tests = false, + Verify = #testspec{merge_tests = true, spec_dir = SpecDir, nodes = [{undefined,Node},{n1,Node1},{n2,Node2}], alias = [{to1,TO1V},{to2,TO2V}], @@ -524,7 +524,7 @@ multiple_specs(_Config) -> %%% misc_config_terms(_Config) -> CfgDir = "../cfgs/to1", - + TODir = "../tests/to1", Spec = [{node,x,n1@h1},{node,y,n2@h2}, @@ -554,7 +554,9 @@ misc_config_terms(_Config) -> {create_priv_dir,[auto_per_tc]}, {create_priv_dir,n1@h1,[manual_per_tc]}, - {create_priv_dir,n2@h2,[auto_per_run]} + {create_priv_dir,n2@h2,[auto_per_run]}, + + {suites,n1@h1,TODir,[x_SUITE]} ], {ok,SpecDir} = file:get_cwd(), @@ -599,7 +601,9 @@ misc_config_terms(_Config) -> {n2@h2,CSS2}], create_priv_dir = [{Node,[auto_per_tc]}, {n1@h1,[manual_per_tc]}, - {n2@h2,[auto_per_run]}] + {n2@h2,[auto_per_run]}], + tests = [{{n1@h1,get_absdir(filename:join(SpecDir,TODir))}, + [{x_SUITE,[all]}]}] }, verify_result(Verify,ListResult,FileResult). @@ -688,10 +692,10 @@ define_names_1(_Config) -> %%% HELP FUNCTIONS %%%----------------------------------------------------------------- -verify_result(Verify,ListResult,FileResult) -> +verify_result(VerificationRec,ListResult,FileResult) -> {_,TSLTuples} = rec2proplist(ListResult), {_,TSFTuples} = rec2proplist(FileResult), - {_,VTuples} = rec2proplist(Verify), + {_,VTuples} = rec2proplist(VerificationRec), VResult = (catch lists:foldl(fun({Tag,Val},{[{Tag,Val}|TSL],[{Tag,Val}|TSF]}) -> {TSL,TSF}; @@ -720,6 +724,8 @@ read_config(S) -> rec2proplist(E={error,_What}) -> exit({invalid_testspec_record,E}); +rec2proplist([{Specs,Rec}]) when is_list(Specs) -> + rec2proplist(Rec); rec2proplist(Rec) -> [RecName|RecList] = tuple_to_list(Rec), FieldNames = diff --git a/lib/common_test/test/ct_testspec_3_SUITE.erl b/lib/common_test/test/ct_testspec_3_SUITE.erl new file mode 100644 index 0000000000..e2f57c2c64 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE.erl @@ -0,0 +1,745 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2009-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_testspec_1_SUITE +%%% +%%% Description: +%%% Test test specifications +%%% +%%% The suites used for the test are located in the data directory. +%%%------------------------------------------------------------------- +-module(ct_testspec_3_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + +-define(eh, ct_test_support_eh). + +%%-------------------------------------------------------------------- +%% TEST SERVER CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Description: Since Common Test starts another Test Server +%% instance, the tests need to be performed on a separate node (or +%% there will be clashes with logging processes etc). +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + DataDir = ?config(data_dir, Config), + Config1 = ct_test_support:init_per_suite(Config), + SpecsDir1 = filename:join(DataDir, "specs1"), + SpecsDir2 = filename:join(DataDir, "specs2"), + [{specs_dir1,SpecsDir1},{specs_dir2,SpecsDir2} | Config1]. + +end_per_suite(Config) -> + ct_test_support:end_per_suite(Config). + +init_per_testcase(TestCase, Config) -> + ct_test_support:init_per_testcase(TestCase, Config). + +end_per_testcase(TestCase, Config) -> + ct_test_support:end_per_testcase(TestCase, Config). + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [start_separate, + start_join, + incl_separate1, + incl_separate2, + incl_join1, + incl_join2, + incl_both1, + incl_both2, + incl_both_and_join1, + incl_both_and_join2, + rec_incl_separate1, + rec_incl_separate2, + rec_incl_join1, + rec_incl_join2, + rec_incl_separate_join1, + rec_incl_separate_join2, + rec_incl_join_separate1, + rec_incl_join_separate2 + ]. + +groups() -> + []. + +init_per_group(_GroupName, Config) -> + Config. + +end_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +%%%----------------------------------------------------------------- +%%% + +start_separate(Config) -> + Specs = [fname(specs_dir1, "flat_spec1", Config), + fname(specs_dir2, "flat_spec2", Config)], + setup_and_execute(start_separate, Specs, [], Config). + +%%%----------------------------------------------------------------- +%%% + +start_join(Config) -> + Specs = [fname(specs_dir1, "flat_spec1", Config), + fname(specs_dir2, "flat_spec2", Config)], + setup_and_execute(start_join, Specs, [{join_specs,true}], Config). + +%%%----------------------------------------------------------------- +%%% + +incl_separate1(Config) -> + Specs = [fname(specs_dir1, "spec_sep1", Config), + fname(specs_dir2, "spec_sep2", Config)], + setup_and_execute(incl_separate1, Specs, [], Config). + +incl_separate2(Config) -> + Specs = [fname(specs_dir1, "spec_sep1", Config), + fname(specs_dir2, "spec_sep2", Config)], + setup_and_execute(incl_separate2, Specs, [{join_specs,true}], Config). + +%%%----------------------------------------------------------------- +%%% + +incl_join1(Config) -> + Specs = [fname(specs_dir1, "spec_join1", Config), + fname(specs_dir2, "spec_join2", Config)], + setup_and_execute(incl_join1, Specs, [], Config). + +incl_join2(Config) -> + Specs = [fname(specs_dir1, "spec_join1", Config), + fname(specs_dir2, "spec_join2", Config)], + setup_and_execute(incl_join2, Specs, [{join_specs,true}], Config). + +%%%----------------------------------------------------------------- +%%% + +incl_both1(Config) -> + Specs = [fname(specs_dir1, "spec_both1", Config), + fname(specs_dir2, "spec_both2", Config)], + setup_and_execute(incl_both1, Specs, [], Config). + +incl_both2(Config) -> + Specs = [fname(specs_dir1, "spec_both1", Config), + fname(specs_dir2, "spec_both2", Config)], + setup_and_execute(incl_both2, Specs, [{join_specs,true}], Config). + +%%%----------------------------------------------------------------- +%%% + +incl_both_and_join1(Config) -> + Specs = [fname(specs_dir1, "spec_both_join1", Config), + fname(specs_dir2, "spec_both_join2", Config)], + setup_and_execute(incl_both_and_join1, Specs, [], Config). + +incl_both_and_join2(Config) -> + Specs = [fname(specs_dir1, "spec_both_join1", Config), + fname(specs_dir2, "spec_both_join2", Config)], + setup_and_execute(incl_both_and_join2, Specs, [{join_specs,true}], Config). + +%%%----------------------------------------------------------------- +%%% + +rec_incl_separate1(Config) -> + Specs = [fname(specs_dir1, "rec_spec_sep1", Config), + fname(specs_dir2, "rec_spec_sep2", Config)], + setup_and_execute(rec_incl_separate1, Specs, [], Config). + +rec_incl_separate2(Config) -> + Specs = [fname(specs_dir1, "rec_spec_sep1", Config), + fname(specs_dir2, "rec_spec_sep2", Config)], + setup_and_execute(rec_incl_separate2, Specs, [{join_specs,true}], Config). + +%%%----------------------------------------------------------------- +%%% + +rec_incl_join1(Config) -> + Specs = [fname(specs_dir1, "rec_spec_join1", Config), + fname(specs_dir2, "rec_spec_join2", Config)], + setup_and_execute(rec_incl_join1, Specs, [], Config). + +rec_incl_join2(Config) -> + Specs = [fname(specs_dir1, "rec_spec_join1", Config), + fname(specs_dir2, "rec_spec_join2", Config)], + setup_and_execute(rec_incl_join2, Specs, [{join_specs,true}], Config). + + +%%%----------------------------------------------------------------- +%%% + +rec_incl_separate_join1(Config) -> + Specs = [fname(specs_dir1, "rec_spec_sep_join1", Config), + fname(specs_dir2, "rec_spec_sep_join2", Config)], + setup_and_execute(rec_incl_separate_join1, Specs, [], Config). + +rec_incl_separate_join2(Config) -> + Specs = [fname(specs_dir1, "rec_spec_sep_join1", Config), + fname(specs_dir2, "rec_spec_sep_join2", Config)], + setup_and_execute(rec_incl_separate_join2, Specs, + [{join_specs,true}], Config). + +%%%----------------------------------------------------------------- +%%% + +rec_incl_join_separate1(Config) -> + Specs = [fname(specs_dir1, "rec_spec_join_sep1", Config), + fname(specs_dir2, "rec_spec_join_sep2", Config)], + setup_and_execute(rec_incl_join_separate1, Specs, [], Config). + +rec_incl_join_separate2(Config) -> + Specs = [fname(specs_dir1, "rec_spec_join_sep1", Config), + fname(specs_dir2, "rec_spec_join_sep2", Config)], + setup_and_execute(rec_incl_join_separate2, Specs, + [{join_specs,true}], Config). + + +%%%----------------------------------------------------------------- +%%% HELP FUNCTIONS +%%%----------------------------------------------------------------- + +fname(Tag, File, Config) -> + filename:join(?config(Tag, Config), File). + +check_parameter(TCID) -> + {ok,{config,TCID}}. + +read_config(TCID) -> + {ok,[{tcname,list_to_atom(TCID)}]}. + +setup_and_execute(TCName, Specs, TestOpts, Config) -> + + TestID = {userconfig,{?MODULE,atom_to_list(TCName)}}, + TestTerms = [TestID,{spec,Specs},{label,TCName}] ++ TestOpts, + + {Opts,ERPid} = setup(TestTerms, Config), + + case ct_test_support:run(Opts, Config) of + ok -> + ok; + Error -> + ct:pal("Error executing with opts: ~p", [Opts]), + exit(Error) + end, + + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(TCName, + reformat(Events, ?eh), + ?config(priv_dir, Config), + Opts), + + TestEvents = events_to_check(TCName), + ok = ct_test_support:verify_events(TestEvents, Events, Config). + +setup(Test, Config) when is_tuple(Test) -> + setup([Test], Config); +setup(Tests, Config) -> + Opts0 = ct_test_support:get_opts(Config), + Level = ?config(trace_level, Config), + EvHArgs = [{cbm,ct_test_support},{trace_level,Level}], + Opts = Opts0 ++ Tests ++ [{event_handler,{?eh,EvHArgs}}], + ERPid = ct_test_support:start_event_receiver(Config), + {Opts,ERPid}. + +reformat(Events, EH) -> + ct_test_support:reformat(Events, EH). +%reformat(Events, _EH) -> +% Events. + +%%%----------------------------------------------------------------- +%%% TEST EVENTS +%%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). + +test_events(start_separate) -> + [{?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{3,2,15}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}]; + +test_events(start_join) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{4,4,20}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{4,8,{4,4}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}]; + +test_events(incl_separate1) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{3,2,15}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,5}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{3,2,15}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}]; + +test_events(incl_separate2) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,5}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{3,2,15}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{3,2,15}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}]; + +test_events(incl_join1) -> + [{?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{4,4,20}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{4,8,{4,4}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{4,4,20}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{4,8,{4,4}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}]; + +test_events(incl_join2) -> + [{?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{4,4,20}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{4,8,{4,4}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}]; + +test_events(incl_both1) -> + [{?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{3,2,15}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}]; + +test_events(incl_both2) -> + [{?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{4,4,20}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,ok_tc}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{4,8,{4,4}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{3,2,15}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t12_SUITE,init_per_suite}}, + {?eh,tc_done,{t12_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t12_SUITE,end_per_suite}}, + {?eh,tc_done,{t12_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t22_SUITE,init_per_suite}}, + {?eh,tc_done,{t22_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{3,6,{3,3}}}, + {?eh,tc_start,{t22_SUITE,end_per_suite}}, + {?eh,tc_done,{t22_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}, + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,10}}, + {?eh,tc_start,{t11_SUITE,init_per_suite}}, + {?eh,tc_done,{t11_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{1,2,{1,1}}}, + {?eh,tc_start,{t11_SUITE,end_per_suite}}, + {?eh,tc_done,{t11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{t21_SUITE,init_per_suite}}, + {?eh,tc_done,{t21_SUITE,init_per_suite,ok}}, + {?eh,test_stats,{2,4,{2,2}}}, + {?eh,tc_start,{t21_SUITE,end_per_suite}}, + {?eh,tc_done,{t21_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]}]; + +test_events(incl_both_and_join1) -> []; +test_events(incl_both_and_join2) -> []; +test_events(rec_incl_separate1) -> []; +test_events(rec_incl_separate2) -> []; +test_events(rec_incl_join1) -> []; +test_events(rec_incl_join2) -> []; +test_events(rec_incl_separate_join1) -> []; +test_events(rec_incl_separate_join2) -> []; +test_events(rec_incl_join_separate1) -> []; +test_events(rec_incl_join_separate2) -> []; + +test_events(_) -> + []. + + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg11 b/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg11 new file mode 100644 index 0000000000..bc672568da --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg11 @@ -0,0 +1 @@ +{file, cfg11}.
\ No newline at end of file diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg12 b/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg12 new file mode 100644 index 0000000000..30f2cf6857 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg12 @@ -0,0 +1 @@ +{file, cfg12}.
\ No newline at end of file diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg13 b/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg13 new file mode 100644 index 0000000000..1860ec78e5 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/config1/cfg13 @@ -0,0 +1 @@ +{file, cfg13}.
\ No newline at end of file diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/config2/cfg21 b/lib/common_test/test/ct_testspec_3_SUITE_data/config2/cfg21 new file mode 100644 index 0000000000..b18d35443e --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/config2/cfg21 @@ -0,0 +1 @@ +{file, cfg21}.
\ No newline at end of file diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/flat_spec1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/flat_spec1 new file mode 100644 index 0000000000..eff87222ea --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/flat_spec1 @@ -0,0 +1,4 @@ +{config, "../config1/cfg11"}. +{suites, "../tests1", t11_SUITE}. +{suites, "../tests1", t11_SUITE}. +{suites, "../tests2", t21_SUITE}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_join1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_join1 new file mode 100644 index 0000000000..a3387f48a3 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_join1 @@ -0,0 +1,2 @@ +{specs,join,"spec_join1"}. +{specs,join,"../specs2/spec_join2"}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_join_sep1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_join_sep1 new file mode 100644 index 0000000000..fe127eb4b9 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_join_sep1 @@ -0,0 +1,5 @@ +{specs,join,"spec_sep1"}. +{specs,join,"../specs2/spec_sep2"}. + +{config, "../config1/cfg13"}. +{suites, "../tests2", t23_SUITE}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_sep1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_sep1 new file mode 100644 index 0000000000..c778aa68a6 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_sep1 @@ -0,0 +1,2 @@ +{specs,separate,"spec_sep1"}. +{specs,separate,"../specs2/spec_sep2"}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_sep_join1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_sep_join1 new file mode 100644 index 0000000000..7cb5a05fff --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/rec_spec_sep_join1 @@ -0,0 +1,2 @@ +{specs,separate,"spec_join1"}. +{specs,separate,"../specs2/spec_join2"}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_both1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_both1 new file mode 100644 index 0000000000..46111614dc --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_both1 @@ -0,0 +1,2 @@ +{specs, join, "../specs1/flat_spec1"}. +{specs, separate, "../specs2/flat_spec2"}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_both_join1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_both_join1 new file mode 100644 index 0000000000..f52b3ed030 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_both_join1 @@ -0,0 +1,6 @@ +{specs, join, "../specs1/flat_spec1"}. +{specs, separate, "../specs2/flat_spec2"}. +{merge_tests,false}. +{config, "../config1/cfg12"}. +{suites, "../tests1", t11_SUITE}. +{suites, "../tests2", t21_SUITE}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_join1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_join1 new file mode 100644 index 0000000000..baaaf35be4 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_join1 @@ -0,0 +1 @@ +{specs, join, ["../specs2/flat_spec2", "flat_spec1"]}.
\ No newline at end of file diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_sep1 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_sep1 new file mode 100644 index 0000000000..89456c35e0 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs1/spec_sep1 @@ -0,0 +1,3 @@ +{specs, separate, "../specs2/flat_spec2"}. +{specs, separate, "flat_spec1"}. + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/flat_spec2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/flat_spec2 new file mode 100644 index 0000000000..758d1e2514 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/flat_spec2 @@ -0,0 +1,5 @@ +{merge_tests, false}. +{config, "../config2/cfg21"}. +{suites, "../tests1", t12_SUITE}. +{suites, "../tests1", t12_SUITE}. +{suites, "../tests2", t22_SUITE}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_join2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_join2 new file mode 100644 index 0000000000..19d3a3d8e2 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_join2 @@ -0,0 +1,5 @@ +{specs,join,"spec_join2"}. +{specs,join,"../specs1/spec_join1"}. + +{config, "../config1/cfg13"}. +{suites, "../tests2", t23_SUITE}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_join_sep2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_join_sep2 new file mode 100644 index 0000000000..930e68c847 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_join_sep2 @@ -0,0 +1,5 @@ +{specs,join,"spec_sep2"}. +{specs,join,"../specs1/spec_sep1"}. + +{config, "../config1/cfg13"}. +{suites, "../tests2", t23_SUITE}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_sep2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_sep2 new file mode 100644 index 0000000000..5026f329a7 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_sep2 @@ -0,0 +1,5 @@ +{specs,separate,"spec_sep2"}. +{specs,separate,"../specs1/spec_sep1"}. + +{config, "../config1/cfg13"}. +{suites, "../tests2", t23_SUITE}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_sep_join2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_sep_join2 new file mode 100644 index 0000000000..17057088b4 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/rec_spec_sep_join2 @@ -0,0 +1,5 @@ +{specs,separate,"spec_join2"}. +{specs,separate,"../specs1/spec_join1"}. + +{config, "../config1/cfg13"}. +{suites, "../tests2", t23_SUITE}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_both2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_both2 new file mode 100644 index 0000000000..4c83115d23 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_both2 @@ -0,0 +1,4 @@ +{specs, separate, "../specs1/flat_spec1"}. +{specs, join, "../specs2/flat_spec2"}. +{config, "../config1/cfg12"}. + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_both_join2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_both_join2 new file mode 100644 index 0000000000..ad81bfb4cc --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_both_join2 @@ -0,0 +1,9 @@ +{merge_tests,true}. +{config, "../config1/cfg12"}. +{suites, "../tests1", t11_SUITE}. +{suites, "../tests2", t21_SUITE}. +{suites, "../tests1", t12_SUITE}. +{suites, "../tests2", t22_SUITE}. + +{specs, separate, "../specs1/flat_spec1"}. +{specs, join, "../specs2/flat_spec2"}. diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_join2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_join2 new file mode 100644 index 0000000000..d652dbd78f --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_join2 @@ -0,0 +1,5 @@ +{specs, join, ["../specs1/flat_spec1"]}. +{specs, join, ["flat_spec2"]}. +{config, "../config1/cfg12"}. +{suites, "../tests2", t22_SUITE}. + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_sep2 b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_sep2 new file mode 100644 index 0000000000..8d37f508b8 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/specs2/spec_sep2 @@ -0,0 +1,5 @@ +{specs, separate, "../specs1/flat_spec1"}. +{specs, separate, "flat_spec2"}. +{config, "../config1/cfg12"}. +{suites, "../tests2", t22_SUITE}. + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/tests1/t11_SUITE.erl b/lib/common_test/test/ct_testspec_3_SUITE_data/tests1/t11_SUITE.erl new file mode 100644 index 0000000000..bbe79ed3fe --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/tests1/t11_SUITE.erl @@ -0,0 +1,175 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(t11_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% @spec suite() -> Info +%% Info = [tuple()] +%% @end +%%-------------------------------------------------------------------- +suite() -> + [{require,file}, + {require,tcname}, + {timetrap,{seconds,1}}]. + +%%-------------------------------------------------------------------- +%% @spec init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + %% verify that expected config file can be read + case {ct:get_config(tcname),ct:get_config(file,undefined,[all])} of + {start_separate,[cfg11]} -> ok; + {start_join,[cfg11,cfg21]} -> ok; + {incl_separate1,[cfg11]} -> ok; + {incl_separate2,[cfg11]} -> ok; + {incl_join1,[cfg21,cfg11]} -> ok; + {incl_join1,[cfg12,cfg11,cfg21]} -> ok; + {incl_join2,[cfg21,cfg11,cfg12]} -> ok; + {incl_both1,[cfg11]} -> ok; + {incl_both2,[cfg11,cfg12,cfg21]} -> ok; + {incl_both2,[cfg11]} -> ok; + _ -> ok + + end, + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_testcase(autoskip_tc, Config) -> + exit(kaboom), + Config; + +init_per_testcase(userskip_tc, Config) -> + {skip,"user skipped"}; + +init_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} | {fail,Reason} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%% @end +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% @spec all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +all() -> + [ok_tc, exit_tc, to_tc, autoskip_tc, userskip_tc]. + +%%-------------------------------------------------------------------- +%% @spec TestCase(Config0) -> +%% ok | exit() | {skip,Reason} | {comment,Comment} | +%% {save_config,Config1} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% Comment = term() +%% @end +%%-------------------------------------------------------------------- +ok_tc(_) -> + ok. + +exit_tc(_) -> + exit(kaboom), + ok. + +to_tc(_) -> + ct:timetrap(1), + ct:sleep(100), + ok. + +autoskip_tc(_) -> + ok. + +userskip_tc(_) -> + ok. + + + + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/tests1/t12_SUITE.erl b/lib/common_test/test/ct_testspec_3_SUITE_data/tests1/t12_SUITE.erl new file mode 100644 index 0000000000..810298d348 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/tests1/t12_SUITE.erl @@ -0,0 +1,175 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(t12_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% @spec suite() -> Info +%% Info = [tuple()] +%% @end +%%-------------------------------------------------------------------- +suite() -> + [{require,file}, + {require,tcname}, + {timetrap,{seconds,30}}]. + +%%-------------------------------------------------------------------- +%% @spec init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + %% verify that expected config file can be read + case {ct:get_config(tcname),ct:get_config(file,undefined,[all])} of + {start_separate,[cfg21]} -> ok; + {start_join,[cfg11,cfg21]} -> ok; + {incl_separate1,[cfg21]} -> ok; + {incl_separate2,[cfg21]} -> ok; + {incl_join1,[cfg21,cfg11]} -> ok; + {incl_join1,[cfg12,cfg11,cfg21]} -> ok; + {incl_join2,[cfg21,cfg11,cfg12]} -> ok; + {incl_both1,[cfg21]} -> ok; + {incl_both1,[cfg12,cfg21]} -> ok; + {incl_both2,[cfg11,cfg12,cfg21]} -> ok; + {incl_both2,[cfg21]} -> ok; + _ -> ok + end, + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_testcase(autoskip_tc, Config) -> + exit(kaboom), + Config; + +init_per_testcase(userskip_tc, Config) -> + {skip,"user skipped"}; + +init_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} | {fail,Reason} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%% @end +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% @spec all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +all() -> + [ok_tc, exit_tc, to_tc, autoskip_tc, userskip_tc]. + +%%-------------------------------------------------------------------- +%% @spec TestCase(Config0) -> +%% ok | exit() | {skip,Reason} | {comment,Comment} | +%% {save_config,Config1} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% Comment = term() +%% @end +%%-------------------------------------------------------------------- +ok_tc(_) -> + ok. + +exit_tc(_) -> + exit(kaboom), + ok. + +to_tc(_) -> + ct:timetrap(1), + ct:sleep(100), + ok. + +autoskip_tc(_) -> + ok. + +userskip_tc(_) -> + ok. + + + + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t21_SUITE.erl b/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t21_SUITE.erl new file mode 100644 index 0000000000..9348cd8caf --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t21_SUITE.erl @@ -0,0 +1,174 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(t21_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% @spec suite() -> Info +%% Info = [tuple()] +%% @end +%%-------------------------------------------------------------------- +suite() -> + [{require,file}, + {require,tcname}, + {timetrap,{seconds,1}}]. + +%%-------------------------------------------------------------------- +%% @spec init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + %% verify that expected config file can be read + case {ct:get_config(tcname),ct:get_config(file,undefined,[all])} of + {start_separate,[cfg11]} -> ok; + {start_join,[cfg11,cfg21]} -> ok; + {incl_separate1,[cfg11]} -> ok; + {incl_separate2,[cfg11]} -> ok; + {incl_join1,[cfg21,cfg11]} -> ok; + {incl_join1,[cfg12,cfg11,cfg21]} -> ok; + {incl_join2,[cfg21,cfg11,cfg12]} -> ok; + {incl_both1,[cfg11]} -> ok; + {incl_both2,[cfg11,cfg12,cfg21]} -> ok; + {incl_both2,[cfg11]} -> ok; + _ -> ok + end, + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_testcase(autoskip_tc, Config) -> + exit(kaboom), + Config; + +init_per_testcase(userskip_tc, Config) -> + {skip,"user skipped"}; + +init_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} | {fail,Reason} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%% @end +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% @spec all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +all() -> + [ok_tc, exit_tc, to_tc, autoskip_tc, userskip_tc]. + +%%-------------------------------------------------------------------- +%% @spec TestCase(Config0) -> +%% ok | exit() | {skip,Reason} | {comment,Comment} | +%% {save_config,Config1} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% Comment = term() +%% @end +%%-------------------------------------------------------------------- +ok_tc(_) -> + ok. + +exit_tc(_) -> + exit(kaboom), + ok. + +to_tc(_) -> + ct:timetrap(1), + ct:sleep(100), + ok. + +autoskip_tc(_) -> + ok. + +userskip_tc(_) -> + ok. + + + + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t22_SUITE.erl b/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t22_SUITE.erl new file mode 100644 index 0000000000..a92018ec70 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t22_SUITE.erl @@ -0,0 +1,177 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(t22_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% @spec suite() -> Info +%% Info = [tuple()] +%% @end +%%-------------------------------------------------------------------- +suite() -> + [{require,file}, + {require,tcname}, + {timetrap,{seconds,30}}]. + +%%-------------------------------------------------------------------- +%% @spec init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + %% verify that expected config file can be read + case {ct:get_config(tcname),ct:get_config(file,undefined,[all])} of + {start_separate,[cfg21]} -> ok; + {start_join,[cfg11,cfg21]} -> ok; + {incl_separate1,[cfg12]} -> ok; + {incl_separate1,[cfg21]} -> ok; + {incl_separate2,[cfg12]} -> ok; + {incl_separate2,[cfg21]} -> ok; + {incl_join1,[cfg21,cfg11]} -> ok; + {incl_join1,[cfg12,cfg11,cfg21]} -> ok; + {incl_join2,[cfg21,cfg11,cfg12]} -> ok; + {incl_both1,[cfg21]} -> ok; + {incl_both1,[cfg12,cfg21]} -> ok; + {incl_both2,[cfg11,cfg12,cfg21]} -> ok; + {incl_both2,[cfg21]} -> ok; + _ -> ok + end, + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_testcase(autoskip_tc, Config) -> + exit(kaboom), + Config; + +init_per_testcase(userskip_tc, Config) -> + {skip,"user skipped"}; + +init_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} | {fail,Reason} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%% @end +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% @spec all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +all() -> + [ok_tc, exit_tc, to_tc, autoskip_tc, userskip_tc]. + +%%-------------------------------------------------------------------- +%% @spec TestCase(Config0) -> +%% ok | exit() | {skip,Reason} | {comment,Comment} | +%% {save_config,Config1} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% Comment = term() +%% @end +%%-------------------------------------------------------------------- +ok_tc(_) -> + ok. + +exit_tc(_) -> + exit(kaboom), + ok. + +to_tc(_) -> + ct:timetrap(1), + ct:sleep(100), + ok. + +autoskip_tc(_) -> + ok. + +userskip_tc(_) -> + ok. + + + + diff --git a/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t23_SUITE.erl b/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t23_SUITE.erl new file mode 100644 index 0000000000..d01fac3144 --- /dev/null +++ b/lib/common_test/test/ct_testspec_3_SUITE_data/tests2/t23_SUITE.erl @@ -0,0 +1,158 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(t23_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% @spec suite() -> Info +%% Info = [tuple()] +%% @end +%%-------------------------------------------------------------------- +suite() -> + [{timetrap,{seconds,30}}]. + +%%-------------------------------------------------------------------- +%% @spec init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% @end +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +init_per_testcase(autoskip_tc, Config) -> + exit(kaboom), + Config; + +init_per_testcase(userskip_tc, Config) -> + {skip,"user skipped"}; + +init_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% @spec end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} | {fail,Reason} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% @spec groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%% @end +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% @spec all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%% @end +%%-------------------------------------------------------------------- +all() -> + [ok_tc, exit_tc, to_tc, autoskip_tc, userskip_tc]. + +%%-------------------------------------------------------------------- +%% @spec TestCase(Config0) -> +%% ok | exit() | {skip,Reason} | {comment,Comment} | +%% {save_config,Config1} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%% Comment = term() +%% @end +%%-------------------------------------------------------------------- +ok_tc(_) -> + ok. + +exit_tc(_) -> + exit(kaboom), + ok. + +to_tc(_) -> + ct:timetrap(1), + ct:sleep(100), + ok. + +autoskip_tc(_) -> + ok. + +userskip_tc(_) -> + ok. + + + + |