diff options
| -rw-r--r-- | lib/common_test/doc/src/Makefile | 4 | ||||
| -rw-r--r-- | lib/common_test/doc/src/common_test_app.xml | 12 | ||||
| -rw-r--r-- | lib/common_test/doc/src/ct_hooks.xml (renamed from lib/common_test/doc/src/ct_suite_callbacks.xml) | 179 | ||||
| -rw-r--r-- | lib/common_test/doc/src/ct_hooks_chapter.xml (renamed from lib/common_test/doc/src/suite_callbacks_chapter.xml) | 173 | ||||
| -rw-r--r-- | lib/common_test/doc/src/part.xml | 2 | ||||
| -rw-r--r-- | lib/common_test/doc/src/ref_man.xml | 2 | ||||
| -rw-r--r-- | lib/common_test/doc/src/run_test_chapter.xml | 14 |
7 files changed, 199 insertions, 187 deletions
diff --git a/lib/common_test/doc/src/Makefile b/lib/common_test/doc/src/Makefile index a3f740852c..a914dd0c19 100644 --- a/lib/common_test/doc/src/Makefile +++ b/lib/common_test/doc/src/Makefile @@ -52,7 +52,7 @@ CT_XML_FILES = $(CT_MODULES:=.xml) XML_APPLICATION_FILES = ref_man.xml XML_REF1_FILES = ct_run.xml -XML_REF3_FILES = $(CT_XML_FILES) ct_suite_callbacks.xml +XML_REF3_FILES = $(CT_XML_FILES) ct_hooks.xml XML_REF6_FILES = common_test_app.xml XML_PART_FILES = part.xml @@ -71,7 +71,7 @@ XML_CHAPTER_FILES = \ cover_chapter.xml \ ct_master_chapter.xml \ event_handler_chapter.xml \ - suite_callbacks_chapter.xml \ + ct_hooks_chapter.xml \ dependencies_chapter.xml \ notes.xml \ notes_history.xml diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml index a735dc85a8..b4e4b45d62 100644 --- a/lib/common_test/doc/src/common_test_app.xml +++ b/lib/common_test/doc/src/common_test_app.xml @@ -132,7 +132,7 @@ <v> Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} | - {suite_callbacks, SCBs}</v> + {ct_hooks, CTHs}</v> <v> Time = MilliSec | {seconds,integer()} | {minutes,integer()} | {hours,integer()}</v> <v> MilliSec = integer()</v> @@ -144,9 +144,9 @@ <v> UserData = term()</v> <v> Conns = [atom()]</v> <v> CSSFile = string()</v> - <v> SCBs = [SCBModule | {SCBModule, SCBInitArgs}]</v> - <v> SCBModule = atom()</v> - <v> SCBInitArgs = term()</v> + <v> CTHs = [CTHModule | {CTHModule, CTHInitArgs}]</v> + <v> CTHModule = atom()</v> + <v> CTHInitArgs = term()</v> </type> <desc> @@ -175,8 +175,8 @@ specify arbitrary test suite related information which can be read by calling <c>ct:userdata/2</c>.</p> - <p>The <c>suite_callbacks</c> tag specifies which - <seealso marker="suite_callbacks_chapter">Suite Callbacks</seealso> + <p>The <c>ct_hooks</c> tag specifies which + <seealso marker="ct_hooks_chapter">Common Test Hooks</seealso> are to be run together with this suite.</p> <p>Other tuples than the ones defined will simply be ignored.</p> diff --git a/lib/common_test/doc/src/ct_suite_callbacks.xml b/lib/common_test/doc/src/ct_hooks.xml index cea5804825..0d59ce3b22 100644 --- a/lib/common_test/doc/src/ct_suite_callbacks.xml +++ b/lib/common_test/doc/src/ct_hooks.xml @@ -22,7 +22,7 @@ </legalnotice> - <title>Suite Callbacks</title> + <title>Common Test Hooks</title> <prepared>Lukas Larsson</prepared> <responsible>Lukas Larsson</responsible> <docno></docno> @@ -30,9 +30,9 @@ <checked></checked> <date>2010-12-02</date> <rev>PA1</rev> - <file>suite_callback.sgml</file> + <file>ct_hooks.sgml</file> </header> - <module>ct_suite_callback</module> + <module>ct_hooks</module> <modulesummary>A callback interface on top of Common Test</modulesummary> <description> @@ -43,46 +43,46 @@ that there might be bugs and that the interface might change inbetween releases.</p></warning> - <p>The <em>Suite Callback</em> (henceforth called SCB) framework allows + <p>The <em>Common Test Hook</em> (henceforth called CTH) framework allows extensions of the default behaviour of Common Test by means of callbacks before and after all test suite calls. It is meant for advanced users of Common Test which want to abstract out behaviour which is common to multiple test suites. </p> - <p>In brief, Suite Callbacks allows you to:</p> + <p>In brief, Common Test Hooks allows you to:</p> <list> <item>Manipulate the runtime config before each suite - configuration calls</item> + configuration call</item> <item>Manipulate the return of all suite configuration calls and in extension the result of the test themselves.</item> </list> - <p>The following sections describe the mandatory and optional SCB + <p>The following sections describe the mandatory and optional CTH functions Common Test will call during test execution. For more details - see <seealso marker="suite_callbacks_chapter">Suite Callbacks</seealso> in + see <seealso marker="ct_hooks_chapter">Common Test Hooks</seealso> in the User's Guide.</p> - <p>For information about how to add a SCB to your suite see - <seealso marker="suite_callbacks_chapter#installing">Installing an SCB + <p>For information about how to add a CTH to your suite see + <seealso marker="ct_hooks_chapter#installing">Installing a CTH </seealso> in the User's Guide.</p> <note><p>See the - <seealso marker="suite_callbacks_chapter#example">Example SCB</seealso> - in the User's Guide for a minimal example of an SCB. </p></note> + <seealso marker="ct_hooks_chapter#example">Example CTH</seealso> + in the User's Guide for a minimal example of a CTH. </p></note> </description> <section> <title>CALLBACK FUNCTIONS</title> <p>The following functions define the callback interface - for a suite callback.</p> + for a Common Test Hook.</p> </section> <funcs> <func> <name>Module:init(Id, Opts) -> State</name> - <fsummary>Initiates the Suite Callback</fsummary> + <fsummary>Initiates the Common Test Hook</fsummary> <type> <v>Id = reference() | term()</v> <v>Opts = term()</v> @@ -94,29 +94,31 @@ <p>Always called before any other callback function. Use this to initiate any common state. - It should return a state for this SCB.</p> + It should return a state for this CTH.</p> <p><c>Id</c> is the return value of <seealso marker="#Module:id-1">id/1</seealso>, or a <c>reference</c> + (created using + <seealso marker="erts:erlang#make_ref-0">make_ref/0</seealso>) if <seealso marker="#Module:id-1">id/1</seealso> is not implemented. </p> <p>For details about when init is called see - <seealso marker="suite_callbacks_chapter#scope">scope</seealso> + <seealso marker="ct_hooks_chapter#scope">scope</seealso> in the User's Guide.</p> </desc> </func> <func> - <name>Module:pre_init_per_suite(SuiteName, Config, SCBState) -> + <name>Module:pre_init_per_suite(SuiteName, Config, CTHState) -> Result</name> <fsummary>Called before init_per_suite</fsummary> <type> <v>SuiteName = atom()</v> <v>Config = NewConfig = [{Key,Value}]</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {Return, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {Return, NewCTHState}</v> <v>Return = NewConfig | SkipOrFail</v> <v>SkipOrFail = {fail, Reason} | {skip, Reason}</v> <v>Key = atom()</v> @@ -140,7 +142,7 @@ <p><c>Config</c> is the original config list of the test suite.</p> - <p><c>SCBState</c> is the current internal state of the SCB.</p> + <p><c>CTHState</c> is the current internal state of the CTH.</p> <p><c>Return</c> is the result of the init_per_suite function. If it is <c>{skip,Reason}</c> or <c>{fail,Reason}</c> @@ -149,19 +151,19 @@ to be skipped/failed respectively. If a <c>NewConfig</c> list is returned, <seealso marker="common_test#Module:init_per_suite-1"> init_per_suite</seealso> will be called with that <c>NewConfig</c> list. - See <seealso marker="suite_callbacks_chapter#manipulating"> - Manipulating tests</seealso> in the User's Guide for more details.</p> + See <seealso marker="ct_hooks_chapter#pre"> + Pre Hooks</seealso> in the User's Guide for more details.</p> - <p>Note that this function is only called if the SCB has been added + <p>Note that this function is only called if the CTH has been added before init_per_suite is run, see - <seealso marker="suite_callbacks_chapter#scope">SCB Scoping</seealso> + <seealso marker="ct_hooks_chapter#scope">CTH Scoping</seealso> in the User's Guide for details.</p> </desc> </func> <func> - <name>Module:post_init_per_suite(SuiteName, Config, Return, SCBState) -> + <name>Module:post_init_per_suite(SuiteName, Config, Return, CTHState) -> Result</name> <fsummary>Called after init_per_suite</fsummary> <type> @@ -169,8 +171,8 @@ <v>Config = [{Key,Value}]</v> <v>Return = NewReturn = Config | SkipOrFail | term()</v> <v>SkipOrFail = {fail, Reason} | {skip, Reason} | term()</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewReturn, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> <v>Key = atom()</v> <v>Value = term()</v> <v>Reason = term()</v> @@ -178,6 +180,12 @@ <desc> <p> OPTIONAL </p> + + <p>This function is called after + <seealso marker="common_test#Module:init_per_suite-1"> + init_per_suite</seealso> if it exists. It typically contains extra + checks to make sure that all the correct dependencies have + been started correctly.</p> <p><c>Return</c> is what <seealso marker="common_test#Module:init_per_suite-1">init_per_suite @@ -191,30 +199,27 @@ </seealso>. It is here possible to recover from a failure in <seealso marker="common_test#Module:init_per_suite-1">init_per_suite </seealso> by returning the <c>ConfigList</c> with the <c>tc_status</c> - element removed.</p> + element removed. See <seealso marker="ct_hooks_chapter#post"> + Post Hooks</seealso> in the User's Guide for more details.</p> - <p><c>SCBState</c> is the current internal state of the SCB.</p> - - <p>This function is called after - <seealso marker="common_test#Module:init_per_suite-1"> - init_per_suite</seealso> if it exists.</p> + <p><c>CTHState</c> is the current internal state of the CTH.</p> - <p>Note that this function is only called if the SCB has been added + <p>Note that this function is only called if the CTH has been added before or in init_per_suite, see - <seealso marker="suite_callbacks_chapter#scope">SCB Scoping</seealso> + <seealso marker="ct_hooks_chapter#scope">CTH Scoping</seealso> in the User's Guide for details.</p> </desc> </func> <func> - <name>Module:pre_init_per_group(GroupName, Config, SCBState) -> + <name>Module:pre_init_per_group(GroupName, Config, CTHState) -> Result</name> <fsummary>Called before init_per_group</fsummary> <type> <v>GroupName = atom()</v> <v>Config = NewConfig = [{Key,Value}]</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewConfig | SkipOrFail, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v> <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> <v>Key = atom()</v> <v>Value = term()</v> @@ -227,7 +232,7 @@ <p>This function is called before <seealso marker="common_test#Module:init_per_group-2"> init_per_group</seealso> if it exists. It behaves the same way as - <seealso marker="ct_suite_callback#Module:pre_init_per_suite-3"> + <seealso marker="ct_hooks#Module:pre_init_per_suite-3"> pre_init_per_suite</seealso>, but for the <seealso marker="common_test#Module:init_per_group-2"> init_per_group</seealso> instead.</p> @@ -235,7 +240,7 @@ </func> <func> - <name>Module:post_init_per_group(GroupName, Config, Return, SCBState) -> + <name>Module:post_init_per_group(GroupName, Config, Return, CTHState) -> Result</name> <fsummary>Called after init_per_group</fsummary> <type> @@ -243,8 +248,8 @@ <v>Config = [{Key,Value}]</v> <v>Return = NewReturn = Config | SkipOrFail | term()</v> <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewReturn, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> <v>Key = atom()</v> <v>Value = term()</v> <v>Reason = term()</v> @@ -256,7 +261,7 @@ <p>This function is called after <seealso marker="common_test#Module:init_per_group-2"> init_per_group</seealso> if it exists. It behaves the same way as - <seealso marker="ct_suite_callback#Module:post_init_per_suite-4"> + <seealso marker="ct_hooks#Module:post_init_per_suite-4"> post_init_per_suite</seealso>, but for the <seealso marker="common_test#Module:init_per_group-2"> init_per_group</seealso> instead.</p> @@ -264,14 +269,14 @@ </func> <func> - <name>Module:pre_init_per_testcase(TestcaseName, Config, SCBState) -> + <name>Module:pre_init_per_testcase(TestcaseName, Config, CTHState) -> Result</name> <fsummary>Called before init_per_testcase</fsummary> <type> <v>TestcaseName = atom()</v> <v>Config = NewConfig = [{Key,Value}]</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewConfig | SkipOrFail, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v> <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> <v>Key = atom()</v> <v>Value = term()</v> @@ -284,19 +289,19 @@ <p>This function is called before <seealso marker="common_test#Module:init_per_testcase-2"> init_per_testcase</seealso> if it exists. It behaves the same way as - <seealso marker="ct_suite_callback#Module:pre_init_per_suite-3"> + <seealso marker="ct_hooks#Module:pre_init_per_suite-3"> pre_init_per_suite</seealso>, but for the <seealso marker="common_test#Module:init_per_testcase-2"> init_per_testcase</seealso> function instead.</p> - <p>Note that it is not possible to add SCB's here right now, + <p>Note that it is not possible to add CTH's here right now, that feature might be added later, but it would right now break backwards compatability.</p> </desc> </func> <func> - <name>Module:post_end_per_testcase(TestcaseName, Config, Return, SCBState) + <name>Module:post_end_per_testcase(TestcaseName, Config, Return, CTHState) -> Result</name> <fsummary>Called after end_per_testcase</fsummary> <type> @@ -304,8 +309,8 @@ <v>Config = [{Key,Value}]</v> <v>Return = NewReturn = Config | SkipOrFail | term()</v> <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewReturn, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> <v>Key = atom()</v> <v>Value = term()</v> <v>Reason = term()</v> @@ -317,7 +322,7 @@ <p>This function is called after <seealso marker="common_test#Module:end_per_testcase-2"> end_per_testcase</seealso> if it exists. It behaves the same way as - <seealso marker="ct_suite_callback#Module:post_init_per_suite-4"> + <seealso marker="ct_hooks#Module:post_init_per_suite-4"> post_init_per_suite</seealso>, but for the <seealso marker="common_test#Module:end_per_testcase-2"> end_per_testcase</seealso> function instead.</p> @@ -325,14 +330,14 @@ </func> <func> - <name>Module:pre_end_per_group(GroupName, Config, SCBState) -> + <name>Module:pre_end_per_group(GroupName, Config, CTHState) -> Result</name> <fsummary>Called before end_per_group</fsummary> <type> <v>GroupName = atom()</v> <v>Config = NewConfig = [{Key,Value}]</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewConfig | SkipOrFail, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v> <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> <v>Key = atom()</v> <v>Value = term()</v> @@ -345,7 +350,7 @@ <p>This function is called before <seealso marker="common_test#Module:end_per_group-2"> end_per_group</seealso> if it exists. It behaves the same way as - <seealso marker="ct_suite_callback#Module:pre_init_per_suite-3"> + <seealso marker="ct_hooks#Module:pre_init_per_suite-3"> pre_init_per_suite</seealso>, but for the <seealso marker="common_test#Module:end_per_group-2"> end_per_group</seealso> function instead.</p> @@ -353,7 +358,7 @@ </func> <func> - <name>Module:post_end_per_group(GroupName, Config, Return, SCBState) -> + <name>Module:post_end_per_group(GroupName, Config, Return, CTHState) -> Result</name> <fsummary>Called after end_per_group</fsummary> <type> @@ -361,8 +366,8 @@ <v>Config = [{Key,Value}]</v> <v>Return = NewReturn = Config | SkipOrFail | term()</v> <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewReturn, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> <v>Key = atom()</v> <v>Value = term()</v> <v>Reason = term()</v> @@ -374,7 +379,7 @@ <p>This function is called after <seealso marker="common_test#Module:end_per_group-2"> end_per_group</seealso> if it exists. It behaves the same way as - <seealso marker="ct_suite_callback#Module:post_init_per_suite-4"> + <seealso marker="ct_hooks#Module:post_init_per_suite-4"> post_init_per_suite</seealso>, but for the <seealso marker="common_test#Module:end_per_group-2"> end_per_group</seealso> function instead.</p> @@ -382,14 +387,14 @@ </func> <func> - <name>Module:pre_end_per_suite(SuiteName, Config, SCBState) -> + <name>Module:pre_end_per_suite(SuiteName, Config, CTHState) -> Result</name> <fsummary>Called before end_per_suite</fsummary> <type> <v>SuiteName = atom()</v> <v>Config = NewConfig = [{Key,Value}]</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewConfig | SkipOrFail, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v> <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> <v>Key = atom()</v> <v>Value = term()</v> @@ -402,7 +407,7 @@ <p>This function is called before <seealso marker="common_test#Module:end_per_suite-1"> end_per_suite</seealso> if it exists. It behaves the same way as - <seealso marker="ct_suite_callback#Module:pre_init_per_suite-3"> + <seealso marker="ct_hooks#Module:pre_init_per_suite-3"> pre_init_per_suite</seealso>, but for the <seealso marker="common_test#Module:end_per_suite-2"> end_per_suite</seealso> function instead.</p> @@ -410,7 +415,7 @@ </func> <func> - <name>Module:post_end_per_suite(SuiteName, Config, Return, SCBState) -> + <name>Module:post_end_per_suite(SuiteName, Config, Return, CTHState) -> Result</name> <fsummary>Called after end_per_suite</fsummary> <type> @@ -418,8 +423,8 @@ <v>Config = [{Key,Value}]</v> <v>Return = NewReturn = Config | SkipOrFail | term()</v> <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> - <v>SCBState = NewSCBState = term()</v> - <v>Result = {NewReturn, NewSCBState}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> <v>Key = atom()</v> <v>Value = term()</v> <v>Reason = term()</v> @@ -431,7 +436,7 @@ <p>This function is called after <seealso marker="common_test#Module:end_per_suite-1"> end_per_suite</seealso> if it exists. It behaves the same way as - <seealso marker="ct_suite_callback#Module:post_init_per_suite-4"> + <seealso marker="ct_hooks#Module:post_init_per_suite-4"> post_init_per_suite</seealso>, but for the <seealso marker="common_test#Module:end_per_suite-2"> end_per_suite</seealso> function instead.</p> @@ -439,14 +444,14 @@ </func> <func> - <name>Module:on_tc_fail(TestcaseName, Reason, SCBState) -> - NewSCBState</name> - <fsummary>Called after the SCB scope ends</fsummary> + <name>Module:on_tc_fail(TestcaseName, Reason, CTHState) -> + NewCTHState</name> + <fsummary>Called after the CTH scope ends</fsummary> <type> <v>TestcaseName = init_per_suite | end_per_suite | init_per_group | end_per_group | atom()</v> <v>Reason = term()</v> - <v>SCBState = NewSCBState = term()</v> + <v>CTHState = NewCTHState = term()</v> </type> <desc> @@ -470,14 +475,14 @@ </func> <func> - <name>Module:on_tc_skip(TestcaseName, Reason, SCBState) -> - NewSCBState</name> - <fsummary>Called after the SCB scope ends</fsummary> + <name>Module:on_tc_skip(TestcaseName, Reason, CTHState) -> + NewCTHState</name> + <fsummary>Called after the CTH scope ends</fsummary> <type> <v>TestcaseName = end_per_suite | init_per_group | end_per_group | atom()</v> <v>Reason = {tc_auto_skip | tc_user_skip, term()}</v> - <v>SCBState = NewSCBState = term()</v> + <v>CTHState = NewCTHState = term()</v> </type> <desc> @@ -502,24 +507,24 @@ </func> <func> - <name>Module:terminate(SCBState)</name> - <fsummary>Called after the SCB scope ends</fsummary> + <name>Module:terminate(CTHState)</name> + <fsummary>Called after the CTH scope ends</fsummary> <type> - <v>SCBState = term()</v> + <v>CTHState = term()</v> </type> <desc> <p> OPTIONAL </p> - <p>This function is called at the end of an SCB's - <seealso marker="suite_callbacks_chapter#scope">scope</seealso>. + <p>This function is called at the end of a CTH's + <seealso marker="ct_hooks_chapter#scope">scope</seealso>. </p> </desc> </func> <func> <name>Module:id(Opts) -> Id</name> - <fsummary>Called before the init function of an SCB</fsummary> + <fsummary>Called before the init function of a CTH</fsummary> <type> <v>Opts = term()</v> <v>Id = term()</v> @@ -528,18 +533,18 @@ <desc> <p> OPTIONAL </p> - <p>The <c>Id</c> is used to uniquely identify an SCB instance, - if two SCB's return the same <c>Id</c> the second SCB is ignored - and subsequent calls to the SCB will only be made to the first + <p>The <c>Id</c> is used to uniquely identify a CTH instance, + if two CTH's return the same <c>Id</c> the second CTH is ignored + and subsequent calls to the CTH will only be made to the first instance. For more information see - <seealso marker="suite_callbacks_chapter#installing">Installing an SCB + <seealso marker="ct_hooks_chapter#installing">Installing a CTH </seealso> in the User's Guide. </p> <p>This function should NOT have any side effects as it might be called multiple times by Common Test.</p> - <p>If not implemented the SCB will act as if this function returned a + <p>If not implemented the CTH will act as if this function returned a call to <c>make_ref/0</c>.</p> </desc> </func> diff --git a/lib/common_test/doc/src/suite_callbacks_chapter.xml b/lib/common_test/doc/src/ct_hooks_chapter.xml index 89f78898d4..fc5ab48e1b 100644 --- a/lib/common_test/doc/src/suite_callbacks_chapter.xml +++ b/lib/common_test/doc/src/ct_hooks_chapter.xml @@ -21,12 +21,12 @@ </legalnotice> - <title>Suite Callbacks</title> + <title>Common Test Hooks</title> <prepared>Lukas Larsson</prepared> <docno></docno> <date></date> <rev></rev> - <file>suite_callbacks_chapter.xml</file> + <file>ct_hooks_chapter.xml</file> </header> <marker id="general"></marker> @@ -38,74 +38,75 @@ that there might be bugs and that the interface might change inbetween releases.</p></warning> <p> - The <em>Suite Callback</em> (henceforth called SCB) framework allows - extensions of the default behaviour of Common Test by means of callbacks - before and after all test suite calls. SCBs allow advanced Common Test + The <em>Common Test Hook</em> (henceforth called CTH) framework allows + extensions of the default behaviour of Common Test by means of hooks + before and after all test suite calls. CTHs allow advanced Common Test users to abstract out behaviour which is common to multiple test suites without littering all test suites with library calls. Some example usages are: logging, starting and monitoring external systems, building C files needed by the tests and much more!</p> - <p>In brief, Suite Callbacks allows you to:</p> + <p>In brief, Common Test Hooks allows you to:</p> <list> <item>Manipulate the runtime config before each suite - configuration calls</item> + configuration call</item> <item>Manipulate the return of all suite configuration calls and in extension the result of the test themselves.</item> </list> - <p>The following sections describe how to use SCBs, when they are run - and how to manipulate your test results in an SCB</p> + <p>The following sections describe how to use CTHs, when they are run + and how to manipulate your test results in a CTH</p> - <warning><p>When executing within an SCB all timetraps are shutoff. So - if your SCB never returns, the entire test run will be stalled!</p> + <warning><p>When executing within a CTH all timetraps are shutoff. So + if your CTH never returns, the entire test run will be stalled!</p> </warning> </section> <marker id="installing"></marker> <section> - <title>Installing an SCB</title> - <p>There are multiple ways to install an SCB in your test run. You can do it + <title>Installing a CTH</title> + <p>There are multiple ways to install a CTH in your test run. You can do it for all tests in a run, for specific test suites and for specific groups - within a test suite. If you want an SCB to be present in all test suites - within your testrun there are three different ways to accomplish that.</p> + within a test suite. If you want a CTH to be present in all test suites + within your test run there are three different ways to accomplish that. + </p> <list> - <item>Add <c>-suite_callbacks</c> as an argument to + <item>Add <c>-ct_hooks</c> as an argument to <seealso marker="run_test_chapter#ct_run">ct_run</seealso>. - To add multiple SCBs using this method append them to each other + To add multiple CTHs using this method append them to each other using the keyword <c>and</c>, i.e. - <c>ct_run -suite_callbacks scb1 [{debug,true}] and scb2 ...</c>.</item> - <item>Add the <c>suite_callbacks</c> tag to your + <c>ct_run -ct_hooks cth1 [{debug,true}] and cth2 ...</c>.</item> + <item>Add the <c>ct_hooks</c> tag to your <seealso marker="run_test_chapter#test_specifications"> Test Specification</seealso></item> - <item>Add the <c>suite_callbacks</c> tag to your call to + <item>Add the <c>ct_hooks</c> tag to your call to <seealso marker="ct#run_test-1">ct:run_test/1</seealso></item> </list> - <p>You can also add SCBs within a test suite. This is done by returning - <c>{suite_callbacks,[SCB]}</c> in the config list from + <p>You can also add CTHs within a test suite. This is done by returning + <c>{ct_hooks,[CTH]}</c> in the config list from <seealso marker="common_test#Module:suite-0">suite/0</seealso>, <seealso marker="common_test#Module:init_per_suite-1"> init_per_suite/1</seealso> or <seealso marker="common_test#Module:init_per_group-2"> - init_per_group/2</seealso>. <c>SCB</c> in this case can be either - only the module name of the SCB or a tuple with the module name and the - initial arguments to the SCB. Eg: - <c>{suite_callbacks,[my_scb_module]}</c> or - <c>{suite_callbacks,[{my_scb_module,[{debug,true}]}]}</c></p> + init_per_group/2</seealso>. <c>CTH</c> in this case can be either + only the module name of the CTH or a tuple with the module name and the + initial arguments to the CTH. Eg: + <c>{ct_hooks,[my_cth_module]}</c> or + <c>{ct_hooks,[{my_cth_module,[{debug,true}]}]}</c></p> <section> - <title>Overriding SCBs</title> - <p>By default each installation of an SCB will cause a new instance of it + <title>Overriding CTHs</title> + <p>By default each installation of a CTH will cause a new instance of it to be activated. This can cause problems if you want to be able to - override SCBs in testspecifications while still having them in the + override CTHs in test specifications while still having them in the suite info function. The - <seealso marker="ct_suite_callback#Module:id-1">id/1</seealso> + <seealso marker="ct_hooks#Module:id-1">id/1</seealso> callback exists to address this problem. By returning the same - <c>id</c> in both places, Common Test knows that this SCB + <c>id</c> in both places, Common Test knows that this CTH has already been installed and will not try to install it again.</p> </section> @@ -113,19 +114,19 @@ <marker id="scope"/> <section> - <title>SCB Scope</title> - <p>Once the SCB is installed into a certain test run it will be there until - it's scope is expired. The scope of an SCB depends on when it is + <title>CTH Scope</title> + <p>Once the CTH is installed into a certain test run it will be there until + its scope is expired. The scope of a CTH depends on when it is installed. - The <seealso marker="ct_suite_callback#Module:init-2">init/2</seealso> is + The <seealso marker="ct_hooks#Module:init-2">init/2</seealso> is called at the beginning of the scope and the - <seealso marker="ct_suite_callback#Module:terminate-1">terminate/1 + <seealso marker="ct_hooks#Module:terminate-1">terminate/1 </seealso> function is called when the scope ends.</p> <table> <row> - <cell><em>SCB Installed in</em></cell> - <cell><em>SCB scope begins before</em></cell> - <cell><em>SCB scope ends after</em></cell> + <cell><em>CTH Installed in</em></cell> + <cell><em>CTH scope begins before</em></cell> + <cell><em>CTH scope ends after</em></cell> </row> <row> <cell><seealso marker="run_test_chapter#ct_run">ct_run</seealso></cell> @@ -146,28 +147,28 @@ <row> <cell><seealso marker="common_test#Module:suite-0">suite/0 </seealso></cell> - <cell><seealso marker="ct_suite_callback#Module:pre_init_per_suite-3"> + <cell><seealso marker="ct_hooks#Module:pre_init_per_suite-3"> pre_init_per_suite/3</seealso> is called.</cell> - <cell><seealso marker="ct_suite_callback#Module:post_end_per_suite-4"> + <cell><seealso marker="ct_hooks#Module:post_end_per_suite-4"> post_end_per_suite/4</seealso> has been called for that test suite.</cell> </row> <row> <cell><seealso marker="common_test#Module:init_per_suite-1"> init_per_suite/1</seealso></cell> - <cell><seealso marker="ct_suite_callback#Module:post_init_per_suite-4"> + <cell><seealso marker="ct_hooks#Module:post_init_per_suite-4"> post_init_per_suite/4</seealso> is called.</cell> - <cell><seealso marker="ct_suite_callback#Module:post_end_per_suite-4"> + <cell><seealso marker="ct_hooks#Module:post_end_per_suite-4"> post_end_per_suite/4</seealso> has been called for that test suite.</cell> </row> <row> <cell><seealso marker="common_test#Module:init_per_group-2"> init_per_group/2</seealso></cell> - <cell><seealso marker="ct_suite_callback#Module:post_init_per_group-4"> + <cell><seealso marker="ct_hooks#Module:post_init_per_group-4"> post_init_per_group/4</seealso> is called.</cell> - <cell><seealso marker="ct_suite_callback#Module:post_end_per_suite-4"> + <cell><seealso marker="ct_hooks#Module:post_end_per_suite-4"> post_end_per_group/4</seealso> has been called for that group.</cell> </row> - <tcaption>Scope of an SCB</tcaption> + <tcaption>Scope of a CTH</tcaption> </table> <section> @@ -186,83 +187,89 @@ <marker id="manipulating"/> <section> <title>Manipulating tests</title> - <p>It is through SCB's possible to manipulate the results of tests and - configuration functions. The main purpose of doing this with SCBs is to + <p>It is through CTHs possible to manipulate the results of tests and + configuration functions. The main purpose of doing this with CTHs is to allow common patterns to be abstracted out from test test suites and applied to multiple test suites without duplicating any code. All of the callback - functions for an SCB follow a common interface, this interface is + functions for a CTH follow a common interface, this interface is described below.</p> + <p>It is only possible to hook into test function which exists in the test + suite. So in order for a CTH to hook in before + <seealso marker="common_test#Module:init_per_suite-1">init_per_suite</seealso>, + the <seealso marker="common_test#Module:init_per_suite-1">init_per_suite</seealso> + function must exist in the test suite.</p> + <marker id="pre"/> <section> - <title>Pre test manipulation</title> + <title>Pre Hooks</title> <p> - It is possible in an SCB to hook in behaviour before + It is possible in a CTH to hook in behaviour before <seealso marker="common_test#Module:init_per_suite-1">init_per_suite</seealso>, <seealso marker="common_test#Module:init_per_suite-1">init_per_group</seealso>, <seealso marker="common_test#Module:init_per_suite-1">init_per_testcase</seealso>, <seealso marker="common_test#Module:init_per_suite-1">end_per_group</seealso> and <seealso marker="common_test#Module:init_per_suite-1">end_per_suite</seealso>. - This is done in the SCB functions called pre_<name of function>. - All of these function take the same three arguments: <c>Name</c>, - <c>Config</c> and <c>SCBState</c>. The return value of the SCB function + This is done in the CTH functions called pre_<name of function>. + All of these functions take the same three arguments: <c>Name</c>, + <c>Config</c> and <c>CTHState</c>. The return value of the CTH function is always a combination of an result for the suite/group/test and an - updated <c>SCBState</c>. If you want the test suite to continue on + updated <c>CTHState</c>. If you want the test suite to continue on executing you should return the config list which you want the test to use as the result. If you for some reason want to skip/fail the test, return a tuple with <c>skip</c> or <c>fail</c> and a reason as the result. Example: </p> - <code>pre_init_per_suite(SuiteName, Config, SCBState) -> + <code>pre_init_per_suite(SuiteName, Config, CTHState) -> case db:connect() of {error,_Reason} -> - {{fail, "Could not connect to DB"}, SCBState}; + {{fail, "Could not connect to DB"}, CTHState}; {ok, Handle} -> - {[{db_handle, Handle} | Config], SCBState#state{ handle = Handle }} + {[{db_handle, Handle} | Config], CTHState#state{ handle = Handle }} end.</code> </section> <marker id="post"/> <section> - <title>Post test manipulation</title> - <p>It is also possible in an SCB to hook in behaviour after + <title>Post Hooks</title> + <p>It is also possible in a CTH to hook in behaviour after <seealso marker="common_test#Module:init_per_suite-1">init_per_suite</seealso>, <seealso marker="common_test#Module:init_per_suite-1">init_per_group</seealso>, <seealso marker="common_test#Module:init_per_suite-1">end_per_testcase</seealso>, <seealso marker="common_test#Module:init_per_suite-1">end_per_group</seealso> and <seealso marker="common_test#Module:init_per_suite-1">end_per_suite</seealso>. - This is done in the SCB functions called post_<name of function>. + This is done in the CTH functions called post_<name of function>. All of these function take the same four arguments: <c>Name</c>, - <c>Config</c>, <c>Return</c> and <c>SCBState</c>. <c>Config</c> in this + <c>Config</c>, <c>Return</c> and <c>CTHState</c>. <c>Config</c> in this case is the same <c>Config</c> as the testcase is called with. <c>Return</c> is the value returned by the testcase. If the testcase failed by crashing, <c>Return</c> will be <c>{'EXIT',{{Error,Reason},Stacktrace}}</c>.</p> - <p>The return value of the SCB function is always a combination of an - result for the suite/group/test and an updated <c>SCBState</c>. If + <p>The return value of the CTH function is always a combination of an + result for the suite/group/test and an updated <c>CTHState</c>. If you want the callback to not affect the outcome of the test you should - return the <c>Return</c> data as it is given to the SCB. You can also + return the <c>Return</c> data as it is given to the CTH. You can also modify the result of the test. By returning the <c>Config</c> list with the <c>tc_status</c> element removed you can recover from a test failure. As in all the pre hooks, it is also possible to fail/skip the test case in the post hook. Example: </p> - <code>post_end_per_testcase(_TC, Config, {'EXIT',{_,_}}, SCBState) -> + <code>post_end_per_testcase(_TC, Config, {'EXIT',{_,_}}, CTHState) -> case db:check_consistency() of true -> %% DB is good, pass the test. - {proplists:delete(tc_status, Config), SCBState}; + {proplists:delete(tc_status, Config), CTHState}; false -> %% DB is not good, mark as skipped instead of failing - {{skip, "DB is inconsisten!"}, SCBState} + {{skip, "DB is inconsisten!"}, CTHState} end; -post_end_per_testcase(_TC, Config, Return, SCBState) -> +post_end_per_testcase(_TC, Config, Return, CTHState) -> %% Do nothing if tc does not crash. - {Return, SCBState}.</code> + {Return, CTHState}.</code> - <note>Recovering from a testcase failure using SCBs should only be done as + <note>Recovering from a testcase failure using CTHs should only be done as a last resort. If used wrongly it could become very difficult to determine which tests pass or fail in a test run</note> @@ -270,11 +277,11 @@ post_end_per_testcase(_TC, Config, Return, SCBState) -> <marker id="skip_n_fail"/> <section> - <title>Skip and Fail</title> + <title>Skip and Fail hooks</title> <p> - After any post hook has been executed for all installed SCBs, - <seealso marker="ct_suite_callback#Module:on_tc_fail-3">on_tc_fail</seealso> - or <seealso marker="ct_suite_callback#Module:on_tc_fail-3">on_tc_skip</seealso> + After any post hook has been executed for all installed CTHs, + <seealso marker="ct_hooks#Module:on_tc_fail-3">on_tc_fail</seealso> + or <seealso marker="ct_hooks#Module:on_tc_fail-3">on_tc_skip</seealso> might be called if the testcase failed or was skipped respectively. You cannot affect the outcome of the tests any further at this point. @@ -285,14 +292,14 @@ post_end_per_testcase(_TC, Config, Return, SCBState) -> <marker id="example"/> <section> - <title>Example SCB</title> - <p>The SCB below will log information about a test run into a format + <title>Example CTH</title> + <p>The CTH below will log information about a test run into a format parseable by <seealso marker="kernel:file#consult-1">file:consult/1</seealso>. </p> - <code>%%% @doc Common Test Example Suite Callback module. --module(example_scb). + <code>%%% @doc Common Test Example Common Test Hook module. +-module(example_cth). -%% Suite Callbacks +%% Callbacks -export([id/1]). -export([init/2]). @@ -316,7 +323,7 @@ post_end_per_testcase(_TC, Config, Return, SCBState) -> -record(state, { file_handle, total, suite_total, ts, tcs, data }). -%% @doc Return a unique id for this SCB. +%% @doc Return a unique id for this CTH. id(Opts) -> proplists:get_value(filename, Opts, "/tmp/file.log"). @@ -379,7 +386,7 @@ on_tc_fail(TC, Reason, State) -> on_tc_skip(TC, Reason, State) -> State. -%% @doc Called when the scope of the SCB is done +%% @doc Called when the scope of the CTH is done terminate(State) -> io:format(State#state.file_handle, "~p.~n", [{test_run, State#state.total, State#state.data}]), diff --git a/lib/common_test/doc/src/part.xml b/lib/common_test/doc/src/part.xml index 1a09ec1da2..41371b60be 100644 --- a/lib/common_test/doc/src/part.xml +++ b/lib/common_test/doc/src/part.xml @@ -75,7 +75,7 @@ <xi:include href="ct_master_chapter.xml"/> <xi:include href="event_handler_chapter.xml"/> <xi:include href="dependencies_chapter.xml"/> - <xi:include href="suite_callbacks_chapter.xml"/> + <xi:include href="ct_hooks_chapter.xml"/> <xi:include href="why_test_chapter.xml"/> </part> diff --git a/lib/common_test/doc/src/ref_man.xml b/lib/common_test/doc/src/ref_man.xml index 100c0fe5d7..631e3871c2 100644 --- a/lib/common_test/doc/src/ref_man.xml +++ b/lib/common_test/doc/src/ref_man.xml @@ -76,7 +76,7 @@ <xi:include href="ct_telnet.xml"/> <xi:include href="unix_telnet.xml"/> <xi:include href="ct_slave.xml"/> - <xi:include href="ct_suite_callbacks.xml"/> + <xi:include href="ct_hooks.xml"/> </application> diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index 81e752680b..7b485bdc35 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -148,8 +148,8 @@ <seealso marker="event_handler_chapter#event_handling">event handlers</seealso>.</item> <item><c><![CDATA[-event_handler_init <event_handlers>]]></c>, to install <seealso marker="event_handler_chapter#event_handling">event handlers</seealso> including start arguments.</item> - <item><c><![CDATA[-suite_callbacks <suite_callbacks>]]></c>, to install - <seealso marker="suite_callbacks_chapter#installing">Suite Callbacks</seealso> including start arguments.</item> + <item><c><![CDATA[-ct_hooks <ct_hooks>]]></c>, to install + <seealso marker="ct_hooks_chapter#installing">Common Test Hooks</seealso> including start arguments.</item> <item><c><![CDATA[-include]]></c>, specifies include directories (see above).</item> <item><c><![CDATA[-no_auto_compile]]></c>, disables the automatic test suite compilation feature (see above).</item> <item><c><![CDATA[-multiply_timetraps <n>]]></c>, extends <seealso marker="write_test_chapter#timetraps">timetrap @@ -444,8 +444,8 @@ {event_handler, EventHandlers, InitArgs}. {event_handler, NodeRefs, EventHandlers, InitArgs}. - {suite_callbacks, SCBModules}. - {suite_callbacks, NodeRefs, SCBModules}. + {ct_hooks, CTHModules}. + {ct_hooks, NodeRefs, CTHModules}. </pre> <p>Test terms:</p> <pre> @@ -484,9 +484,9 @@ LogDir = string() EventHandlers = atom() | [atom()] InitArgs = [term()] - SCBModules = [SCBModule | {SCBModule, SCBInitArgs}] - SCBModule = atom() - SCBInitArgs = term() + CTHModules = [CTHModule | {CTHModule, CTHInitArgs}] + CTHModule = atom() + CTHInitArgs = term() DirRef = DirAlias | Dir Suites = atom() | [atom()] | all Suite = atom() |