aboutsummaryrefslogtreecommitdiffstats
path: root/lib/common_test/doc/src/ct_hooks.xml
diff options
context:
space:
mode:
Diffstat (limited to 'lib/common_test/doc/src/ct_hooks.xml')
-rw-r--r--lib/common_test/doc/src/ct_hooks.xml556
1 files changed, 556 insertions, 0 deletions
diff --git a/lib/common_test/doc/src/ct_hooks.xml b/lib/common_test/doc/src/ct_hooks.xml
new file mode 100644
index 0000000000..0d59ce3b22
--- /dev/null
+++ b/lib/common_test/doc/src/ct_hooks.xml
@@ -0,0 +1,556 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+
+<!DOCTYPE erlref SYSTEM "erlref.dtd">
+
+<erlref>
+ <header>
+ <copyright>
+ <year>2010</year><year>2010</year>
+ <holder>Ericsson AB. All Rights Reserved.</holder>
+ </copyright>
+ <legalnotice>
+ The contents of this file are subject to the Erlang Public License,
+ Version 1.1, (the "License"); you may not use this file except in
+ compliance with the License. You should have received a copy of the
+ Erlang Public License along with this software. If not, it can be
+ retrieved online at http://www.erlang.org/.
+
+ Software distributed under the License is distributed on an "AS IS"
+ basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
+ the License for the specific language governing rights and limitations
+ under the License.
+
+ </legalnotice>
+
+ <title>Common Test Hooks</title>
+ <prepared>Lukas Larsson</prepared>
+ <responsible>Lukas Larsson</responsible>
+ <docno></docno>
+ <approved></approved>
+ <checked></checked>
+ <date>2010-12-02</date>
+ <rev>PA1</rev>
+ <file>ct_hooks.sgml</file>
+ </header>
+ <module>ct_hooks</module>
+ <modulesummary>A callback interface on top of Common Test</modulesummary>
+
+ <description>
+
+ <warning><p>This feature is in alpha release right now. This means that the
+ interface may change in the future and that there may be bugs. We
+ encourage you to use this feature, but be prepared
+ that there might be bugs and that the interface might change
+ inbetween releases.</p></warning>
+
+ <p>The <em>Common Test Hook</em> (henceforth called CTH) framework allows
+ extensions of the default behaviour of Common Test by means of callbacks
+ before and after all test suite calls. It is meant for advanced users of
+ Common Test which want to abstract out behaviour which is common to
+ multiple test suites. </p>
+
+ <p>In brief, Common Test Hooks allows you to:</p>
+
+ <list>
+ <item>Manipulate the runtime config before each suite
+ configuration call</item>
+ <item>Manipulate the return of all suite configuration calls and in
+ extension the result of the test themselves.</item>
+ </list>
+
+ <p>The following sections describe the mandatory and optional CTH
+ functions Common Test will call during test execution. For more details
+ see <seealso marker="ct_hooks_chapter">Common Test Hooks</seealso> in
+ the User's Guide.</p>
+
+ <p>For information about how to add a CTH to your suite see
+ <seealso marker="ct_hooks_chapter#installing">Installing a CTH
+ </seealso> in the User's Guide.</p>
+
+ <note><p>See the
+ <seealso marker="ct_hooks_chapter#example">Example CTH</seealso>
+ in the User's Guide for a minimal example of a CTH. </p></note>
+
+ </description>
+
+ <section>
+ <title>CALLBACK FUNCTIONS</title>
+ <p>The following functions define the callback interface
+ for a Common Test Hook.</p>
+ </section>
+
+ <funcs>
+ <func>
+ <name>Module:init(Id, Opts) -&gt; State</name>
+ <fsummary>Initiates the Common Test Hook</fsummary>
+ <type>
+ <v>Id = reference() | term()</v>
+ <v>Opts = term()</v>
+ <v>State = term()</v>
+ </type>
+
+ <desc>
+ <p> MANDATORY </p>
+
+ <p>Always called before any other callback function.
+ Use this to initiate any common state.
+ It should return a state for this CTH.</p>
+
+ <p><c>Id</c> is the return value of
+ <seealso marker="#Module:id-1">id/1</seealso>, or a <c>reference</c>
+ (created using
+ <seealso marker="erts:erlang#make_ref-0">make_ref/0</seealso>)
+ if <seealso marker="#Module:id-1">id/1</seealso> is not implemented.
+ </p>
+
+ <p>For details about when init is called see
+ <seealso marker="ct_hooks_chapter#scope">scope</seealso>
+ in the User's Guide.</p>
+
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:pre_init_per_suite(SuiteName, Config, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called before init_per_suite</fsummary>
+ <type>
+ <v>SuiteName = atom()</v>
+ <v>Config = NewConfig = [{Key,Value}]</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {Return, NewCTHState}</v>
+ <v>Return = NewConfig | SkipOrFail</v>
+ <v>SkipOrFail = {fail, Reason} | {skip, Reason}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called before
+ <seealso marker="common_test#Module:init_per_suite-1">
+ init_per_suite</seealso> if it exists.
+ It typically contains initialization/logging which needs to be done
+ before init_per_suite is called.
+ If <c>{skip,Reason}</c> or <c>{fail,Reason}</c> is returned,
+ init_per_suite and all test cases of the suite will be skipped and
+ Reason printed in the overview log of the suite.</p>
+
+ <p><c>SuiteName</c> is the name of the suite to be run.</p>
+
+ <p><c>Config</c> is the original config list of the test suite.</p>
+
+ <p><c>CTHState</c> is the current internal state of the CTH.</p>
+
+ <p><c>Return</c> is the result of the init_per_suite function.
+ If it is <c>{skip,Reason}</c> or <c>{fail,Reason}</c>
+ <seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+ </seealso> will never be called, instead the initiation is considered
+ to be skipped/failed respectively. If a <c>NewConfig</c> list
+ is returned, <seealso marker="common_test#Module:init_per_suite-1">
+ init_per_suite</seealso> will be called with that <c>NewConfig</c> list.
+ See <seealso marker="ct_hooks_chapter#pre">
+ Pre Hooks</seealso> in the User's Guide for more details.</p>
+
+
+ <p>Note that this function is only called if the CTH has been added
+ before init_per_suite is run, see
+ <seealso marker="ct_hooks_chapter#scope">CTH Scoping</seealso>
+ in the User's Guide for details.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:post_init_per_suite(SuiteName, Config, Return, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called after init_per_suite</fsummary>
+ <type>
+ <v>SuiteName = atom()</v>
+ <v>Config = [{Key,Value}]</v>
+ <v>Return = NewReturn = Config | SkipOrFail | term()</v>
+ <v>SkipOrFail = {fail, Reason} | {skip, Reason} | term()</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewReturn, NewCTHState}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called after
+ <seealso marker="common_test#Module:init_per_suite-1">
+ init_per_suite</seealso> if it exists. It typically contains extra
+ checks to make sure that all the correct dependencies have
+ been started correctly.</p>
+
+ <p><c>Return</c> is what
+ <seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+ </seealso> returned, i.e. {fail,Reason}, {skip,Reason}, a <c>Config</c>
+ list or a term describing how
+ <seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+ </seealso> failed.</p>
+
+ <p><c>NewReturn</c> is the possibly modified return value of
+ <seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+ </seealso>. It is here possible to recover from a failure in
+ <seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+ </seealso> by returning the <c>ConfigList</c> with the <c>tc_status</c>
+ element removed. See <seealso marker="ct_hooks_chapter#post">
+ Post Hooks</seealso> in the User's Guide for more details.</p>
+
+ <p><c>CTHState</c> is the current internal state of the CTH.</p>
+
+ <p>Note that this function is only called if the CTH has been added
+ before or in init_per_suite, see
+ <seealso marker="ct_hooks_chapter#scope">CTH Scoping</seealso>
+ in the User's Guide for details.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:pre_init_per_group(GroupName, Config, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called before init_per_group</fsummary>
+ <type>
+ <v>GroupName = atom()</v>
+ <v>Config = NewConfig = [{Key,Value}]</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v>
+ <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called before
+ <seealso marker="common_test#Module:init_per_group-2">
+ init_per_group</seealso> if it exists. It behaves the same way as
+ <seealso marker="ct_hooks#Module:pre_init_per_suite-3">
+ pre_init_per_suite</seealso>, but for the
+ <seealso marker="common_test#Module:init_per_group-2">
+ init_per_group</seealso> instead.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:post_init_per_group(GroupName, Config, Return, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called after init_per_group</fsummary>
+ <type>
+ <v>GroupName = atom()</v>
+ <v>Config = [{Key,Value}]</v>
+ <v>Return = NewReturn = Config | SkipOrFail | term()</v>
+ <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewReturn, NewCTHState}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called after
+ <seealso marker="common_test#Module:init_per_group-2">
+ init_per_group</seealso> if it exists. It behaves the same way as
+ <seealso marker="ct_hooks#Module:post_init_per_suite-4">
+ post_init_per_suite</seealso>, but for the
+ <seealso marker="common_test#Module:init_per_group-2">
+ init_per_group</seealso> instead.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:pre_init_per_testcase(TestcaseName, Config, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called before init_per_testcase</fsummary>
+ <type>
+ <v>TestcaseName = atom()</v>
+ <v>Config = NewConfig = [{Key,Value}]</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v>
+ <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called before
+ <seealso marker="common_test#Module:init_per_testcase-2">
+ init_per_testcase</seealso> if it exists. It behaves the same way as
+ <seealso marker="ct_hooks#Module:pre_init_per_suite-3">
+ pre_init_per_suite</seealso>, but for the
+ <seealso marker="common_test#Module:init_per_testcase-2">
+ init_per_testcase</seealso> function instead.</p>
+
+ <p>Note that it is not possible to add CTH's here right now,
+ that feature might be added later,
+ but it would right now break backwards compatability.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:post_end_per_testcase(TestcaseName, Config, Return, CTHState)
+ -&gt; Result</name>
+ <fsummary>Called after end_per_testcase</fsummary>
+ <type>
+ <v>TestcaseName = atom()</v>
+ <v>Config = [{Key,Value}]</v>
+ <v>Return = NewReturn = Config | SkipOrFail | term()</v>
+ <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewReturn, NewCTHState}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called after
+ <seealso marker="common_test#Module:end_per_testcase-2">
+ end_per_testcase</seealso> if it exists. It behaves the same way as
+ <seealso marker="ct_hooks#Module:post_init_per_suite-4">
+ post_init_per_suite</seealso>, but for the
+ <seealso marker="common_test#Module:end_per_testcase-2">
+ end_per_testcase</seealso> function instead.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:pre_end_per_group(GroupName, Config, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called before end_per_group</fsummary>
+ <type>
+ <v>GroupName = atom()</v>
+ <v>Config = NewConfig = [{Key,Value}]</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v>
+ <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called before
+ <seealso marker="common_test#Module:end_per_group-2">
+ end_per_group</seealso> if it exists. It behaves the same way as
+ <seealso marker="ct_hooks#Module:pre_init_per_suite-3">
+ pre_init_per_suite</seealso>, but for the
+ <seealso marker="common_test#Module:end_per_group-2">
+ end_per_group</seealso> function instead.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:post_end_per_group(GroupName, Config, Return, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called after end_per_group</fsummary>
+ <type>
+ <v>GroupName = atom()</v>
+ <v>Config = [{Key,Value}]</v>
+ <v>Return = NewReturn = Config | SkipOrFail | term()</v>
+ <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewReturn, NewCTHState}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called after
+ <seealso marker="common_test#Module:end_per_group-2">
+ end_per_group</seealso> if it exists. It behaves the same way as
+ <seealso marker="ct_hooks#Module:post_init_per_suite-4">
+ post_init_per_suite</seealso>, but for the
+ <seealso marker="common_test#Module:end_per_group-2">
+ end_per_group</seealso> function instead.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:pre_end_per_suite(SuiteName, Config, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called before end_per_suite</fsummary>
+ <type>
+ <v>SuiteName = atom()</v>
+ <v>Config = NewConfig = [{Key,Value}]</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v>
+ <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called before
+ <seealso marker="common_test#Module:end_per_suite-1">
+ end_per_suite</seealso> if it exists. It behaves the same way as
+ <seealso marker="ct_hooks#Module:pre_init_per_suite-3">
+ pre_init_per_suite</seealso>, but for the
+ <seealso marker="common_test#Module:end_per_suite-2">
+ end_per_suite</seealso> function instead.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:post_end_per_suite(SuiteName, Config, Return, CTHState) -&gt;
+ Result</name>
+ <fsummary>Called after end_per_suite</fsummary>
+ <type>
+ <v>SuiteName = atom()</v>
+ <v>Config = [{Key,Value}]</v>
+ <v>Return = NewReturn = Config | SkipOrFail | term()</v>
+ <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+ <v>CTHState = NewCTHState = term()</v>
+ <v>Result = {NewReturn, NewCTHState}</v>
+ <v>Key = atom()</v>
+ <v>Value = term()</v>
+ <v>Reason = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called after
+ <seealso marker="common_test#Module:end_per_suite-1">
+ end_per_suite</seealso> if it exists. It behaves the same way as
+ <seealso marker="ct_hooks#Module:post_init_per_suite-4">
+ post_init_per_suite</seealso>, but for the
+ <seealso marker="common_test#Module:end_per_suite-2">
+ end_per_suite</seealso> function instead.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:on_tc_fail(TestcaseName, Reason, CTHState) -&gt;
+ NewCTHState</name>
+ <fsummary>Called after the CTH scope ends</fsummary>
+ <type>
+ <v>TestcaseName = init_per_suite | end_per_suite |
+ init_per_group | end_per_group | atom()</v>
+ <v>Reason = term()</v>
+ <v>CTHState = NewCTHState = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called whenever a testcase fails.
+ It is called after the post function has been called for
+ the testcase which failed. i.e.
+ if init_per_suite fails this function is called after
+ <seealso marker="#Module:post_init_per_suite-4">
+ post_init_per_suite</seealso>, and if a testcase fails it is called
+ after <seealso marker="#Module:post_end_per_testcase-4">
+ post_end_per_testcase</seealso>.</p>
+
+ <p>The data which comes with the Reason follows the same format as the
+ <seealso marker="event_handler_chapter#failreason">FailReason
+ </seealso> in the <seealso marker="event_handler_chapter#tc_done">tc_done</seealso> event.
+ See <seealso marker="event_handler_chapter#events">Event Handling
+ </seealso> in the User's Guide for details.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:on_tc_skip(TestcaseName, Reason, CTHState) -&gt;
+ NewCTHState</name>
+ <fsummary>Called after the CTH scope ends</fsummary>
+ <type>
+ <v>TestcaseName = end_per_suite | init_per_group |
+ end_per_group | atom()</v>
+ <v>Reason = {tc_auto_skip | tc_user_skip, term()}</v>
+ <v>CTHState = NewCTHState = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called whenever a testcase is skipped.
+ It is called after the post function has been called for the
+ testcase which was skipped.
+ i.e. if init_per_group is skipped this function is called after
+ <seealso marker="#Module:post_init_per_suite-4">post_init_per_group
+ </seealso>, and if a testcase is skipped it is called after
+ <seealso marker="#Module:post_end_per_testcase-4">post_end_per_testcase
+ </seealso>.</p>
+
+ <p>The data which comes with the Reason follows the same format as
+ <seealso marker="event_handler_chapter#tc_auto_skip">tc_auto_skip
+ </seealso> and <seealso marker="event_handler_chapter#tc_user_skip">
+ tc_user_skip</seealso> events.
+ See <seealso marker="event_handler_chapter#events">Event Handling
+ </seealso> in the User's Guide for details.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:terminate(CTHState)</name>
+ <fsummary>Called after the CTH scope ends</fsummary>
+ <type>
+ <v>CTHState = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>This function is called at the end of a CTH's
+ <seealso marker="ct_hooks_chapter#scope">scope</seealso>.
+ </p>
+ </desc>
+ </func>
+
+ <func>
+ <name>Module:id(Opts) -&gt; Id</name>
+ <fsummary>Called before the init function of a CTH</fsummary>
+ <type>
+ <v>Opts = term()</v>
+ <v>Id = term()</v>
+ </type>
+
+ <desc>
+ <p> OPTIONAL </p>
+
+ <p>The <c>Id</c> is used to uniquely identify a CTH instance,
+ if two CTH's return the same <c>Id</c> the second CTH is ignored
+ and subsequent calls to the CTH will only be made to the first
+ instance. For more information see
+ <seealso marker="ct_hooks_chapter#installing">Installing a CTH
+ </seealso> in the User's Guide.
+ </p>
+
+ <p>This function should NOT have any side effects as it might
+ be called multiple times by Common Test.</p>
+
+ <p>If not implemented the CTH will act as if this function returned a
+ call to <c>make_ref/0</c>.</p>
+ </desc>
+ </func>
+
+ </funcs>
+
+</erlref>
+
+