diff options
Diffstat (limited to 'lib/common_test/doc/src/ct_hooks.xml')
| -rw-r--r-- | lib/common_test/doc/src/ct_hooks.xml | 556 |
1 files changed, 556 insertions, 0 deletions
diff --git a/lib/common_test/doc/src/ct_hooks.xml b/lib/common_test/doc/src/ct_hooks.xml new file mode 100644 index 0000000000..0d59ce3b22 --- /dev/null +++ b/lib/common_test/doc/src/ct_hooks.xml @@ -0,0 +1,556 @@ +<?xml version="1.0" encoding="UTF-8" ?> + +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>2010</year><year>2010</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Common Test Hooks</title> + <prepared>Lukas Larsson</prepared> + <responsible>Lukas Larsson</responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>2010-12-02</date> + <rev>PA1</rev> + <file>ct_hooks.sgml</file> + </header> + <module>ct_hooks</module> + <modulesummary>A callback interface on top of Common Test</modulesummary> + + <description> + + <warning><p>This feature is in alpha release right now. This means that the + interface may change in the future and that there may be bugs. We + encourage you to use this feature, but be prepared + that there might be bugs and that the interface might change + inbetween releases.</p></warning> + + <p>The <em>Common Test Hook</em> (henceforth called CTH) framework allows + extensions of the default behaviour of Common Test by means of callbacks + before and after all test suite calls. It is meant for advanced users of + Common Test which want to abstract out behaviour which is common to + multiple test suites. </p> + + <p>In brief, Common Test Hooks allows you to:</p> + + <list> + <item>Manipulate the runtime config before each suite + configuration call</item> + <item>Manipulate the return of all suite configuration calls and in + extension the result of the test themselves.</item> + </list> + + <p>The following sections describe the mandatory and optional CTH + functions Common Test will call during test execution. For more details + see <seealso marker="ct_hooks_chapter">Common Test Hooks</seealso> in + the User's Guide.</p> + + <p>For information about how to add a CTH to your suite see + <seealso marker="ct_hooks_chapter#installing">Installing a CTH + </seealso> in the User's Guide.</p> + + <note><p>See the + <seealso marker="ct_hooks_chapter#example">Example CTH</seealso> + in the User's Guide for a minimal example of a CTH. </p></note> + + </description> + + <section> + <title>CALLBACK FUNCTIONS</title> + <p>The following functions define the callback interface + for a Common Test Hook.</p> + </section> + + <funcs> + <func> + <name>Module:init(Id, Opts) -> State</name> + <fsummary>Initiates the Common Test Hook</fsummary> + <type> + <v>Id = reference() | term()</v> + <v>Opts = term()</v> + <v>State = term()</v> + </type> + + <desc> + <p> MANDATORY </p> + + <p>Always called before any other callback function. + Use this to initiate any common state. + It should return a state for this CTH.</p> + + <p><c>Id</c> is the return value of + <seealso marker="#Module:id-1">id/1</seealso>, or a <c>reference</c> + (created using + <seealso marker="erts:erlang#make_ref-0">make_ref/0</seealso>) + if <seealso marker="#Module:id-1">id/1</seealso> is not implemented. + </p> + + <p>For details about when init is called see + <seealso marker="ct_hooks_chapter#scope">scope</seealso> + in the User's Guide.</p> + + </desc> + </func> + + <func> + <name>Module:pre_init_per_suite(SuiteName, Config, CTHState) -> + Result</name> + <fsummary>Called before init_per_suite</fsummary> + <type> + <v>SuiteName = atom()</v> + <v>Config = NewConfig = [{Key,Value}]</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {Return, NewCTHState}</v> + <v>Return = NewConfig | SkipOrFail</v> + <v>SkipOrFail = {fail, Reason} | {skip, Reason}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called before + <seealso marker="common_test#Module:init_per_suite-1"> + init_per_suite</seealso> if it exists. + It typically contains initialization/logging which needs to be done + before init_per_suite is called. + If <c>{skip,Reason}</c> or <c>{fail,Reason}</c> is returned, + init_per_suite and all test cases of the suite will be skipped and + Reason printed in the overview log of the suite.</p> + + <p><c>SuiteName</c> is the name of the suite to be run.</p> + + <p><c>Config</c> is the original config list of the test suite.</p> + + <p><c>CTHState</c> is the current internal state of the CTH.</p> + + <p><c>Return</c> is the result of the init_per_suite function. + If it is <c>{skip,Reason}</c> or <c>{fail,Reason}</c> + <seealso marker="common_test#Module:init_per_suite-1">init_per_suite + </seealso> will never be called, instead the initiation is considered + to be skipped/failed respectively. If a <c>NewConfig</c> list + is returned, <seealso marker="common_test#Module:init_per_suite-1"> + init_per_suite</seealso> will be called with that <c>NewConfig</c> list. + See <seealso marker="ct_hooks_chapter#pre"> + Pre Hooks</seealso> in the User's Guide for more details.</p> + + + <p>Note that this function is only called if the CTH has been added + before init_per_suite is run, see + <seealso marker="ct_hooks_chapter#scope">CTH Scoping</seealso> + in the User's Guide for details.</p> + </desc> + </func> + + <func> + <name>Module:post_init_per_suite(SuiteName, Config, Return, CTHState) -> + Result</name> + <fsummary>Called after init_per_suite</fsummary> + <type> + <v>SuiteName = atom()</v> + <v>Config = [{Key,Value}]</v> + <v>Return = NewReturn = Config | SkipOrFail | term()</v> + <v>SkipOrFail = {fail, Reason} | {skip, Reason} | term()</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called after + <seealso marker="common_test#Module:init_per_suite-1"> + init_per_suite</seealso> if it exists. It typically contains extra + checks to make sure that all the correct dependencies have + been started correctly.</p> + + <p><c>Return</c> is what + <seealso marker="common_test#Module:init_per_suite-1">init_per_suite + </seealso> returned, i.e. {fail,Reason}, {skip,Reason}, a <c>Config</c> + list or a term describing how + <seealso marker="common_test#Module:init_per_suite-1">init_per_suite + </seealso> failed.</p> + + <p><c>NewReturn</c> is the possibly modified return value of + <seealso marker="common_test#Module:init_per_suite-1">init_per_suite + </seealso>. It is here possible to recover from a failure in + <seealso marker="common_test#Module:init_per_suite-1">init_per_suite + </seealso> by returning the <c>ConfigList</c> with the <c>tc_status</c> + element removed. See <seealso marker="ct_hooks_chapter#post"> + Post Hooks</seealso> in the User's Guide for more details.</p> + + <p><c>CTHState</c> is the current internal state of the CTH.</p> + + <p>Note that this function is only called if the CTH has been added + before or in init_per_suite, see + <seealso marker="ct_hooks_chapter#scope">CTH Scoping</seealso> + in the User's Guide for details.</p> + </desc> + </func> + + <func> + <name>Module:pre_init_per_group(GroupName, Config, CTHState) -> + Result</name> + <fsummary>Called before init_per_group</fsummary> + <type> + <v>GroupName = atom()</v> + <v>Config = NewConfig = [{Key,Value}]</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v> + <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called before + <seealso marker="common_test#Module:init_per_group-2"> + init_per_group</seealso> if it exists. It behaves the same way as + <seealso marker="ct_hooks#Module:pre_init_per_suite-3"> + pre_init_per_suite</seealso>, but for the + <seealso marker="common_test#Module:init_per_group-2"> + init_per_group</seealso> instead.</p> + </desc> + </func> + + <func> + <name>Module:post_init_per_group(GroupName, Config, Return, CTHState) -> + Result</name> + <fsummary>Called after init_per_group</fsummary> + <type> + <v>GroupName = atom()</v> + <v>Config = [{Key,Value}]</v> + <v>Return = NewReturn = Config | SkipOrFail | term()</v> + <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called after + <seealso marker="common_test#Module:init_per_group-2"> + init_per_group</seealso> if it exists. It behaves the same way as + <seealso marker="ct_hooks#Module:post_init_per_suite-4"> + post_init_per_suite</seealso>, but for the + <seealso marker="common_test#Module:init_per_group-2"> + init_per_group</seealso> instead.</p> + </desc> + </func> + + <func> + <name>Module:pre_init_per_testcase(TestcaseName, Config, CTHState) -> + Result</name> + <fsummary>Called before init_per_testcase</fsummary> + <type> + <v>TestcaseName = atom()</v> + <v>Config = NewConfig = [{Key,Value}]</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v> + <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called before + <seealso marker="common_test#Module:init_per_testcase-2"> + init_per_testcase</seealso> if it exists. It behaves the same way as + <seealso marker="ct_hooks#Module:pre_init_per_suite-3"> + pre_init_per_suite</seealso>, but for the + <seealso marker="common_test#Module:init_per_testcase-2"> + init_per_testcase</seealso> function instead.</p> + + <p>Note that it is not possible to add CTH's here right now, + that feature might be added later, + but it would right now break backwards compatability.</p> + </desc> + </func> + + <func> + <name>Module:post_end_per_testcase(TestcaseName, Config, Return, CTHState) + -> Result</name> + <fsummary>Called after end_per_testcase</fsummary> + <type> + <v>TestcaseName = atom()</v> + <v>Config = [{Key,Value}]</v> + <v>Return = NewReturn = Config | SkipOrFail | term()</v> + <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called after + <seealso marker="common_test#Module:end_per_testcase-2"> + end_per_testcase</seealso> if it exists. It behaves the same way as + <seealso marker="ct_hooks#Module:post_init_per_suite-4"> + post_init_per_suite</seealso>, but for the + <seealso marker="common_test#Module:end_per_testcase-2"> + end_per_testcase</seealso> function instead.</p> + </desc> + </func> + + <func> + <name>Module:pre_end_per_group(GroupName, Config, CTHState) -> + Result</name> + <fsummary>Called before end_per_group</fsummary> + <type> + <v>GroupName = atom()</v> + <v>Config = NewConfig = [{Key,Value}]</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v> + <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called before + <seealso marker="common_test#Module:end_per_group-2"> + end_per_group</seealso> if it exists. It behaves the same way as + <seealso marker="ct_hooks#Module:pre_init_per_suite-3"> + pre_init_per_suite</seealso>, but for the + <seealso marker="common_test#Module:end_per_group-2"> + end_per_group</seealso> function instead.</p> + </desc> + </func> + + <func> + <name>Module:post_end_per_group(GroupName, Config, Return, CTHState) -> + Result</name> + <fsummary>Called after end_per_group</fsummary> + <type> + <v>GroupName = atom()</v> + <v>Config = [{Key,Value}]</v> + <v>Return = NewReturn = Config | SkipOrFail | term()</v> + <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called after + <seealso marker="common_test#Module:end_per_group-2"> + end_per_group</seealso> if it exists. It behaves the same way as + <seealso marker="ct_hooks#Module:post_init_per_suite-4"> + post_init_per_suite</seealso>, but for the + <seealso marker="common_test#Module:end_per_group-2"> + end_per_group</seealso> function instead.</p> + </desc> + </func> + + <func> + <name>Module:pre_end_per_suite(SuiteName, Config, CTHState) -> + Result</name> + <fsummary>Called before end_per_suite</fsummary> + <type> + <v>SuiteName = atom()</v> + <v>Config = NewConfig = [{Key,Value}]</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewConfig | SkipOrFail, NewCTHState}</v> + <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called before + <seealso marker="common_test#Module:end_per_suite-1"> + end_per_suite</seealso> if it exists. It behaves the same way as + <seealso marker="ct_hooks#Module:pre_init_per_suite-3"> + pre_init_per_suite</seealso>, but for the + <seealso marker="common_test#Module:end_per_suite-2"> + end_per_suite</seealso> function instead.</p> + </desc> + </func> + + <func> + <name>Module:post_end_per_suite(SuiteName, Config, Return, CTHState) -> + Result</name> + <fsummary>Called after end_per_suite</fsummary> + <type> + <v>SuiteName = atom()</v> + <v>Config = [{Key,Value}]</v> + <v>Return = NewReturn = Config | SkipOrFail | term()</v> + <v>SkipOrFail = {fail,Reason} | {skip, Reason}</v> + <v>CTHState = NewCTHState = term()</v> + <v>Result = {NewReturn, NewCTHState}</v> + <v>Key = atom()</v> + <v>Value = term()</v> + <v>Reason = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called after + <seealso marker="common_test#Module:end_per_suite-1"> + end_per_suite</seealso> if it exists. It behaves the same way as + <seealso marker="ct_hooks#Module:post_init_per_suite-4"> + post_init_per_suite</seealso>, but for the + <seealso marker="common_test#Module:end_per_suite-2"> + end_per_suite</seealso> function instead.</p> + </desc> + </func> + + <func> + <name>Module:on_tc_fail(TestcaseName, Reason, CTHState) -> + NewCTHState</name> + <fsummary>Called after the CTH scope ends</fsummary> + <type> + <v>TestcaseName = init_per_suite | end_per_suite | + init_per_group | end_per_group | atom()</v> + <v>Reason = term()</v> + <v>CTHState = NewCTHState = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called whenever a testcase fails. + It is called after the post function has been called for + the testcase which failed. i.e. + if init_per_suite fails this function is called after + <seealso marker="#Module:post_init_per_suite-4"> + post_init_per_suite</seealso>, and if a testcase fails it is called + after <seealso marker="#Module:post_end_per_testcase-4"> + post_end_per_testcase</seealso>.</p> + + <p>The data which comes with the Reason follows the same format as the + <seealso marker="event_handler_chapter#failreason">FailReason + </seealso> in the <seealso marker="event_handler_chapter#tc_done">tc_done</seealso> event. + See <seealso marker="event_handler_chapter#events">Event Handling + </seealso> in the User's Guide for details.</p> + </desc> + </func> + + <func> + <name>Module:on_tc_skip(TestcaseName, Reason, CTHState) -> + NewCTHState</name> + <fsummary>Called after the CTH scope ends</fsummary> + <type> + <v>TestcaseName = end_per_suite | init_per_group | + end_per_group | atom()</v> + <v>Reason = {tc_auto_skip | tc_user_skip, term()}</v> + <v>CTHState = NewCTHState = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called whenever a testcase is skipped. + It is called after the post function has been called for the + testcase which was skipped. + i.e. if init_per_group is skipped this function is called after + <seealso marker="#Module:post_init_per_suite-4">post_init_per_group + </seealso>, and if a testcase is skipped it is called after + <seealso marker="#Module:post_end_per_testcase-4">post_end_per_testcase + </seealso>.</p> + + <p>The data which comes with the Reason follows the same format as + <seealso marker="event_handler_chapter#tc_auto_skip">tc_auto_skip + </seealso> and <seealso marker="event_handler_chapter#tc_user_skip"> + tc_user_skip</seealso> events. + See <seealso marker="event_handler_chapter#events">Event Handling + </seealso> in the User's Guide for details.</p> + </desc> + </func> + + <func> + <name>Module:terminate(CTHState)</name> + <fsummary>Called after the CTH scope ends</fsummary> + <type> + <v>CTHState = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>This function is called at the end of a CTH's + <seealso marker="ct_hooks_chapter#scope">scope</seealso>. + </p> + </desc> + </func> + + <func> + <name>Module:id(Opts) -> Id</name> + <fsummary>Called before the init function of a CTH</fsummary> + <type> + <v>Opts = term()</v> + <v>Id = term()</v> + </type> + + <desc> + <p> OPTIONAL </p> + + <p>The <c>Id</c> is used to uniquely identify a CTH instance, + if two CTH's return the same <c>Id</c> the second CTH is ignored + and subsequent calls to the CTH will only be made to the first + instance. For more information see + <seealso marker="ct_hooks_chapter#installing">Installing a CTH + </seealso> in the User's Guide. + </p> + + <p>This function should NOT have any side effects as it might + be called multiple times by Common Test.</p> + + <p>If not implemented the CTH will act as if this function returned a + call to <c>make_ref/0</c>.</p> + </desc> + </func> + + </funcs> + +</erlref> + + |