Rename Suite Callback to Common Test hook in documentation

1 files changed, 556 insertions, 0 deletions

diff --git a/lib/common_test/doc/src/ct_hooks.xml b/lib/common_test/doc/src/ct_hooks.xml
new file mode 100644
index 0000000000..0d59ce3b22
--- /dev/null
+++ b/lib/common_test/doc/src/ct_hooks.xml
@@ -0,0 +1,556 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+
+<!DOCTYPE erlref SYSTEM "erlref.dtd">
+
+<erlref>
+  <header>
+    <copyright>
+      <year>2010</year><year>2010</year>
+      <holder>Ericsson AB. All Rights Reserved.</holder>
+    </copyright>
+    <legalnotice>
+      The contents of this file are subject to the Erlang Public License,
+      Version 1.1, (the "License"); you may not use this file except in
+      compliance with the License. You should have received a copy of the
+      Erlang Public License along with this software. If not, it can be
+      retrieved online at http://www.erlang.org/.
+
+      Software distributed under the License is distributed on an "AS IS"
+      basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
+      the License for the specific language governing rights and limitations
+      under the License.
+
+    </legalnotice>
+
+    <title>Common Test Hooks</title>
+    <prepared>Lukas Larsson</prepared>
+    <responsible>Lukas Larsson</responsible>
+    <docno></docno>
+    <approved></approved>
+    <checked></checked>
+    <date>2010-12-02</date>
+    <rev>PA1</rev>
+    <file>ct_hooks.sgml</file>
+  </header>
+  <module>ct_hooks</module> 
+  <modulesummary>A callback interface on top of Common Test</modulesummary>
+
+  <description>
+
+    <warning><p>This feature is in alpha release right now. This means that the 
+	interface may change in the future and that there may be bugs. We 
+	encourage you to use this feature, but be prepared 
+	that there might be bugs and that the interface might change
+	inbetween releases.</p></warning>
+
+    <p>The <em>Common Test Hook</em> (henceforth called CTH) framework allows 
+      extensions of the default behaviour of Common Test by means of callbacks 
+      before and after all test suite calls. It is meant for advanced users of
+      Common Test which want to abstract out behaviour which is common to
+      multiple test suites. </p>
+
+    <p>In brief, Common Test Hooks allows you to:</p>
+
+    <list>
+      <item>Manipulate the runtime config before each suite 
+      configuration call</item>
+      <item>Manipulate the return of all suite configuration calls and in 
+      extension the result of the test themselves.</item>
+    </list>
+    
+    <p>The following sections describe the mandatory and optional CTH
+    functions Common Test will call during test execution. For more details
+    see <seealso marker="ct_hooks_chapter">Common Test Hooks</seealso> in 
+    the User's Guide.</p>
+
+    <p>For information about how to add a CTH to your suite see 
+    <seealso marker="ct_hooks_chapter#installing">Installing a CTH
+    </seealso> in the User's Guide.</p>
+
+    <note><p>See the
+	<seealso marker="ct_hooks_chapter#example">Example CTH</seealso>
+	in the User's Guide for a minimal example of a CTH. </p></note>
+    
+  </description>
+
+  <section>
+    <title>CALLBACK FUNCTIONS</title>
+    <p>The following functions define the callback interface
+      for a Common Test Hook.</p>
+  </section>
+  
+  <funcs>
+    <func>
+      <name>Module:init(Id, Opts) -&gt; State</name>
+      <fsummary>Initiates the Common Test Hook</fsummary>
+      <type>
+	<v>Id = reference() | term()</v>
+	<v>Opts = term()</v>
+	<v>State = term()</v>
+      </type>
+      
+      <desc>	
+	<p> MANDATORY </p>
+
+	<p>Always called before any other callback function. 
+	  Use this to initiate any common state. 
+	  It should return a state for this CTH.</p>
+
+	<p><c>Id</c> is the return value of 
+	  <seealso marker="#Module:id-1">id/1</seealso>, or a <c>reference</c>
+	  (created using 
+	  <seealso marker="erts:erlang#make_ref-0">make_ref/0</seealso>)
+	  if <seealso marker="#Module:id-1">id/1</seealso> is not implemented.
+	</p>
+
+	<p>For details about when init is called see
+	  <seealso marker="ct_hooks_chapter#scope">scope</seealso>
+	  in the User's Guide.</p>
+	      
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:pre_init_per_suite(SuiteName, Config, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called before init_per_suite</fsummary>
+      <type>
+	<v>SuiteName = atom()</v>
+	<v>Config = NewConfig = [{Key,Value}]</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {Return, NewCTHState}</v>
+	<v>Return = NewConfig | SkipOrFail</v>
+	<v>SkipOrFail = {fail, Reason} | {skip, Reason}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called before 
+	<seealso marker="common_test#Module:init_per_suite-1">
+	  init_per_suite</seealso> if it exists. 
+	It typically contains initialization/logging which needs to be done 
+	before init_per_suite is called. 
+	If <c>{skip,Reason}</c> or <c>{fail,Reason}</c> is returned, 
+	init_per_suite and all test cases of the suite will be skipped and
+	Reason printed in the overview log of the suite.</p>
+	
+	<p><c>SuiteName</c> is the name of the suite to be run.</p>
+
+	<p><c>Config</c> is the original config list of the test suite.</p>
+
+	<p><c>CTHState</c> is the current internal state of the CTH.</p>
+
+	<p><c>Return</c> is the result of the init_per_suite function.
+	If it is <c>{skip,Reason}</c> or <c>{fail,Reason}</c> 
+	<seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+	</seealso> will never be called, instead the initiation is considered 
+	to be skipped/failed respectively. If a <c>NewConfig</c> list
+	is returned, <seealso marker="common_test#Module:init_per_suite-1">
+	init_per_suite</seealso> will be called with that <c>NewConfig</c> list.
+	See <seealso marker="ct_hooks_chapter#pre">
+	Pre Hooks</seealso> in the User's Guide for more details.</p>
+	
+	
+	<p>Note that this function is only called if the CTH has been added 
+	before init_per_suite is run, see 
+	<seealso marker="ct_hooks_chapter#scope">CTH Scoping</seealso> 
+	in the User's Guide for details.</p>
+      </desc>
+    </func>
+    
+    <func>
+      <name>Module:post_init_per_suite(SuiteName, Config, Return, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called after init_per_suite</fsummary>
+      <type>
+	<v>SuiteName = atom()</v>
+	<v>Config = [{Key,Value}]</v>
+	<v>Return = NewReturn = Config | SkipOrFail | term()</v>
+	<v>SkipOrFail = {fail, Reason} | {skip, Reason} | term()</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewReturn, NewCTHState}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called after
+	<seealso marker="common_test#Module:init_per_suite-1">
+	  init_per_suite</seealso> if it exists. It typically contains extra 
+	checks to make sure that all the correct dependencies have
+	been started correctly.</p>
+	
+	<p><c>Return</c> is what 
+	<seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+	</seealso> returned, i.e. {fail,Reason}, {skip,Reason}, a <c>Config</c>
+	list or a term describing how 
+	<seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+	</seealso> failed.</p>
+
+	<p><c>NewReturn</c> is the possibly modified return value of
+	<seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+	</seealso>. It is here possible to recover from a failure in 
+	<seealso marker="common_test#Module:init_per_suite-1">init_per_suite
+	</seealso> by returning the <c>ConfigList</c> with the <c>tc_status</c>
+	element removed. See <seealso marker="ct_hooks_chapter#post">
+	Post Hooks</seealso> in the User's Guide for more details.</p>
+
+	<p><c>CTHState</c> is the current internal state of the CTH.</p>
+
+	<p>Note that this function is only called if the CTH has been added 
+	before or in init_per_suite, see 
+	<seealso marker="ct_hooks_chapter#scope">CTH Scoping</seealso> 
+	in the User's Guide for details.</p>
+      </desc>
+    </func>
+    
+    <func>
+      <name>Module:pre_init_per_group(GroupName, Config, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called before init_per_group</fsummary>
+      <type>
+	<v>GroupName = atom()</v>
+	<v>Config = NewConfig = [{Key,Value}]</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewConfig | SkipOrFail, NewCTHState}</v>
+	<v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+	
+	<p>This function is called before
+	<seealso marker="common_test#Module:init_per_group-2">
+	  init_per_group</seealso> if it exists. It behaves the same way as 
+	<seealso marker="ct_hooks#Module:pre_init_per_suite-3">
+	pre_init_per_suite</seealso>, but for the
+	<seealso marker="common_test#Module:init_per_group-2">
+	init_per_group</seealso> instead.</p>
+      </desc>
+    </func>
+    
+    <func>
+      <name>Module:post_init_per_group(GroupName, Config, Return, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called after init_per_group</fsummary>
+      <type>
+	<v>GroupName = atom()</v>
+	<v>Config = [{Key,Value}]</v>
+	<v>Return = NewReturn = Config | SkipOrFail | term()</v>
+	<v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewReturn, NewCTHState}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called after
+	<seealso marker="common_test#Module:init_per_group-2">
+	  init_per_group</seealso> if it exists. It behaves the same way as 
+	<seealso marker="ct_hooks#Module:post_init_per_suite-4">
+	post_init_per_suite</seealso>, but for the
+	<seealso marker="common_test#Module:init_per_group-2">
+	init_per_group</seealso> instead.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:pre_init_per_testcase(TestcaseName, Config, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called before init_per_testcase</fsummary>
+      <type>
+	<v>TestcaseName = atom()</v>
+	<v>Config = NewConfig = [{Key,Value}]</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewConfig | SkipOrFail, NewCTHState}</v>
+	<v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called before
+	<seealso marker="common_test#Module:init_per_testcase-2">
+	  init_per_testcase</seealso> if it exists. It behaves the same way as 
+	<seealso marker="ct_hooks#Module:pre_init_per_suite-3">
+	pre_init_per_suite</seealso>, but for the
+	<seealso marker="common_test#Module:init_per_testcase-2">
+	init_per_testcase</seealso> function instead.</p>
+
+	<p>Note that it is not possible to add CTH's here right now, 
+	that feature might be added later, 
+	but it would right now break backwards compatability.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:post_end_per_testcase(TestcaseName, Config, Return, CTHState)
+               -&gt; Result</name>
+      <fsummary>Called after end_per_testcase</fsummary>
+      <type>
+	<v>TestcaseName = atom()</v>
+	<v>Config = [{Key,Value}]</v>
+	<v>Return = NewReturn = Config | SkipOrFail | term()</v>
+	<v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewReturn, NewCTHState}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called after
+	<seealso marker="common_test#Module:end_per_testcase-2">
+	  end_per_testcase</seealso> if it exists. It behaves the same way as 
+	<seealso marker="ct_hooks#Module:post_init_per_suite-4">
+	post_init_per_suite</seealso>, but for the
+	<seealso marker="common_test#Module:end_per_testcase-2">
+	end_per_testcase</seealso> function instead.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:pre_end_per_group(GroupName, Config, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called before end_per_group</fsummary>
+      <type>
+	<v>GroupName = atom()</v>
+	<v>Config = NewConfig = [{Key,Value}]</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewConfig | SkipOrFail, NewCTHState}</v>
+	<v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+	
+	<p>This function is called before
+	<seealso marker="common_test#Module:end_per_group-2">
+	end_per_group</seealso> if it exists. It behaves the same way as 
+	<seealso marker="ct_hooks#Module:pre_init_per_suite-3">
+	pre_init_per_suite</seealso>, but for the
+	<seealso marker="common_test#Module:end_per_group-2">
+	end_per_group</seealso> function instead.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:post_end_per_group(GroupName, Config, Return, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called after end_per_group</fsummary>
+      <type>
+	<v>GroupName = atom()</v>
+	<v>Config = [{Key,Value}]</v>
+	<v>Return = NewReturn = Config | SkipOrFail | term()</v>
+	<v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewReturn, NewCTHState}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called after
+	<seealso marker="common_test#Module:end_per_group-2">
+	  end_per_group</seealso> if it exists. It behaves the same way as 
+	<seealso marker="ct_hooks#Module:post_init_per_suite-4">
+	post_init_per_suite</seealso>, but for the
+	<seealso marker="common_test#Module:end_per_group-2">
+	end_per_group</seealso> function instead.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:pre_end_per_suite(SuiteName, Config, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called before end_per_suite</fsummary>
+      <type>
+	<v>SuiteName = atom()</v>
+	<v>Config = NewConfig = [{Key,Value}]</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewConfig | SkipOrFail, NewCTHState}</v>
+	<v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called before
+	<seealso marker="common_test#Module:end_per_suite-1">
+	  end_per_suite</seealso> if it exists. It behaves the same way as 
+	<seealso marker="ct_hooks#Module:pre_init_per_suite-3">
+	pre_init_per_suite</seealso>, but for the
+	<seealso marker="common_test#Module:end_per_suite-2">
+	end_per_suite</seealso> function instead.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:post_end_per_suite(SuiteName, Config, Return, CTHState) -&gt; 
+              Result</name>
+      <fsummary>Called after end_per_suite</fsummary>
+      <type>
+	<v>SuiteName = atom()</v>
+	<v>Config = [{Key,Value}]</v>
+	<v>Return = NewReturn = Config | SkipOrFail | term()</v>
+	<v>SkipOrFail = {fail,Reason} | {skip, Reason}</v>
+	<v>CTHState = NewCTHState = term()</v>
+	<v>Result = {NewReturn, NewCTHState}</v>
+	<v>Key = atom()</v>
+	<v>Value = term()</v>
+	<v>Reason = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called after
+	<seealso marker="common_test#Module:end_per_suite-1">
+	  end_per_suite</seealso> if it exists. It behaves the same way as 
+	<seealso marker="ct_hooks#Module:post_init_per_suite-4">
+	post_init_per_suite</seealso>, but for the
+	<seealso marker="common_test#Module:end_per_suite-2">
+	end_per_suite</seealso> function instead.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:on_tc_fail(TestcaseName, Reason, CTHState) -&gt; 
+              NewCTHState</name>
+      <fsummary>Called after the CTH scope ends</fsummary>
+      <type>
+	<v>TestcaseName = init_per_suite | end_per_suite | 
+	                  init_per_group | end_per_group | atom()</v>
+	<v>Reason = term()</v>
+	<v>CTHState = NewCTHState = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called whenever a testcase fails. 
+	It is called after the post function has been called for 
+	the testcase which failed. i.e. 
+	if init_per_suite fails this function is called after 
+	<seealso marker="#Module:post_init_per_suite-4">
+	  post_init_per_suite</seealso>, and if a testcase fails it is called 
+	  after <seealso marker="#Module:post_end_per_testcase-4">
+	  post_end_per_testcase</seealso>.</p>
+
+	<p>The data which comes with the Reason follows the same format as the
+	<seealso marker="event_handler_chapter#failreason">FailReason
+	</seealso> in the <seealso marker="event_handler_chapter#tc_done">tc_done</seealso> event.
+	See <seealso marker="event_handler_chapter#events">Event Handling
+	</seealso> in the User's Guide for details.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:on_tc_skip(TestcaseName, Reason, CTHState) -&gt; 
+              NewCTHState</name>
+      <fsummary>Called after the CTH scope ends</fsummary>
+      <type>
+	<v>TestcaseName = end_per_suite | init_per_group | 
+                          end_per_group | atom()</v>
+	<v>Reason = {tc_auto_skip | tc_user_skip, term()}</v>
+	<v>CTHState = NewCTHState = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called whenever a testcase is skipped. 
+	It is called after the post function has been called for the 
+	testcase which was skipped. 
+	i.e. if init_per_group is skipped this function is called after 
+	<seealso marker="#Module:post_init_per_suite-4">post_init_per_group
+	</seealso>, and if a testcase is skipped it is called after 
+	<seealso marker="#Module:post_end_per_testcase-4">post_end_per_testcase
+	</seealso>.</p>
+
+	<p>The data which comes with the Reason follows the same format as 
+	<seealso marker="event_handler_chapter#tc_auto_skip">tc_auto_skip
+	</seealso> and <seealso marker="event_handler_chapter#tc_user_skip">
+	tc_user_skip</seealso> events.
+	See <seealso marker="event_handler_chapter#events">Event Handling
+	</seealso> in the User's Guide for details.</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:terminate(CTHState)</name>
+      <fsummary>Called after the CTH scope ends</fsummary>
+      <type>
+	<v>CTHState = term()</v>
+      </type>
+      
+      <desc>	
+	<p> OPTIONAL </p>
+
+	<p>This function is called at the end of a CTH's 
+	<seealso marker="ct_hooks_chapter#scope">scope</seealso>. 
+	</p>
+      </desc>
+    </func>
+
+    <func>
+      <name>Module:id(Opts) -&gt; Id</name>
+      <fsummary>Called before the init function of a CTH</fsummary>
+      <type>
+	<v>Opts = term()</v>
+	<v>Id = term()</v>
+      </type>
+      
+      <desc>
+	<p> OPTIONAL </p>
+	
+	<p>The <c>Id</c> is used to uniquely identify a CTH instance, 
+	if two CTH's return the same <c>Id</c> the second CTH is ignored
+	and subsequent calls to the CTH will only be made to the first
+	instance. For more information see 
+	<seealso marker="ct_hooks_chapter#installing">Installing a CTH
+	</seealso> in the User's Guide.
+        </p>
+
+	<p>This function should NOT have any side effects as it might 
+	be called multiple times by Common Test.</p>
+	
+	<p>If not implemented the CTH will act as if this function returned a
+	  call to <c>make_ref/0</c>.</p>
+      </desc>
+    </func>
+
+  </funcs>
+
+</erlref>
+
+