20102010 Ericsson AB. All Rights Reserved. The contents of this file are subject to the Erlang Public License, Version 1.1, (the "License"); you may not use this file except in compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. Suite Callbacks Lukas Larsson Lukas Larsson 2010-12-02 PA1 suite_callback.sgml
ct_suite_callback A callback interface on top of Common Test

This feature is in alpha release right now. This means that the interface may change in the future and that there may be bugs. We encourage you to use this feature, but be prepared that there might be bugs and that the interface might change inbetween releases.

The Suite Callback (henceforth called SCB) framework allows extensions of the default behaviour of Common Test by means of callbacks before and after all test suite calls. It is meant for advanced users of Common Test which want to abstract out behaviour which is common to multiple test suites.

In brief, Suite Callbacks allows you to:

Manipulate the runtime config before each suite configuration calls Manipulate the return of all suite configuration calls and in extension the result of the test themselves.

The following sections describe the mandatory and optional SCB functions Common Test will call during test execution. For more details see Suite Callbacks in the User's Guide.

For information about how to add a SCB to your suite see Installing an SCB in the User's Guide.

See the Example SCB in the User's Guide for a minimal example of an SCB.

CALLBACK FUNCTIONS

The following functions define the callback interface for a suite callback.

Module:init(Id, Opts) -> State Initiates the Suite Callback Id = reference() | term() Opts = term() State = term()

MANDATORY

Always called before any other callback function. Use this to initiate any common state. It should return a state for this SCB.

Id is the return value of id/1, or a reference if id/1 is not implemented.

For details about when init is called see scope in the User's Guide.

 Module:pre_init_per_suite(SuiteName, Config, SCBState) -> Result Called before init_per_suite SuiteName = atom() Config = NewConfig = [{Key,Value}] SCBState = NewSCBState = term() Result = {Return, NewSCBState} Return = NewConfig | SkipOrFail SkipOrFail = {fail, Reason} | {skip, Reason} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called before init_per_suite if it exists. It typically contains initialization/logging which needs to be done before init_per_suite is called. If {skip,Reason} or {fail,Reason} is returned, init_per_suite and all test cases of the suite will be skipped and Reason printed in the overview log of the suite.

SuiteName is the name of the suite to be run.

Config is the original config list of the test suite.

SCBState is the current internal state of the SCB.

Return is the result of the init_per_suite function. If it is {skip,Reason} or {fail,Reason} init_per_suite will never be called, instead the initiation is considered to be skipped/failed respectively. If a NewConfig list is returned, init_per_suite will be called with that NewConfig list. See Manipulating tests in the User's Guide for more details.

Note that this function is only called if the SCB has been added before init_per_suite is run, see SCB Scoping in the User's Guide for details.

 Module:post_init_per_suite(SuiteName, Config, Return, SCBState) -> Result Called after init_per_suite SuiteName = atom() Config = [{Key,Value}] Return = NewReturn = Config | SkipOrFail | term() SkipOrFail = {fail, Reason} | {skip, Reason} | term() SCBState = NewSCBState = term() Result = {NewReturn, NewSCBState} Key = atom() Value = term() Reason = term()

OPTIONAL

Return is what init_per_suite returned, i.e. {fail,Reason}, {skip,Reason}, a Config list or a term describing how init_per_suite failed.

NewReturn is the possibly modified return value of init_per_suite . It is here possible to recover from a failure in init_per_suite by returning the ConfigList with the tc_status element removed.

SCBState is the current internal state of the SCB.

This function is called after init_per_suite if it exists.

Note that this function is only called if the SCB has been added before or in init_per_suite, see SCB Scoping in the User's Guide for details.

 Module:pre_init_per_group(GroupName, Config, SCBState) -> Result Called before init_per_group GroupName = atom() Config = NewConfig = [{Key,Value}] SCBState = NewSCBState = term() Result = {NewConfig | SkipOrFail, NewSCBState} SkipOrFail = {fail,Reason} | {skip, Reason} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called before init_per_group if it exists. It behaves the same way as pre_init_per_suite, but for the init_per_group instead.

 Module:post_init_per_group(GroupName, Config, Return, SCBState) -> Result Called after init_per_group GroupName = atom() Config = [{Key,Value}] Return = NewReturn = Config | SkipOrFail | term() SkipOrFail = {fail,Reason} | {skip, Reason} SCBState = NewSCBState = term() Result = {NewReturn, NewSCBState} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called after init_per_group if it exists. It behaves the same way as post_init_per_suite, but for the init_per_group instead.

 Module:pre_init_per_testcase(TestcaseName, Config, SCBState) -> Result Called before init_per_testcase TestcaseName = atom() Config = NewConfig = [{Key,Value}] SCBState = NewSCBState = term() Result = {NewConfig | SkipOrFail, NewSCBState} SkipOrFail = {fail,Reason} | {skip, Reason} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called before init_per_testcase if it exists. It behaves the same way as pre_init_per_suite, but for the init_per_testcase function instead.

Note that it is not possible to add SCB's here right now, that feature might be added later, but it would right now break backwards compatability.

 Module:post_end_per_testcase(TestcaseName, Config, Return, SCBState) -> Result Called after end_per_testcase TestcaseName = atom() Config = [{Key,Value}] Return = NewReturn = Config | SkipOrFail | term() SkipOrFail = {fail,Reason} | {skip, Reason} SCBState = NewSCBState = term() Result = {NewReturn, NewSCBState} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called after end_per_testcase if it exists. It behaves the same way as post_init_per_suite, but for the end_per_testcase function instead.

 Module:pre_end_per_group(GroupName, Config, SCBState) -> Result Called before end_per_group GroupName = atom() Config = NewConfig = [{Key,Value}] SCBState = NewSCBState = term() Result = {NewConfig | SkipOrFail, NewSCBState} SkipOrFail = {fail,Reason} | {skip, Reason} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called before end_per_group if it exists. It behaves the same way as pre_init_per_suite, but for the end_per_group function instead.

 Module:post_end_per_group(GroupName, Config, Return, SCBState) -> Result Called after end_per_group GroupName = atom() Config = [{Key,Value}] Return = NewReturn = Config | SkipOrFail | term() SkipOrFail = {fail,Reason} | {skip, Reason} SCBState = NewSCBState = term() Result = {NewReturn, NewSCBState} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called after end_per_group if it exists. It behaves the same way as post_init_per_suite, but for the end_per_group function instead.

 Module:pre_end_per_suite(SuiteName, Config, SCBState) -> Result Called before end_per_suite SuiteName = atom() Config = NewConfig = [{Key,Value}] SCBState = NewSCBState = term() Result = {NewConfig | SkipOrFail, NewSCBState} SkipOrFail = {fail,Reason} | {skip, Reason} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called before end_per_suite if it exists. It behaves the same way as pre_init_per_suite, but for the end_per_suite function instead.

 Module:post_end_per_suite(SuiteName, Config, Return, SCBState) -> Result Called after end_per_suite SuiteName = atom() Config = [{Key,Value}] Return = NewReturn = Config | SkipOrFail | term() SkipOrFail = {fail,Reason} | {skip, Reason} SCBState = NewSCBState = term() Result = {NewReturn, NewSCBState} Key = atom() Value = term() Reason = term()

OPTIONAL

This function is called after end_per_suite if it exists. It behaves the same way as post_init_per_suite, but for the end_per_suite function instead.

 Module:on_tc_fail(TestcaseName, Reason, SCBState) -> NewSCBState Called after the SCB scope ends TestcaseName = init_per_suite | end_per_suite | init_per_group | end_per_group | atom() Reason = term() SCBState = NewSCBState = term()

OPTIONAL

This function is called whenever a testcase fails. It is called after the post function has been called for the testcase which failed. i.e. if init_per_suite fails this function is called after post_init_per_suite, and if a testcase fails it is called after post_end_per_testcase.

The data which comes with the Reason follows the same format as the FailReason in the tc_done event. See Event Handling in the User's Guide for details.

 Module:on_tc_skip(TestcaseName, Reason, SCBState) -> NewSCBState Called after the SCB scope ends TestcaseName = end_per_suite | init_per_group | end_per_group | atom() Reason = {tc_auto_skip | tc_user_skip, term()} SCBState = NewSCBState = term()

OPTIONAL

This function is called whenever a testcase is skipped. It is called after the post function has been called for the testcase which was skipped. i.e. if init_per_group is skipped this function is called after post_init_per_group , and if a testcase is skipped it is called after post_end_per_testcase .

The data which comes with the Reason follows the same format as tc_auto_skip and tc_user_skip events. See Event Handling in the User's Guide for details.

 Module:terminate(SCBState) Called after the SCB scope ends SCBState = term()

OPTIONAL

This function is called at the end of an SCB's scope.

Module:id(Opts) -> Id Called before the init function of an SCB Opts = term() Id = term()

OPTIONAL

The Id is used to uniquely identify an SCB instance, if two SCB's return the same Id the second SCB is ignored and subsequent calls to the SCB will only be made to the first instance. For more information see Installing an SCB in the User's Guide.

This function should NOT have any side effects as it might be called multiple times by Common Test.

If not implemented the SCB will act as if this function returned a call to make_ref/0.