From 68d7535fe0cccd50622884f704edeb9d8bb47430 Mon Sep 17 00:00:00 2001 From: tmanevik Date: Fri, 18 Dec 2015 12:21:32 +0100 Subject: Common Test: Editorial changes 3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reference Manual files from Pär Wennstad added --- lib/common_test/doc/src/ct_hooks.xml | 805 +++++++++++++++++------------------ 1 file changed, 390 insertions(+), 415 deletions(-) (limited to 'lib/common_test/doc/src/ct_hooks.xml') diff --git a/lib/common_test/doc/src/ct_hooks.xml b/lib/common_test/doc/src/ct_hooks.xml index a9f9450dd7..9c959945d2 100644 --- a/lib/common_test/doc/src/ct_hooks.xml +++ b/lib/common_test/doc/src/ct_hooks.xml @@ -1,5 +1,4 @@ - @@ -12,7 +11,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - + http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software @@ -34,539 +33,515 @@ ct_hooks.sgml ct_hooks - A callback interface on top of Common Test + A callback interface on top of Common Test. -

The Common Test Hook (henceforth called CTH) framework allows - extensions of the default behaviour of Common Test by means of callbacks - before and after all test suite calls. It is meant for advanced users of - Common Test which want to abstract out behaviour which is common to - multiple test suites.

+

The Common Test Hook (CTH) framework allows extensions of the + default behavior of Common Test by callbacks before and after all + test suite calls. It is intended for advanced users of Common Test + who want to abstract out behavior that is common to multiple test suites. +

-

In brief, Common Test Hooks allows you to:

+

In brief, CTH allows you to:

- - Manipulate the runtime config before each suite - configuration call - Manipulate the return of all suite configuration calls and in - extension the result of the test themselves. + +

Manipulate the runtime configuration before each suite + configuration call.

 +

Manipulate the return of all suite configuration calls and by + extension the result of the test themselves.

The following sections describe the mandatory and optional CTH - functions Common Test will call during test execution. For more details - see Common Test Hooks in - the User's Guide.

+ functions that Common Test calls during test execution. + For more details, see section + Common Test Hooks in the + User's Guide.

-

For information about how to add a CTH to your suite see - Installing a CTH - in the User's Guide.

+

For information about how to add a CTH to your suite, see section + Installing a CTH + in the User's Guide.

+ +

For a minimal example of a CTH, see section + Example CTH + in the User's Guide.

 -

See the - Example CTH - in the User's Guide for a minimal example of a CTH.

-
- CALLBACK FUNCTIONS -

The following functions define the callback interface - for a Common Test Hook.

+ Callback Functions +

The following functions define the callback interface for a CTH.

- Module:init(Id, Opts) -> {ok, State} | - {ok, State, Priority} - Initiates the Common Test Hook + Module:init(Id, Opts) -> {ok, State} | {ok, State, Priority} + Initiates the Common Test Hook. - Id = reference() | term() - Opts = term() - State = term() - Priority = integer() + Id = reference() | term() + Opts = term() + State = term() + Priority = integer() - -

MANDATORY

- -

Always called before any other callback function. - Use this to initiate any common state. - It should return a state for this CTH.

- -

Id is the return value of - id/1, or a reference - (created using - make_ref/0) - if id/1 is not implemented. -

- -

Priority is the relative priority of this hook. Hooks with a - lower priority will be executed first. If no priority is given, - it will be set to 0.

- -

For details about when init is called see - scope - in the User's Guide.

- +

MANDATORY

+ +

This function is always called before any other callback function. + Use it to initiate any common state. It is to return a state for + this CTH.

+ +

Id is either the return value of + ct_hooks:id/1, + or a reference (created using + erlang:make_ref/0 + in ERTS) if + ct_hooks:id/1 + is not implemented.

+ +

Priority is the relative priority of this hook. Hooks with a + lower priority are executed first. If no priority is specified, it + is set to 0.

+ +

For details about when init is called, see section + CTH Scope + in the User's Guide.

 - Module:pre_init_per_suite(SuiteName, InitData, CTHState) -> - Result - Called before init_per_suite + Module:pre_init_per_suite(SuiteName, InitData, CTHState) -> Result + Called before init_per_suite. - SuiteName = atom() - InitData = Config | SkipOrFail - Config = NewConfig = [{Key,Value}] - CTHState = NewCTHState = term() - Result = {Return, NewCTHState} - Return = NewConfig | SkipOrFail - SkipOrFail = {fail, Reason} | {skip, Reason} - Key = atom() - Value = term() - Reason = term() + SuiteName = atom() + InitData = Config | SkipOrFail + Config = NewConfig = [{Key,Value}] + CTHState = NewCTHState = term() + Result = {Return, NewCTHState} + Return = NewConfig | SkipOrFail + SkipOrFail = {fail, Reason} | {skip, Reason} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called before - - init_per_suite if it exists. - It typically contains initialization/logging which needs to be done - before init_per_suite is called. - If {skip,Reason} or {fail,Reason} is returned, - init_per_suite and all test cases of the suite will be skipped and - Reason printed in the overview log of the suite.

- -

SuiteName is the name of the suite to be run.

- -

InitData is the original config list of the test suite, or - a SkipOrFail tuple if a previous CTH has returned this.

- -

CTHState is the current internal state of the CTH.

- -

Return is the result of the init_per_suite function. - If it is {skip,Reason} or {fail,Reason} - init_per_suite - will never be called, instead the initiation is considered - to be skipped/failed respectively. If a NewConfig list - is returned, - init_per_suite will be called with that NewConfig list. - See - Pre Hooks in the User's Guide for more details.

- - -

Note that this function is only called if the CTH has been added - before init_per_suite is run, see - CTH Scoping - in the User's Guide for details.

+

OPTIONAL

+ +

This function is called before + init_per_suite + if it exists. It typically contains initialization/logging that must + be done before init_per_suite is called. If + {skip,Reason} or {fail,Reason} is returned, + init_per_suite and all test cases of the suite are skipped + and Reason printed in the overview log of the suite.

+ +

SuiteName is the name of the suite to be run.

+ +

InitData is the original configuration list of the test + suite, or a SkipOrFail tuple if a previous CTH has returned + this.

+ +

CTHState is the current internal state of the CTH.

+ +

Return is the result of the init_per_suite function. + If it is {skip,Reason} or {fail,Reason}, + init_per_suite + is never called, instead the initiation is considered to be + skipped or failed, respectively. If a NewConfig list is + returned, + init_per_suite + is called with that NewConfig list. For more details, see + section Pre Hooks + in the User's Guide.

+ +

This function is called only if the CTH is added before + init_per_suite is run. For details, see section + CTH Scope + in the User's Guide.

 - + - Module:post_init_per_suite(SuiteName, Config, Return, CTHState) -> - Result - Called after init_per_suite + Module:post_init_per_suite(SuiteName, Config, Return, CTHState) -> Result + Called after init_per_suite. - SuiteName = atom() - Config = [{Key,Value}] - Return = NewReturn = Config | SkipOrFail | term() - SkipOrFail = {fail, Reason} | {skip, Reason} | term() - CTHState = NewCTHState = term() - Result = {NewReturn, NewCTHState} - Key = atom() - Value = term() - Reason = term() + SuiteName = atom() + Config = [{Key,Value}] + Return = NewReturn = Config | SkipOrFail | term() + SkipOrFail = {fail, Reason} | {skip, Reason} | term() + CTHState = NewCTHState = term() + Result = {NewReturn, NewCTHState} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called after - - init_per_suite if it exists. It typically contains extra - checks to make sure that all the correct dependencies have - been started correctly.

- -

Return is what - init_per_suite - returned, i.e. {fail,Reason}, {skip,Reason}, a Config - list or a term describing how - init_per_suite - failed.

- -

NewReturn is the possibly modified return value of - init_per_suite - . It is here possible to recover from a failure in - init_per_suite - by returning the ConfigList with the tc_status - element removed. See - Post Hooks in the User's Guide for more details.

- -

CTHState is the current internal state of the CTH.

- -

Note that this function is only called if the CTH has been added - before or in init_per_suite, see - CTH Scoping - in the User's Guide for details.

+

OPTIONAL

+ +

This function is called after + init_per_suite + if it exists. It typically contains extra checks to ensure that all + the correct dependencies are started correctly.

+ +

Return is what + init_per_suite + returned, that is, {fail,Reason}, {skip,Reason}, a + Config list, or a term describing how + init_per_suite + failed.

+ +

NewReturn is the possibly modified return value of + init_per_suite. + To recover from a failure in + init_per_suite, + return ConfigList with the tc_status element removed. + For more details, see + Post Hooks in + section "Manipulating Tests" in the User's Guide.

+ +

CTHState is the current internal state of the CTH.

+ +

This function is called only if the CTH is added before or in + init_per_suite. For details, see section + CTH Scope + in the User's Guide.

 - + - Module:pre_init_per_group(GroupName, InitData, CTHState) -> - Result - Called before init_per_group + Module:pre_init_per_group(GroupName, InitData, CTHState) -> Result + Called before init_per_group. - GroupName = atom() - InitData = Config | SkipOrFail - Config = NewConfig = [{Key,Value}] - CTHState = NewCTHState = term() - Result = {NewConfig | SkipOrFail, NewCTHState} - SkipOrFail = {fail,Reason} | {skip, Reason} - Key = atom() - Value = term() - Reason = term() + GroupName = atom() + InitData = Config | SkipOrFail + Config = NewConfig = [{Key,Value}] + CTHState = NewCTHState = term() + Result = {NewConfig | SkipOrFail, NewCTHState} + SkipOrFail = {fail,Reason} | {skip, Reason} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called before - - init_per_group if it exists. It behaves the same way as - - pre_init_per_suite, but for the - - init_per_group instead.

+

OPTIONAL

+ +

This function is called before + init_per_group + if it exists. It behaves the same way as + pre_init_per_suite, + but for function + init_per_group + instead.

 - + - Module:post_init_per_group(GroupName, Config, Return, CTHState) -> - Result - Called after init_per_group + Module:post_init_per_group(GroupName, Config, Return, CTHState) -> Result + Called after init_per_group. - GroupName = atom() - Config = [{Key,Value}] - Return = NewReturn = Config | SkipOrFail | term() - SkipOrFail = {fail,Reason} | {skip, Reason} - CTHState = NewCTHState = term() - Result = {NewReturn, NewCTHState} - Key = atom() - Value = term() - Reason = term() + GroupName = atom() + Config = [{Key,Value}] + Return = NewReturn = Config | SkipOrFail | term() + SkipOrFail = {fail,Reason} | {skip, Reason} + CTHState = NewCTHState = term() + Result = {NewReturn, NewCTHState} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called after - - init_per_group if it exists. It behaves the same way as - - post_init_per_suite, but for the - - init_per_group instead.

+

OPTIONAL

+ +

This function is called after + init_per_group + if it exists. It behaves the same way as + post_init_per_suite, + but for function + init_per_group + instead.

 - Module:pre_init_per_testcase(TestcaseName, InitData, CTHState) -> - Result - Called before init_per_testcase + Module:pre_init_per_testcase(TestcaseName, InitData, CTHState) -> Result + Called before init_per_testcase. - TestcaseName = atom() - InitData = Config | SkipOrFail - Config = NewConfig = [{Key,Value}] - CTHState = NewCTHState = term() - Result = {NewConfig | SkipOrFail, NewCTHState} - SkipOrFail = {fail,Reason} | {skip, Reason} - Key = atom() - Value = term() - Reason = term() + TestcaseName = atom() + InitData = Config | SkipOrFail + Config = NewConfig = [{Key,Value}] + CTHState = NewCTHState = term() + Result = {NewConfig | SkipOrFail, NewCTHState} + SkipOrFail = {fail,Reason} | {skip, Reason} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called before - - init_per_testcase if it exists. It behaves the same way as - - pre_init_per_suite, but for the - - init_per_testcase function instead.

- -

Note that it is not possible to add CTH's here right now, - that feature might be added later, - but it would right now break backwards compatibility.

+

OPTIONAL

+ +

This function is called before + init_per_testcase + if it exists. It behaves the same way as + pre_init_per_suite, + but for function + init_per_testcase + instead.

+ +

CTHs cannot be added here right now. That feature may be added in + a later release, but it would right now break backwards + compatibility.

 - Module:post_end_per_testcase(TestcaseName, Config, Return, CTHState) - -> Result - Called after end_per_testcase + Module:post_end_per_testcase(TestcaseName, Config, Return, CTHState) -> Result + Called after end_per_testcase. - TestcaseName = atom() - Config = [{Key,Value}] - Return = NewReturn = Config | SkipOrFail | term() - SkipOrFail = {fail,Reason} | {skip, Reason} - CTHState = NewCTHState = term() - Result = {NewReturn, NewCTHState} - Key = atom() - Value = term() - Reason = term() + TestcaseName = atom() + Config = [{Key,Value}] + Return = NewReturn = Config | SkipOrFail | term() + SkipOrFail = {fail,Reason} | {skip, Reason} + CTHState = NewCTHState = term() + Result = {NewReturn, NewCTHState} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called after - - end_per_testcase if it exists. It behaves the same way as - - post_init_per_suite, but for the - - end_per_testcase function instead.

+

OPTIONAL

+ +

This function is called after + end_per_testcase + if it exists. It behaves the same way as + post_init_per_suite, + but for function + end_per_testcase + instead.

 - Module:pre_end_per_group(GroupName, EndData, CTHState) -> - Result - Called before end_per_group + Module:pre_end_per_group(GroupName, EndData, CTHState) -> Result + Called before end_per_group. - GroupName = atom() - EndData = Config | SkipOrFail - Config = NewConfig = [{Key,Value}] - CTHState = NewCTHState = term() - Result = {NewConfig | SkipOrFail, NewCTHState} - SkipOrFail = {fail,Reason} | {skip, Reason} - Key = atom() - Value = term() - Reason = term() + GroupName = atom() + EndData = Config | SkipOrFail + Config = NewConfig = [{Key,Value}] + CTHState = NewCTHState = term() + Result = {NewConfig | SkipOrFail, NewCTHState} + SkipOrFail = {fail,Reason} | {skip, Reason} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called before - - end_per_group if it exists. It behaves the same way as - - pre_init_per_suite, but for the - - end_per_group function instead.

+

OPTIONAL

+ +

This function is called before + end_per_group + if it exists. It behaves the same way as + pre_init_per_suite, + but for function + end_per_group + instead.

 - Module:post_end_per_group(GroupName, Config, Return, CTHState) -> - Result - Called after end_per_group + Module:post_end_per_group(GroupName, Config, Return, CTHState) -> Result + Called after end_per_group. - GroupName = atom() - Config = [{Key,Value}] - Return = NewReturn = Config | SkipOrFail | term() - SkipOrFail = {fail,Reason} | {skip, Reason} - CTHState = NewCTHState = term() - Result = {NewReturn, NewCTHState} - Key = atom() - Value = term() - Reason = term() + GroupName = atom() + Config = [{Key,Value}] + Return = NewReturn = Config | SkipOrFail | term() + SkipOrFail = {fail,Reason} | {skip, Reason} + CTHState = NewCTHState = term() + Result = {NewReturn, NewCTHState} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called after - - end_per_group if it exists. It behaves the same way as - - post_init_per_suite, but for the - - end_per_group function instead.

+

OPTIONAL

+ +

This function is called after + end_per_group + if it exists. It behaves the same way as + post_init_per_suite, + but for function + end_per_group + instead.

 - Module:pre_end_per_suite(SuiteName, EndData, CTHState) -> - Result - Called before end_per_suite + Module:pre_end_per_suite(SuiteName, EndData, CTHState) -> Result + Called before end_per_suite. - SuiteName = atom() - EndData = Config | SkipOrFail - Config = NewConfig = [{Key,Value}] - CTHState = NewCTHState = term() - Result = {NewConfig | SkipOrFail, NewCTHState} - SkipOrFail = {fail,Reason} | {skip, Reason} - Key = atom() - Value = term() - Reason = term() + SuiteName = atom() + EndData = Config | SkipOrFail + Config = NewConfig = [{Key,Value}] + CTHState = NewCTHState = term() + Result = {NewConfig | SkipOrFail, NewCTHState} + SkipOrFail = {fail,Reason} | {skip, Reason} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called before - - end_per_suite if it exists. It behaves the same way as - - pre_init_per_suite, but for the - - end_per_suite function instead.

+

OPTIONAL

+ +

This function is called before + end_per_suite + if it exists. It behaves the same way as + pre_init_per_suite, + but for function + end_per_suite + instead.

 - Module:post_end_per_suite(SuiteName, Config, Return, CTHState) -> - Result - Called after end_per_suite + Module:post_end_per_suite(SuiteName, Config, Return, CTHState) -> Result + Called after end_per_suite. - SuiteName = atom() - Config = [{Key,Value}] - Return = NewReturn = Config | SkipOrFail | term() - SkipOrFail = {fail,Reason} | {skip, Reason} - CTHState = NewCTHState = term() - Result = {NewReturn, NewCTHState} - Key = atom() - Value = term() - Reason = term() + SuiteName = atom() + Config = [{Key,Value}] + Return = NewReturn = Config | SkipOrFail | term() + SkipOrFail = {fail,Reason} | {skip, Reason} + CTHState = NewCTHState = term() + Result = {NewReturn, NewCTHState} + Key = atom() + Value = term() + Reason = term() - -

OPTIONAL

- -

This function is called after - - end_per_suite if it exists. It behaves the same way as - - post_init_per_suite, but for the - - end_per_suite function instead.

+

OPTIONAL

+ +

This function is called after + end_per_suite + if it exists. It behaves the same way as + post_init_per_suite, + but for function + end_per_suite + instead.

 - Module:on_tc_fail(TestName, Reason, CTHState) -> - NewCTHState - Called after the CTH scope ends + Module:on_tc_fail(TestName, Reason, CTHState) -> NewCTHState + Called after the CTH scope ends. - TestName = init_per_suite | end_per_suite | - {init_per_group,GroupName} | {end_per_group,GroupName} | - {FuncName,GroupName} | FuncName - FuncName = atom() - GroupName = atom() - Reason = term() - CTHState = NewCTHState = term() + TestName = init_per_suite | end_per_suite | {init_per_group,GroupName} | {end_per_group,GroupName} | {FuncName,GroupName} | FuncName + FuncName = atom() + GroupName = atom() + Reason = term() + CTHState = NewCTHState = term() - -

OPTIONAL

- -

This function is called whenever a test case (or config function) - fails. It is called after the post function has been called for - the failed test case. I.e. if init_per_suite fails, this function - is called after - - post_init_per_suite, and if a test case fails, it is called - after - post_end_per_testcase. If the failed test case belongs - to a test case group, the first argument is a tuple - {FuncName,GroupName}, otherwise simply the function name.

- -

The data which comes with the Reason follows the same format as the - FailReason - in the tc_done event. - See Event Handling - in the User's Guide for details.

+

OPTIONAL

+ +

This function is called whenever a test case (or configuration + function) fails. It is called after the post function is called + for the failed test case, that is:

+ + +

If init_per_suite fails, this function is called after + post_init_per_suite.

 +

If a test case fails, this funcion is called after + post_end_per_testcase.

 + + +

If the failed test case belongs to a test case group, the first + argument is a tuple {FuncName,GroupName}, otherwise only + the function name.

+ +

The data that comes with Reason follows the same format as + FailReason + in event + tc_done. + For details, see section + Event Handling + in the User's Guide.

 - Module:on_tc_skip(TestName, Reason, CTHState) -> - NewCTHState - Called after the CTH scope ends + Module:on_tc_skip(TestName, Reason, CTHState) -> NewCTHState + Called after the CTH scope ends. - TestName = init_per_suite | end_per_suite | - {init_per_group,GroupName} | {end_per_group,GroupName} | - {FuncName,GroupName} | FuncName - FuncName = atom() - GroupName = atom() - Reason = {tc_auto_skip | tc_user_skip, term()} - CTHState = NewCTHState = term() + TestName = init_per_suite | end_per_suite | {init_per_group,GroupName} | {end_per_group,GroupName} | {FuncName,GroupName} | FuncName + FuncName = atom() + GroupName = atom() + Reason = {tc_auto_skip | tc_user_skip, term()} + CTHState = NewCTHState = term() - -

OPTIONAL

- -

This function is called whenever a test case (or config function) - is skipped. It is called after the post function has been called - for the skipped test case. I.e. if init_per_group is skipped, this - function is called after - - post_init_per_group, and if a test case is skipped, - it is called after - - post_end_per_testcase. If the skipped test case belongs to a - test case group, the first argument is a tuple {FuncName,GroupName}, - otherwise simply the function name.

- -

The data which comes with the Reason follows the same format as - tc_auto_skip - and - tc_user_skip events. - See Event Handling - in the User's Guide for details.

+

OPTIONAL

+ +

This function is called whenever a test case (or configuration + function) is skipped. It is called after the post function is called + for the skipped test case, that is:

+ + +

If init_per_group is skipped, this function is + called after + post_init_per_group.

 +

If a test case is skipped, this function is called after + post_end_per_testcase.

 + + +

If the skipped test case belongs to a test case group, the first + argument is a tuple {FuncName,GroupName}, otherwise only + the function name.

+ +

The data that comes with Reason follows the same format as + events + tc_auto_skip + and + tc_user_skip + For details, see section + Event Handling + in the User's Guide.

 Module:terminate(CTHState) - Called after the CTH scope ends + Called after the CTH scope ends. - CTHState = term() + CTHState = term() - -

OPTIONAL

+

OPTIONAL

-

This function is called at the end of a CTH's - scope. -

+

This function is called at the end of a CTH + scope.

 Module:id(Opts) -> Id - Called before the init function of a CTH + Called before the init function of a CTH. - Opts = term() - Id = term() + Opts = term() + Id = term() - -

OPTIONAL

- -

The Id is used to uniquely identify a CTH instance, - if two CTH's return the same Id the second CTH is ignored - and subsequent calls to the CTH will only be made to the first - instance. For more information see - Installing a CTH - in the User's Guide. -

- -

This function should NOT have any side effects as it might - be called multiple times by Common Test.

+

OPTIONAL

-

If not implemented the CTH will act as if this function returned a - call to make_ref/0.

- +

The Id identifies a CTH instance uniquely. If two CTHs return + the same Id, the second CTH is ignored and subsequent calls to + the CTH are only made to the first instance. For details, see section + Installing a CTH + in the User's Guide.

+ +

This function is not to have any side effects, as it can + be called multiple times by Common Test.

+ +

If not implemented, the CTH acts as if this function returned a call + to make_ref/0.

+ - -- cgit v1.2.3