<feed xmlns='http://www.w3.org/2005/Atom'>
<title>otp.git/lib/common_test/test, branch maint-19</title>
<subtitle>Mirror of Erlang/OTP repository.
</subtitle>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/'/>
<entry>
<title>Update copyright year</title>
<updated>2017-03-10T14:00:46+00:00</updated>
<author>
<name>Rickard Green</name>
<email>rickard@erlang.org</email>
</author>
<published>2017-03-10T14:00:46+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=26c3cd82529836cb5b6eefbf7f92f318fd91f847'/>
<id>26c3cd82529836cb5b6eefbf7f92f318fd91f847</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'siri/ct_hooks/callbacks-on-skip/OTP-10599' into maint</title>
<updated>2017-03-06T07:51:22+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-03-06T07:51:22+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=0e0155410734e04556ceecb1222dd2ef956b4177'/>
<id>0e0155410734e04556ceecb1222dd2ef956b4177</id>
<content type='text'>
* siri/ct_hooks/callbacks-on-skip/OTP-10599:
  [ct] Update built-in ct hooks with new Suite parameter
  [cth_surefire] Handle skips from test spec
  [ct] Remove excessive skipped/failed tag in hook function
  [ct] Send tc_start event on force_stop and failed sequence
  [ct] Correctly handle process died in init and end_per_testcase
  [ct] Fix faulty hook callbacks for skipped tests
  Add dummy end_per_suite/1
  [ct] Fix hooks and fail when one of init/end_per_* does not exit
  [ct] Add Suite argument to hook callback functions
  [ct] Fix function_clause in ct_framework when hook function crashes
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* siri/ct_hooks/callbacks-on-skip/OTP-10599:
  [ct] Update built-in ct hooks with new Suite parameter
  [cth_surefire] Handle skips from test spec
  [ct] Remove excessive skipped/failed tag in hook function
  [ct] Send tc_start event on force_stop and failed sequence
  [ct] Correctly handle process died in init and end_per_testcase
  [ct] Fix faulty hook callbacks for skipped tests
  Add dummy end_per_suite/1
  [ct] Fix hooks and fail when one of init/end_per_* does not exit
  [ct] Add Suite argument to hook callback functions
  [ct] Fix function_clause in ct_framework when hook function crashes
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'siri/ct/get_dirs_from_testspec/OTP-14132' into maint</title>
<updated>2017-02-22T09:04:47+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-02-22T09:04:47+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=b1d9a9f0785ded0a70a4e0669a6e2b5234ab1163'/>
<id>b1d9a9f0785ded0a70a4e0669a6e2b5234ab1163</id>
<content type='text'>
* siri/ct/get_dirs_from_testspec/OTP-14132:
  [ct] Add ct_testspec:get_tests/1
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* siri/ct/get_dirs_from_testspec/OTP-14132:
  [ct] Add ct_testspec:get_tests/1
</pre>
</div>
</content>
</entry>
<entry>
<title>[cth_surefire] Handle skips from test spec</title>
<updated>2017-02-21T13:30:02+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-01-09T14:41:20+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=b3f2b1d0932f3087fb57c5be30c29c08f4f6244b'/>
<id>b3f2b1d0932f3087fb57c5be30c29c08f4f6244b</id>
<content type='text'>
When a test case is skipped from the test spec, there is no call to
the common_test hook functions 'pre_init_per_testcase' and
'post_end_per_testcase'. This causes cth_surefire to erroneously mark
the previous test case as skipped in the xml report. The actually
skipped test case is not present in the xml report at all. This is now
corrected.

OTP-14129
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When a test case is skipped from the test spec, there is no call to
the common_test hook functions 'pre_init_per_testcase' and
'post_end_per_testcase'. This causes cth_surefire to erroneously mark
the previous test case as skipped in the xml report. The actually
skipped test case is not present in the xml report at all. This is now
corrected.

OTP-14129
</pre>
</div>
</content>
</entry>
<entry>
<title>[ct] Remove excessive skipped/failed tag in hook function</title>
<updated>2017-02-20T11:40:09+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-02-16T16:17:06+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=1a5ba828bf7dda17bdd874b837a3dc8898952d6c'/>
<id>1a5ba828bf7dda17bdd874b837a3dc8898952d6c</id>
<content type='text'>
When skipping or failing in a configuration function, the
configuration function itself would get {auto_skipped,Reason},
{skipped,Reason} or {failed,Reason} in the hook callbacks on_tc_skip
or on_tc_fail. The other test cases that were skipped as a result of
this would only get Reason in the hook callbacks. This is now
corrected.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When skipping or failing in a configuration function, the
configuration function itself would get {auto_skipped,Reason},
{skipped,Reason} or {failed,Reason} in the hook callbacks on_tc_skip
or on_tc_fail. The other test cases that were skipped as a result of
this would only get Reason in the hook callbacks. This is now
corrected.
</pre>
</div>
</content>
</entry>
<entry>
<title>[ct] Send tc_start event on force_stop and failed sequence</title>
<updated>2017-02-20T11:40:09+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-02-16T13:15:32+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=8151495812ffabf560958e5434d3edbe46f29695'/>
<id>8151495812ffabf560958e5434d3edbe46f29695</id>
<content type='text'>
If test cases were skipped because of option {force_stop,skip_rest} or
because of a failed sequence, then no tc_start event would be sent,
only only tc_done. This is now corrected.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
If test cases were skipped because of option {force_stop,skip_rest} or
because of a failed sequence, then no tc_start event would be sent,
only only tc_done. This is now corrected.
</pre>
</div>
</content>
</entry>
<entry>
<title>[ct] Correctly handle process died in init and end_per_testcase</title>
<updated>2017-02-20T11:40:08+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-02-16T11:16:21+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=827732949ba9cd844347bca1a46b2e2055810e69'/>
<id>827732949ba9cd844347bca1a46b2e2055810e69</id>
<content type='text'>
Documentation says that a test case is skipped if init_per_testcase
fails. This is correct if init_per_testcase fails in a normal way, but
if it gets an exit signal causing the process to die, then the case
was earlier marked as failed with reason {skip,...}. This is now
corrected, so the case is auto skipped.

If end_per_testcase crashes, it does not change the result of a test
case, it only adds a warning in the comment field. But if it gets an
exit signal causing the process to die while running end_per_testcase,
then the case was earlier marked as failed in addition to this
warning. This is now corrected, so the result of the test case is not
changed by an exit signal during end_per_testcase.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Documentation says that a test case is skipped if init_per_testcase
fails. This is correct if init_per_testcase fails in a normal way, but
if it gets an exit signal causing the process to die, then the case
was earlier marked as failed with reason {skip,...}. This is now
corrected, so the case is auto skipped.

If end_per_testcase crashes, it does not change the result of a test
case, it only adds a warning in the comment field. But if it gets an
exit signal causing the process to die while running end_per_testcase,
then the case was earlier marked as failed in addition to this
warning. This is now corrected, so the result of the test case is not
changed by an exit signal during end_per_testcase.
</pre>
</div>
</content>
</entry>
<entry>
<title>[ct] Fix faulty hook callbacks for skipped tests</title>
<updated>2017-02-20T11:40:08+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-02-16T14:38:05+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=6b7463094db2ec3c2c0e4f8784dcd64ef186c96b'/>
<id>6b7463094db2ec3c2c0e4f8784dcd64ef186c96b</id>
<content type='text'>
The following bugs are corrected:

* Tests that were skipped before calling pre_init_per_* got faulty
  calls to the corresponding post_init_per_*. E.g. if a test was
  skipped because suite/0 returned a 'require' statement that was not
  fulfilled, then post_init_per_suite would be called, even though
  pre_init_per_suite and init_per_suite were not called.

* Tests that were skipped before or in init_per_testcase got faulty
  calls to pre_end_per_testcase and post_end_per_testcase
  (end_per_testcase is not called in these situations).

Test are added to make sure that the expected callbacks, and only
those, are called when tests are skipped in different ways.

Conflicts:
	lib/common_test/test/ct_hooks_SUITE.erl
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The following bugs are corrected:

* Tests that were skipped before calling pre_init_per_* got faulty
  calls to the corresponding post_init_per_*. E.g. if a test was
  skipped because suite/0 returned a 'require' statement that was not
  fulfilled, then post_init_per_suite would be called, even though
  pre_init_per_suite and init_per_suite were not called.

* Tests that were skipped before or in init_per_testcase got faulty
  calls to pre_end_per_testcase and post_end_per_testcase
  (end_per_testcase is not called in these situations).

Test are added to make sure that the expected callbacks, and only
those, are called when tests are skipped in different ways.

Conflicts:
	lib/common_test/test/ct_hooks_SUITE.erl
</pre>
</div>
</content>
</entry>
<entry>
<title>[ct] Fix hooks and fail when one of init/end_per_* does not exit</title>
<updated>2017-02-20T11:40:08+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-02-16T14:36:07+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=c802f5d8d0ea08154005abfdcb0d958f126f26d1'/>
<id>c802f5d8d0ea08154005abfdcb0d958f126f26d1</id>
<content type='text'>
The following bugs are corrected:

- if init_per_suite is exported from a test suite, but not
  end_per_suite, then pre/post_end_per_suite will be called with
  Suite=ct_framework instead of the correct suite name.

- if end_per_group is exported from a suite, but not init_per_group,
  then end_per_group is never called.

According to the documentation, if implementing an init config
function, you must also implement the end config function, so the two
scenarios above are really not allowed. To make this more visible,
common_test will now mark the non-exported config function as failed
with reason 'undef' if the other function is exported.

For example, if init_per_suite is exported, but not end_per_suite,
then end_per_suite will be marked as failed with reason undef. (If
none of them exist, then they will both be marked as passed since the
default functions in ct_framework are called instead.)

All hook functions are always called with the correct suite name,
i.e. never with Suite=ct_framework.

Conflicts:
	lib/common_test/test/ct_hooks_SUITE.erl
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The following bugs are corrected:

- if init_per_suite is exported from a test suite, but not
  end_per_suite, then pre/post_end_per_suite will be called with
  Suite=ct_framework instead of the correct suite name.

- if end_per_group is exported from a suite, but not init_per_group,
  then end_per_group is never called.

According to the documentation, if implementing an init config
function, you must also implement the end config function, so the two
scenarios above are really not allowed. To make this more visible,
common_test will now mark the non-exported config function as failed
with reason 'undef' if the other function is exported.

For example, if init_per_suite is exported, but not end_per_suite,
then end_per_suite will be marked as failed with reason undef. (If
none of them exist, then they will both be marked as passed since the
default functions in ct_framework are called instead.)

All hook functions are always called with the correct suite name,
i.e. never with Suite=ct_framework.

Conflicts:
	lib/common_test/test/ct_hooks_SUITE.erl
</pre>
</div>
</content>
</entry>
<entry>
<title>[ct] Add Suite argument to hook callback functions</title>
<updated>2017-02-20T11:40:08+00:00</updated>
<author>
<name>Siri Hansen</name>
<email>siri@erlang.org</email>
</author>
<published>2017-01-25T10:24:58+00:00</published>
<link rel='alternate' type='text/html' href='http://git.ninenines.eu/otp.git/commit/?id=a95a22017523229bb8924afbd928d3e16b564fa5'/>
<id>a95a22017523229bb8924afbd928d3e16b564fa5</id>
<content type='text'>
An extra argument, Suite, is added as the first argument to each of
the following hook callback functions:

  - pre_init_per_group
  - post_init_per_group
  - pre_end_per_group
  - post_end_per_group
  - pre_init_per_testcase
  - post_init_per_testcase
  - pre_end_per_testcase
  - post_end_per_testcase
  - on_tc_fail
  - on_tc_skip

For backwards compatibility, if the new function is not exported from
a hook callback module, common_test will fall back to the old
interface and call the function without the Suite argument.

The reason for adding the new argument is that if a test suite is
skipped by a 'skip_suites' statement in the test specification, then
there will be no call to pre/post_init_per_suite, and thus the hook
has no other way of knowing which Suite is skipped when it gets the
on_tc_skip callback. The other callbacks are updated for symmetry.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
An extra argument, Suite, is added as the first argument to each of
the following hook callback functions:

  - pre_init_per_group
  - post_init_per_group
  - pre_end_per_group
  - post_end_per_group
  - pre_init_per_testcase
  - post_init_per_testcase
  - pre_end_per_testcase
  - post_end_per_testcase
  - on_tc_fail
  - on_tc_skip

For backwards compatibility, if the new function is not exported from
a hook callback module, common_test will fall back to the old
interface and call the function without the Suite argument.

The reason for adding the new argument is that if a test suite is
skipped by a 'skip_suites' statement in the test specification, then
there will be no call to pre/post_init_per_suite, and thus the hook
has no other way of knowing which Suite is skipped when it gets the
on_tc_skip callback. The other callbacks are updated for symmetry.
</pre>
</div>
</content>
</entry>
</feed>
