| Age | Commit message (Collapse) | Author |
|---|
|
* siri/ct_hooks/callbacks-on-skip/OTP-10599:
[ct] Update built-in ct hooks with new Suite parameter
[cth_surefire] Handle skips from test spec
[ct] Remove excessive skipped/failed tag in hook function
[ct] Send tc_start event on force_stop and failed sequence
[ct] Correctly handle process died in init and end_per_testcase
[ct] Fix faulty hook callbacks for skipped tests
Add dummy end_per_suite/1
[ct] Fix hooks and fail when one of init/end_per_* does not exit
[ct] Add Suite argument to hook callback functions
[ct] Fix function_clause in ct_framework when hook function crashes
|
|
* siri/ct/get_dirs_from_testspec/OTP-14132:
[ct] Add ct_testspec:get_tests/1
|
|
When a test case is skipped from the test spec, there is no call to
the common_test hook functions 'pre_init_per_testcase' and
'post_end_per_testcase'. This causes cth_surefire to erroneously mark
the previous test case as skipped in the xml report. The actually
skipped test case is not present in the xml report at all. This is now
corrected.
OTP-14129
|
|
When skipping or failing in a configuration function, the
configuration function itself would get {auto_skipped,Reason},
{skipped,Reason} or {failed,Reason} in the hook callbacks on_tc_skip
or on_tc_fail. The other test cases that were skipped as a result of
this would only get Reason in the hook callbacks. This is now
corrected.
|
|
If test cases were skipped because of option {force_stop,skip_rest} or
because of a failed sequence, then no tc_start event would be sent,
only only tc_done. This is now corrected.
|
|
Documentation says that a test case is skipped if init_per_testcase
fails. This is correct if init_per_testcase fails in a normal way, but
if it gets an exit signal causing the process to die, then the case
was earlier marked as failed with reason {skip,...}. This is now
corrected, so the case is auto skipped.
If end_per_testcase crashes, it does not change the result of a test
case, it only adds a warning in the comment field. But if it gets an
exit signal causing the process to die while running end_per_testcase,
then the case was earlier marked as failed in addition to this
warning. This is now corrected, so the result of the test case is not
changed by an exit signal during end_per_testcase.
|
|
The following bugs are corrected:
* Tests that were skipped before calling pre_init_per_* got faulty
calls to the corresponding post_init_per_*. E.g. if a test was
skipped because suite/0 returned a 'require' statement that was not
fulfilled, then post_init_per_suite would be called, even though
pre_init_per_suite and init_per_suite were not called.
* Tests that were skipped before or in init_per_testcase got faulty
calls to pre_end_per_testcase and post_end_per_testcase
(end_per_testcase is not called in these situations).
Test are added to make sure that the expected callbacks, and only
those, are called when tests are skipped in different ways.
Conflicts:
lib/common_test/test/ct_hooks_SUITE.erl
|
|
The following bugs are corrected:
- if init_per_suite is exported from a test suite, but not
end_per_suite, then pre/post_end_per_suite will be called with
Suite=ct_framework instead of the correct suite name.
- if end_per_group is exported from a suite, but not init_per_group,
then end_per_group is never called.
According to the documentation, if implementing an init config
function, you must also implement the end config function, so the two
scenarios above are really not allowed. To make this more visible,
common_test will now mark the non-exported config function as failed
with reason 'undef' if the other function is exported.
For example, if init_per_suite is exported, but not end_per_suite,
then end_per_suite will be marked as failed with reason undef. (If
none of them exist, then they will both be marked as passed since the
default functions in ct_framework are called instead.)
All hook functions are always called with the correct suite name,
i.e. never with Suite=ct_framework.
Conflicts:
lib/common_test/test/ct_hooks_SUITE.erl
|
|
An extra argument, Suite, is added as the first argument to each of
the following hook callback functions:
- pre_init_per_group
- post_init_per_group
- pre_end_per_group
- post_end_per_group
- pre_init_per_testcase
- post_init_per_testcase
- pre_end_per_testcase
- post_end_per_testcase
- on_tc_fail
- on_tc_skip
For backwards compatibility, if the new function is not exported from
a hook callback module, common_test will fall back to the old
interface and call the function without the Suite argument.
The reason for adding the new argument is that if a test suite is
skipped by a 'skip_suites' statement in the test specification, then
there will be no call to pre/post_init_per_suite, and thus the hook
has no other way of knowing which Suite is skipped when it gets the
on_tc_skip callback. The other callbacks are updated for symmetry.
|
|
* egil/cuddle-tests:
stdlib: Increase timetrap for rand_SUITE
common_test: Increase timetrap for cth_hooks_SUITE
compiler: Increase timetrap timeouts for lc_SUITE
|
|
This API function is used by rebar3 instead of implementing test spec
parsing again.
|
|
|
|
|
|
* valgrind needs a lot of time
|
|
epp learned to handle the -warning() directive in 14d72f02, but
make and ct_make were not updated to expect a {warning,_} return
value from epp:parse_erl_form/1.
|
|
|
|
Netconf client tests fail every now and then with
'table_trans_timeout' with no obvious reason. A monitor of the maint
netconf server process is now added - in case this process crashes
during the transaction.
|
|
Tests using system time can be skipped on test hosts set up to do a
lot of time warps.
|
|
The ssh application is not installed on all test hosts. Catching the
call to ssh:start/0 allows the the netconf tests to be nicely skipped
if ssh does not exist.
|
|
* siri/cuddle-master:
Add short sleep when killing slave nodes
[ct test] Do reload_config instead of get_config
|
|
* peppe/common_test/OTP-13615:
Fix problem with incorrect type of timestamps
OTP-13615
|
|
OTP-13615
|
|
The test case config_dynamic_SUITE:test_localtime_update sometimes
failed on slow machines, since the initial value of 'localtime' is
fetched with ct:get_config instead of ct:reload_config. The value is
then from the test run start, not from the time of the ct:get_config
call. This is now corrected.
|
|
* siri/cuddle-master:
[ct test] Wait for event receiver to be unregistered
[ct test] Flush messages from old client after close in telnet server
[ct test] Use monitor instead of ct:sleep
[ct test] Check that crypto is loaded, or skip netconf tests
[ct test] Reduce amount of data in netconfc1_SUITE:get_a_lot
|
|
To avoid badarg when two tests are run within the same test case.
|
|
Since the same erlang process is used for subsequent telnet
connections, messages from old clients must be flushed after close.
|
|
ct_gen_conn_SUITE_data/conn_SUITE.erl uses a lot of ct:sleep(100) when
waiting for processes to exit. This commit removes all the sleeps and
introducess monitor instead.
|
|
Tests for ct_netconfc are failed instead of skipped if crypto can not
be loaded. This is now corrected.
|
|
This test case often fails with table_trans_timeout in the netconf
server (ns.erl) on slow machines. The amount of data was originally
reduced by commit 976214f8d738d4852348496df79f84264d899aba, but was
faulty reverted by commit 171d7e2a161ef9270240aff0fa15a285df21c1ef. It
is now reduced again...
|
|
* peppe/common_test/log_tests:
Write suite that tests the API for printing to log
|
|
* peppe/common_test/fix_failing_tests:
Make sure test node shuts down before skipping suite
Measure file i/o overhead and skip test if the speed is too slow
Skip test cases if cover or debug is running
|
|
|
|
|
|
|
|
* siri/cuddle-master:
Change ct:sleep to timer:sleep where scaling is not wanted
Don't generate ssh key files in netconf client test
Ignore data received in break mode in test telnet_server
Wait for process to die before next call
Retry ct_telnet:get_data if no data is received after short wait
Don't throw rest of line when NOP is received in test telnet_server
Change ct:sleep to timer:sleep in telnet_server
|
|
|
|
|
|
Generating the dsa files can be very slow on some machines. Use
hardcoded files instead.
|
|
The only command handled in break mode is 'q' = 'quit break
mode'. Other data would earlier cause a function_clause
exception. Other data could e.g. be a NOP poll (keep alive). To fix
the problem, any data received in break mode, except 'q', will now be
ignored.
|
|
ct_telnet_own_server_SUITE:large_string tests that the client can
receive a chopped up string. To make sure the right thing is tested, a
SHORT timer is used before calling ct_telnet:get_data, but on some
slow machines the the timer is too short to allow data to be
received. To overcome this, the test now re-tries ct_telnet:get_data a
few times before giving up.
|
|
If telnet command NOP (No Operation) was received in the same tcp
package as other data, then the rest of the data would be regarded
further telnet commands (to proceed IAC, Interprete As Command) and
would never get to telnet_server:do_handle_data/2.
This is now corrected.
|
|
Conflicts:
OTP_VERSION
lib/common_test/test/ct_hooks_SUITE_data/cth/tests/ct_update_config_SUITE.erl
lib/common_test/vsn.mk
|
|
The telnet_server is run on the main test node which will scale and
multiply timers on some test hosts. The other side of the telnet test
(the client) is run on the slave, which does not inherit timer scaling
and multiplication. Therefore, it is better to use timer:sleep in the
server.
|
|
* siri/cuddle-master:
Extend timer in test
Update crashdump_viewer_SUITE with new release numbers
Set longer timetrap when creating dsa files
Check that ssh application exists before starting netconf tests
Improve error control when starting ssh in netconf test
|
|
* peppe/ct_misc_18_patches:
Add flag/option for disabling the character escaping functionality
Fix bug using the wrong lists search function
Fix bug with clashing timestamp values
Fix problem with stylesheet tags getting escaped
Skip pre/post test IO suite if cover or debug is running
Tweak pre_post_io test case to run without failing
Fix various log related problems
|
|
|
|
OTP-13535
The return value of ct:get_timetrap_info/0 has been modified.
|
|
|
|
|
|
ct_netconfc_SUITE ->
netconfc1_SUITE:close_while_waiting_for_chunked_data fails every now
and then with {error,timeout} instead of {error,closed}. To overcome
this, the timeout value in the failing call is now extended.
|
