aboutsummaryrefslogtreecommitdiffstats
path: root/lib/common_test/src/ct_run.erl
AgeCommit message (Collapse)Author
2014-06-05Improve cover analysis via common_testSiri Hansen
This addresses several bugs in common_test (ct) when using the cover analysis mechanism: In a ct test run, one can give a cover spec file which indicates that cover analysis shall be run, including a number of modules and a number of nodes. During the ct test run, multiple jobs may be started in test_server, and when the cover option is used, test_server would cover compile and analyse all given modules for each job. This commit instead allows the compilation and analysis to be explicitly ordered by ct for each test run. This way each module will only be cover compiled and analysed once. The cover log will be located in the common_test log directory (ct_run.<timestamp>), and the "Coverage log" link in each suite.log.html will point to this file. A new button is also added in the top level ct index file, which points to the cover log. This change also reduces the need of using the 'export' and 'import' options, since there is no longer any need to accumulate cover data over multiple test_server jobs. However, these options may still be used for importing and exporting cover data in order to store or accumulate data from multiple ct test runs. The 'nodes' option was earlier only used by ct to start cover on the given nodes before starting the first test_server job. After this job was completed, test_server would stop cover completely and then start it again for the next job without any knowledge of the 'nodes' options. For the next test_server jobs cover would therefore no longer be running on these nodes. Explcit calls to ct_cover:add_nodes had to be done in order to collect data through all test_server jobs. This bug has now been solved, since cover is no longer stopped between each test_server job. Finally, ct no longer stores cover data using ct_util:set_testdata. This was earlier used by ct_cover:add_nodes to make sure no node was added twice.This did, however, cause some problems when ct and cover were out of sync. ct could belive that a node was running cover and thus reject adding this node, while in reality cover had been stopped on the node (e.g. by test_server) so no cover data was collected. ct_cover:add_nodes will now instead use cover:which_nodes to check if a node is already running.
2014-03-05Add flag to abort test run if suites fail to compilePeter Andersson
OTP-11769
2013-09-05Add missing whitespace in stringPeter Andersson
2013-06-05Improve error report if crash due to bad testspecPeter Andersson
2013-04-17Merge branch 'peppe/common_test/log_cache' into maintPeter Andersson
* peppe/common_test/log_cache: Implement cache for the CT logger OTP-10855
2013-04-17Implement cache for the CT loggerPeter Andersson
OTP-10855
2013-03-25[common_test] Add -force_stop skip_rest option when repeating testsSiri Hansen
To allow a repeated test to finish immediatly, 'skip_rest' option can now be used in combination with 'force_stop'. If this option is used, only the current testcase will be completed after timeout is reached when repeating with 'duration' or 'until'.
2013-02-19Merge remote branch 'origin/peppe/common_test/force_stop'Peter Andersson
* origin/peppe/common_test/force_stop: Repair broken force_stop functionality OTP-10832
2013-02-18Repair broken force_stop functionalityPeter Andersson
2013-02-15Solve problem with IO deadlock in ct_util_serverPeter Andersson
2013-01-27Merge branch 'peppe/common_test/testspec_include'Peter Andersson
* peppe/common_test/testspec_include: Add more tests Update documentation Add tests and correct errors Update CT Master Implement support for including test specifications Make it possible to execute one test run per test specification Conflicts: lib/common_test/src/ct_master.erl lib/common_test/src/ct_run.erl lib/common_test/src/ct_testspec.erl OTP-9881
2013-01-25[common_test] Update common test modules to handle unicodeSiri Hansen
* Use UTF-8 encoding for all HTML files, except the HTML version of the test suite generated with erl2html2:convert, which will have the same encoding as the original test suite (.erl) file. * Encode link targets in HTML files with test_server_ctrl:uri_encode/1. * Use unicode modifier 't' with ~s when appropriate. * Use unicode:characters_to_list and unicode:characters_to_binary for conversion between binaries and strings instead of binary_to_list and list_to_binary.
2013-01-24Implement support for including test specificationsPeter Andersson
OTP-9881
2013-01-24Make it possible to execute one test run per test specificationPeter Andersson
OTP-9881
2012-11-20Merge branch 'maint'Peter Andersson
Conflicts: lib/common_test/test/Makefile
2012-11-20Merge remote branch 'origin/peppe/common_test/group_search_r15b03' into maintPeter Andersson
* origin/peppe/common_test/group_search_r15b03: Fix problem with test case order in group specifications Finish the test suite and correct remaining bugs Implement new group search functionality OTP-10466
2012-11-20Merge branch 'maint'Peter Andersson
2012-11-19Finish the test suite and correct remaining bugsPeter Andersson
OTP-10466
2012-11-19Implement new group search functionalityPeter Andersson
OTP-10466
2012-11-19Fix problem with config start option not accepting list of filesPeter Andersson
OTP-10495
2012-10-30[common_test] Add option cover_stopSiri Hansen
By default, test_server will always stop cover (and thus load back the non cover compiled original beam files) after a test with code coverage analysis is completed. The new option allows the common_test user to specify that cover shall not be stopped. This can be useful if there are processes still running old code, i.e. processes that have not done any fully qualified function call after the cover compilation, since loading the original code then will kill those processes. This is only recommended if the erlang node is to be stopped after the test run, or if cover can be manually stopped.
2012-09-19Teach test_server to report severe errors to common_testBjörn Gustavsson
If a severe error occurs in test_server (e.g. failing to write to log files), test_server would terminate without commont_test knowing about it. Since ct_run can now return an exit code, it is important that common_test is aware of severe problem so that it can indicate that an error has occurred.
2012-09-10Fix broken interactive modePeter Andersson
2012-08-27Make sure the test can never hang when ct:break/1/2 is calledPeter Andersson
2012-08-24Add silent_connections term to test specificationPeter Andersson
2012-08-23Fix IO bug causing CT Hooks to hang at startupPeter Andersson
2012-08-23Merge remote branch 'origin/peppe/common_test/break_and_continue' into maintPeter Andersson
* origin/peppe/common_test/break_and_continue: Fix error in documentation Implement support for test case execution break/continue Conflicts: lib/common_test/src/ct.erl lib/common_test/src/ct_run.erl OTP-10127 OTP-10172
2012-08-23Merge remote branch 'origin/peppe/common_test/verbosity_level' into maintPeter Andersson
* origin/peppe/common_test/verbosity_level: Fix doc build error Implement verbosity levels and parameter for log printout importance Conflicts: lib/common_test/src/ct_logs.erl lib/common_test/src/ct_run.erl lib/common_test/src/ct_testspec.erl lib/common_test/src/ct_util.hrl lib/common_test/test/Makefile OTP-9625 OTP-10067
2012-08-23Merge remote branch 'origin/peppe/common_test/testspec_updates' into maintPeter Andersson
* origin/peppe/common_test/testspec_updates: Test new testspec functionality and correct bugs Implement constants and new features for test specifications Conflicts: lib/common_test/src/ct_run.erl lib/common_test/test/Makefile OTP-10049 OTP-10089 OTP-10145
2012-08-21Test new testspec functionality and correct bugsPeter Andersson
OTP-10049 OTP-10089
2012-08-21Implement constants and new features for test specificationsPeter Andersson
OTP-10049 OTP-10089
2012-08-08Implement support for test case execution break/continuePeter Andersson
2012-07-13Implement verbosity levels and parameter for log printout importancePeter Andersson
2012-05-15Implement support for returning valid exit status from ct_runPeter Andersson
Also change the return value of the ct:run_test/1 function. OTP-9865 OTP-10087
2012-03-30Update copyright yearsBjörn-Egil Dahlberg
2012-03-23Merge branch ↵Lukas Larsson
'lukas/common_test/fix_disable_builtin_hooks_from_spec/OTP-10009' into maint * lukas/common_test/fix_disable_builtin_hooks_from_spec/OTP-10009: Change default of enable builtin hooks to undefined Disable built in hooks for emulator
2012-03-21Change default of enable builtin hooks to undefinedLukas Larsson
2012-03-19Fix bug in handling ct_hooks start argumentsPeter Andersson
2012-03-19Run hooks for non-existing config functionsPeter Andersson
2012-03-15Merge remote branch 'origin/peppe/common_test/otp-9634_9659' into maintPeter Andersson
* origin/peppe/common_test/otp-9634_9659: Change the option name 'unique_priv_dir' to 'create_priv_dir' Add test suite for the unique_priv_dir feature Make it possible to use unique priv_dir names Create entry for every repeated test in index.html OTP-9634 OTP-9659
2012-03-14Merge remote branch 'origin/peppe/common_test/otp-9899' into maintPeter Andersson
* origin/peppe/common_test/otp-9899: Fix broken repeat functionality OTP-9899
2012-03-13Change the option name 'unique_priv_dir' to 'create_priv_dir'Peter Andersson
The new values for create_priv_dir are: auto_per_run, auto_per_tc, manual_per_tc
2012-03-13Add test suite for the unique_priv_dir featurePeter Andersson
2012-03-13Make it possible to use unique priv_dir namesPeter Andersson
Introduce the optional feature to have Test Server generate priv_dir directory names that are unique for each test case or config function. The name of the option/flag is 'unique_priv_dir' and it can be set to value 'auto' or 'manual'. If auto, Test Server creates each priv_dir automatically (can be expensive in case of many and/or repeated cases). If manual, the user needs to create the priv_dir explicitly by calling ct:make_priv_dir/0.
2012-03-13Fix broken repeat functionalityPeter Andersson
OTP-9899
2012-03-12Have ct_run clean up the code path after each test runPeter Andersson
OTP-9595
2012-03-12Fix problems with ct_run's code path manipulationPeter Andersson
OTP-9964 OTP-9975
2011-12-07Implement support for overriding group properties with test specificationPeter Andersson
2011-10-04Merge branch 'dev' into majorLukas Larsson
* dev: (38 commits) Update documentation Rid ct_telnet of doc build warnings Create temporary fix for problem with parallel test cases Update primary bootstrap Correct "Missing Suites" link Add documentation on timetraps and start flags Add missing tests for timetrap handling and fix remaining errors Solve problem with ct_init/end_per_group being counted as test cases Fix errors in test suites Fix invalid call to undefined function Fix problem with test_server_ctrl creating invalid conf test Improve info in CT framework log Update vsn.mk for common_test and test_server Enhance logging performance Change order of include files Add link to last executed test suite on index page Fix problem with location value when init config func calls help func Fix crash when CTHook init fails Correct error in test suite Fix error with incorrect notification after end_per_testcase craches ... Conflicts: bootstrap/bin/start.boot bootstrap/bin/start_clean.boot bootstrap/lib/compiler/ebin/beam_asm.beam bootstrap/lib/compiler/ebin/beam_disasm.beam bootstrap/lib/compiler/ebin/compile.beam bootstrap/lib/compiler/ebin/sys_pre_expand.beam bootstrap/lib/kernel/ebin/code.beam bootstrap/lib/kernel/ebin/code_server.beam bootstrap/lib/kernel/ebin/hipe_unified_loader.beam bootstrap/lib/kernel/ebin/inet.beam bootstrap/lib/kernel/ebin/inet_config.beam bootstrap/lib/kernel/ebin/inet_dns.beam bootstrap/lib/stdlib/ebin/beam_lib.beam bootstrap/lib/stdlib/ebin/dets.beam bootstrap/lib/stdlib/ebin/erl_compile.beam bootstrap/lib/stdlib/ebin/erl_internal.beam bootstrap/lib/stdlib/ebin/erl_scan.beam bootstrap/lib/stdlib/ebin/erl_tar.beam bootstrap/lib/stdlib/ebin/io_lib_fread.beam bootstrap/lib/stdlib/ebin/otp_internal.beam bootstrap/lib/stdlib/ebin/sofs.beam bootstrap/lib/stdlib/ebin/supervisor.beam bootstrap/lib/stdlib/ebin/zip.beam lib/common_test/src/ct.erl lib/common_test/src/ct_run.erl lib/common_test/test/ct_error_SUITE.erl lib/common_test/test/ct_repeat_1_SUITE.erl lib/common_test/test/ct_skip_SUITE.erl lib/test_server/src/test_server.erl
2011-09-27Fix invalid call to undefined functionPeter Andersson
OTP-9585
generated by cgit v1.2.3 (git 2.25.1) at 2024-09-04 10:01:44 +0000