diff options
218 files changed, 5906 insertions, 3928 deletions
diff --git a/.gitignore b/.gitignore index 3dcfa79f4d..455d3e5f55 100644 --- a/.gitignore +++ b/.gitignore @@ -360,15 +360,8 @@ JAVADOC-GENERATED # system -/system/doc/pdf/*.pdf -/system/doc/pdf/*.fo -/system/doc/html/*.html -/system/doc/html/*.eix -/system/doc/html/js -/system/doc/html/*/*.html -/system/doc/html/*/*.gif -/system/doc/html/*/*.jpg -/system/doc/html/*/*.eix +/system/doc/pdf +/system/doc/html /system/doc/top/PR.template /system/doc/top/erlresolvelinks.js /system/doc/programming_examples/funs.xml diff --git a/OTP_VERSION b/OTP_VERSION index 0b602e3cc8..6ea9a3bd47 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -19.0-rc0 +19.0 diff --git a/bootstrap/lib/compiler/ebin/beam_bool.beam b/bootstrap/lib/compiler/ebin/beam_bool.beam Binary files differindex a793db8bed..4a302a8709 100644 --- a/bootstrap/lib/compiler/ebin/beam_bool.beam +++ b/bootstrap/lib/compiler/ebin/beam_bool.beam diff --git a/bootstrap/lib/compiler/ebin/beam_split.beam b/bootstrap/lib/compiler/ebin/beam_split.beam Binary files differindex 3a02e2b17c..4a63468419 100644 --- a/bootstrap/lib/compiler/ebin/beam_split.beam +++ b/bootstrap/lib/compiler/ebin/beam_split.beam diff --git a/bootstrap/lib/compiler/ebin/beam_utils.beam b/bootstrap/lib/compiler/ebin/beam_utils.beam Binary files differindex 661cd1c60d..664140540c 100644 --- a/bootstrap/lib/compiler/ebin/beam_utils.beam +++ b/bootstrap/lib/compiler/ebin/beam_utils.beam diff --git a/bootstrap/lib/compiler/ebin/cerl.beam b/bootstrap/lib/compiler/ebin/cerl.beam Binary files differindex 82bd89c17a..dea6cb4ee2 100644 --- a/bootstrap/lib/compiler/ebin/cerl.beam +++ b/bootstrap/lib/compiler/ebin/cerl.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_clauses.beam b/bootstrap/lib/compiler/ebin/cerl_clauses.beam Binary files differindex 95dd3f3cbb..cc4e3a92a3 100644 --- a/bootstrap/lib/compiler/ebin/cerl_clauses.beam +++ b/bootstrap/lib/compiler/ebin/cerl_clauses.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_inline.beam b/bootstrap/lib/compiler/ebin/cerl_inline.beam Binary files differindex ce1f13db6a..dcfb32d866 100644 --- a/bootstrap/lib/compiler/ebin/cerl_inline.beam +++ b/bootstrap/lib/compiler/ebin/cerl_inline.beam diff --git a/bootstrap/lib/compiler/ebin/core_parse.beam b/bootstrap/lib/compiler/ebin/core_parse.beam Binary files differindex baf2c9d70f..920dbc922f 100644 --- a/bootstrap/lib/compiler/ebin/core_parse.beam +++ b/bootstrap/lib/compiler/ebin/core_parse.beam diff --git a/bootstrap/lib/compiler/ebin/core_pp.beam b/bootstrap/lib/compiler/ebin/core_pp.beam Binary files differindex 6047596cb6..e274211c6c 100644 --- a/bootstrap/lib/compiler/ebin/core_pp.beam +++ b/bootstrap/lib/compiler/ebin/core_pp.beam diff --git a/bootstrap/lib/compiler/ebin/rec_env.beam b/bootstrap/lib/compiler/ebin/rec_env.beam Binary files differindex 9388b5d4e1..a7c78175c3 100644 --- a/bootstrap/lib/compiler/ebin/rec_env.beam +++ b/bootstrap/lib/compiler/ebin/rec_env.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_fold.beam b/bootstrap/lib/compiler/ebin/sys_core_fold.beam Binary files differindex 088d69fa2a..7d01ed81d5 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_fold.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_fold.beam diff --git a/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam b/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam Binary files differindex 934a539ad9..58dff1b796 100644 --- a/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam +++ b/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam diff --git a/bootstrap/lib/compiler/ebin/v3_codegen.beam b/bootstrap/lib/compiler/ebin/v3_codegen.beam Binary files differindex 0d76bb21e2..0923d4e678 100644 --- a/bootstrap/lib/compiler/ebin/v3_codegen.beam +++ b/bootstrap/lib/compiler/ebin/v3_codegen.beam diff --git a/bootstrap/lib/compiler/ebin/v3_core.beam b/bootstrap/lib/compiler/ebin/v3_core.beam Binary files differindex 3b3b8f4208..7be50a757f 100644 --- a/bootstrap/lib/compiler/ebin/v3_core.beam +++ b/bootstrap/lib/compiler/ebin/v3_core.beam diff --git a/bootstrap/lib/compiler/ebin/v3_kernel.beam b/bootstrap/lib/compiler/ebin/v3_kernel.beam Binary files differindex 5e3ae80d14..47eaf3b5e8 100644 --- a/bootstrap/lib/compiler/ebin/v3_kernel.beam +++ b/bootstrap/lib/compiler/ebin/v3_kernel.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log.beam b/bootstrap/lib/kernel/ebin/disk_log.beam Binary files differindex 998d4bca2f..5aa8bf7dc1 100644 --- a/bootstrap/lib/kernel/ebin/disk_log.beam +++ b/bootstrap/lib/kernel/ebin/disk_log.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log_1.beam b/bootstrap/lib/kernel/ebin/disk_log_1.beam Binary files differindex fc1c4781f1..cb627c10a5 100644 --- a/bootstrap/lib/kernel/ebin/disk_log_1.beam +++ b/bootstrap/lib/kernel/ebin/disk_log_1.beam diff --git a/bootstrap/lib/kernel/ebin/file.beam b/bootstrap/lib/kernel/ebin/file.beam Binary files differindex c82dada559..28d72dfafc 100644 --- a/bootstrap/lib/kernel/ebin/file.beam +++ b/bootstrap/lib/kernel/ebin/file.beam diff --git a/bootstrap/lib/kernel/ebin/global.beam b/bootstrap/lib/kernel/ebin/global.beam Binary files differindex b461d30baa..465dab3a52 100644 --- a/bootstrap/lib/kernel/ebin/global.beam +++ b/bootstrap/lib/kernel/ebin/global.beam diff --git a/bootstrap/lib/kernel/ebin/group.beam b/bootstrap/lib/kernel/ebin/group.beam Binary files differindex 0511d1fa09..38eb0e7a6c 100644 --- a/bootstrap/lib/kernel/ebin/group.beam +++ b/bootstrap/lib/kernel/ebin/group.beam diff --git a/bootstrap/lib/kernel/ebin/inet.beam b/bootstrap/lib/kernel/ebin/inet.beam Binary files differindex 60469432b9..8cfdf389ad 100644 --- a/bootstrap/lib/kernel/ebin/inet.beam +++ b/bootstrap/lib/kernel/ebin/inet.beam diff --git a/bootstrap/lib/kernel/ebin/inet_db.beam b/bootstrap/lib/kernel/ebin/inet_db.beam Binary files differindex 557852c917..30ab40c93f 100644 --- a/bootstrap/lib/kernel/ebin/inet_db.beam +++ b/bootstrap/lib/kernel/ebin/inet_db.beam diff --git a/bootstrap/lib/kernel/ebin/inet_parse.beam b/bootstrap/lib/kernel/ebin/inet_parse.beam Binary files differindex 30f05d5c09..c5c171aa54 100644 --- a/bootstrap/lib/kernel/ebin/inet_parse.beam +++ b/bootstrap/lib/kernel/ebin/inet_parse.beam diff --git a/bootstrap/lib/kernel/ebin/kernel_config.beam b/bootstrap/lib/kernel/ebin/kernel_config.beam Binary files differindex 8691aff64a..8379f77e94 100644 --- a/bootstrap/lib/kernel/ebin/kernel_config.beam +++ b/bootstrap/lib/kernel/ebin/kernel_config.beam diff --git a/bootstrap/lib/stdlib/ebin/dets.beam b/bootstrap/lib/stdlib/ebin/dets.beam Binary files differindex 754663b531..0dfaca7a48 100644 --- a/bootstrap/lib/stdlib/ebin/dets.beam +++ b/bootstrap/lib/stdlib/ebin/dets.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_v9.beam b/bootstrap/lib/stdlib/ebin/dets_v9.beam Binary files differindex 3e02f4b0be..f381b9b624 100644 --- a/bootstrap/lib/stdlib/ebin/dets_v9.beam +++ b/bootstrap/lib/stdlib/ebin/dets_v9.beam diff --git a/bootstrap/lib/stdlib/ebin/digraph_utils.beam b/bootstrap/lib/stdlib/ebin/digraph_utils.beam Binary files differindex 66777eeb8f..80e91be663 100644 --- a/bootstrap/lib/stdlib/ebin/digraph_utils.beam +++ b/bootstrap/lib/stdlib/ebin/digraph_utils.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_eval.beam b/bootstrap/lib/stdlib/ebin/erl_eval.beam Binary files differindex 8793d70a39..d5bd46593c 100644 --- a/bootstrap/lib/stdlib/ebin/erl_eval.beam +++ b/bootstrap/lib/stdlib/ebin/erl_eval.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam Binary files differindex f6c0d0a732..e8ce1fa0da 100644 --- a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam +++ b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex aeea1e2fcd..ebe80a32a8 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_pp.beam b/bootstrap/lib/stdlib/ebin/erl_pp.beam Binary files differindex 9841362f9d..b3343c9ccb 100644 --- a/bootstrap/lib/stdlib/ebin/erl_pp.beam +++ b/bootstrap/lib/stdlib/ebin/erl_pp.beam diff --git a/bootstrap/lib/stdlib/ebin/file_sorter.beam b/bootstrap/lib/stdlib/ebin/file_sorter.beam Binary files differindex ac4f1a36f1..03d9020bb5 100644 --- a/bootstrap/lib/stdlib/ebin/file_sorter.beam +++ b/bootstrap/lib/stdlib/ebin/file_sorter.beam diff --git a/bootstrap/lib/stdlib/ebin/filelib.beam b/bootstrap/lib/stdlib/ebin/filelib.beam Binary files differindex a9783ed990..904edee66f 100644 --- a/bootstrap/lib/stdlib/ebin/filelib.beam +++ b/bootstrap/lib/stdlib/ebin/filelib.beam diff --git a/bootstrap/lib/stdlib/ebin/filename.beam b/bootstrap/lib/stdlib/ebin/filename.beam Binary files differindex b6248fe460..c129ac4f97 100644 --- a/bootstrap/lib/stdlib/ebin/filename.beam +++ b/bootstrap/lib/stdlib/ebin/filename.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_event.beam b/bootstrap/lib/stdlib/ebin/gen_event.beam Binary files differindex 2486647f46..b53ac26280 100644 --- a/bootstrap/lib/stdlib/ebin/gen_event.beam +++ b/bootstrap/lib/stdlib/ebin/gen_event.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_fsm.beam b/bootstrap/lib/stdlib/ebin/gen_fsm.beam Binary files differindex dbf38090af..595574585b 100644 --- a/bootstrap/lib/stdlib/ebin/gen_fsm.beam +++ b/bootstrap/lib/stdlib/ebin/gen_fsm.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_server.beam b/bootstrap/lib/stdlib/ebin/gen_server.beam Binary files differindex e7026a2a9a..3097207512 100644 --- a/bootstrap/lib/stdlib/ebin/gen_server.beam +++ b/bootstrap/lib/stdlib/ebin/gen_server.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_statem.beam b/bootstrap/lib/stdlib/ebin/gen_statem.beam Binary files differindex 5a3fd2a1a5..3dc104f1c3 100644 --- a/bootstrap/lib/stdlib/ebin/gen_statem.beam +++ b/bootstrap/lib/stdlib/ebin/gen_statem.beam diff --git a/bootstrap/lib/stdlib/ebin/lib.beam b/bootstrap/lib/stdlib/ebin/lib.beam Binary files differindex 541ea59421..06ddba221a 100644 --- a/bootstrap/lib/stdlib/ebin/lib.beam +++ b/bootstrap/lib/stdlib/ebin/lib.beam diff --git a/bootstrap/lib/stdlib/ebin/lists.beam b/bootstrap/lib/stdlib/ebin/lists.beam Binary files differindex 422a9bd214..22705c8a9a 100644 --- a/bootstrap/lib/stdlib/ebin/lists.beam +++ b/bootstrap/lib/stdlib/ebin/lists.beam diff --git a/bootstrap/lib/stdlib/ebin/otp_internal.beam b/bootstrap/lib/stdlib/ebin/otp_internal.beam Binary files differindex e1ebd34afd..ce8fc8a5c5 100644 --- a/bootstrap/lib/stdlib/ebin/otp_internal.beam +++ b/bootstrap/lib/stdlib/ebin/otp_internal.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc.beam b/bootstrap/lib/stdlib/ebin/qlc.beam Binary files differindex f0ed1e961b..fa3240810d 100644 --- a/bootstrap/lib/stdlib/ebin/qlc.beam +++ b/bootstrap/lib/stdlib/ebin/qlc.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc_pt.beam b/bootstrap/lib/stdlib/ebin/qlc_pt.beam Binary files differindex 17c698de60..4a065ec57f 100644 --- a/bootstrap/lib/stdlib/ebin/qlc_pt.beam +++ b/bootstrap/lib/stdlib/ebin/qlc_pt.beam diff --git a/bootstrap/lib/stdlib/ebin/queue.beam b/bootstrap/lib/stdlib/ebin/queue.beam Binary files differindex 4365c5518d..a63a321330 100644 --- a/bootstrap/lib/stdlib/ebin/queue.beam +++ b/bootstrap/lib/stdlib/ebin/queue.beam diff --git a/bootstrap/lib/stdlib/ebin/shell.beam b/bootstrap/lib/stdlib/ebin/shell.beam Binary files differindex 9fa5727aa5..b0c98f4b98 100644 --- a/bootstrap/lib/stdlib/ebin/shell.beam +++ b/bootstrap/lib/stdlib/ebin/shell.beam diff --git a/bootstrap/lib/stdlib/ebin/sofs.beam b/bootstrap/lib/stdlib/ebin/sofs.beam Binary files differindex 01f7810764..5099d4ecc0 100644 --- a/bootstrap/lib/stdlib/ebin/sofs.beam +++ b/bootstrap/lib/stdlib/ebin/sofs.beam diff --git a/erts/configure.in b/erts/configure.in index 11c428ced7..4a63381eb7 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -62,9 +62,6 @@ if test x"${ERL_TOP}/erts" != x"$srcdir"; then fi erl_top=${ERL_TOP} -# Remove old configuration information -/bin/rm -f "$ERL_TOP/erts/CONF_INFO" - # echo XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX # echo X # echo "X srcdir = $srcdir" @@ -102,7 +99,7 @@ ERL_XCOMP_SYSROOT_INIT AC_ISC_POSIX -AC_CONFIG_HEADER($host/config.h:config.h.in include/internal/$host/ethread_header_config.h:include/internal/ethread_header_config.h.in include/$host/erl_int_sizes_config.h:include/erl_int_sizes_config.h.in include/$host/erl_native_features_config.h:include/erl_native_features_config.h.in) +AC_CONFIG_HEADER($host/config.h:config.h.in include/internal/$host/ethread_header_config.h:include/internal/ethread_header_config.h.in include/$host/erl_int_sizes_config.h:include/erl_int_sizes_config.h.in) dnl ---------------------------------------------------------------------- dnl Optional features. dnl ---------------------------------------------------------------------- @@ -140,7 +137,7 @@ AS_HELP_STRING([--enable-dirty-schedulers], [enable dirty scheduler support]), [ case "$enableval" in no) enable_dirty_schedulers=no ;; *) enable_dirty_schedulers=yes ;; - esac ], enable_dirty_schedulers=no) + esac ], enable_dirty_schedulers=default) AC_ARG_ENABLE(smp-support, AS_HELP_STRING([--enable-smp-support], [enable smp support]) @@ -729,8 +726,27 @@ esac AC_SUBST(LIBCARBON) -dnl Check if we should/can build a sharing-preserving emulator +_search_path=/bin:/usr/bin:/usr/local/bin:$PATH +AC_PATH_PROG(RM, rm, false, $_search_path) +if test "$ac_cv_path_RM" = false; then + AC_MSG_ERROR([No 'rm' command found]) +fi + +AC_PATH_PROG(MKDIR, mkdir, false, $_search_path) +if test "$ac_cv_path_MKDIR" = false; then + AC_MSG_ERROR([No 'mkdir' command found]) +fi + +_search_path= + + +# Remove old configuration information. +# Next line should be placed after AC_PATH_PROG(RM, ...), but before +# first output to CONN_INFO. So this is just the right place. +$RM -f "$ERL_TOP/erts/CONF_INFO" + +dnl Check if we should/can build a sharing-preserving emulator AC_MSG_CHECKING(if we are building a sharing-preserving emulator) if test "$enable_sharing_preserving" = "yes"; then AC_DEFINE(SHCOPY, [1], @@ -760,27 +776,13 @@ if test "$ac_cv_prog_AR" = false; then AC_MSG_ERROR([No 'ar' command found in PATH]) fi -_search_path=/bin:/usr/bin:/usr/local/bin:$PATH - -AC_PATH_PROG(RM, rm, false, $_search_path) -if test "$ac_cv_path_RM" = false; then - AC_MSG_ERROR([No 'rm' command found]) -fi - -AC_PATH_PROG(MKDIR, mkdir, false, $_search_path) -if test "$ac_cv_path_MKDIR" = false; then - AC_MSG_ERROR([No 'mkdir' command found]) -fi - -_search_path= - # # Get programs needed for building the documentation # ## Delete previous failed configure results if test -f doc/CONF_INFO; then - rm doc/CONF_INFO + $RM doc/CONF_INFO fi AC_CHECK_PROGS(XSLTPROC, xsltproc) @@ -1004,6 +1006,23 @@ case $ERTS_BUILD_SMP_EMU in ;; esac +AC_MSG_CHECKING(whether dirty schedulers should be enabled) +case $ERTS_BUILD_SMP_EMU-$enable_dirty_schedulers in + yes-yes) + DIRTY_SCHEDULER_SUPPORT=yes;; + yes-default) + ## Maybe yes for OTP 19... + DIRTY_SCHEDULER_SUPPORT=no;; + no-default) + DIRTY_SCHEDULER_SUPPORT=no;; + no-yes) + AC_MSG_ERROR([No smp emulator will be built, but dirty schedulers requested]);; + *) + DIRTY_SCHEDULER_SUPPORT=no;; +esac +AC_MSG_RESULT($DIRTY_SCHEDULER_SUPPORT) +AC_SUBST(DIRTY_SCHEDULER_SUPPORT) + if test $ERTS_BUILD_SMP_EMU = yes; then if test $found_threads = no; then @@ -1206,14 +1225,6 @@ esac if test $emu_threads != yes; then enable_lock_check=no enable_lock_count=no - AC_MSG_CHECKING(whether dirty schedulers should be enabled) - if test "x$enable_dirty_schedulers" != "xno"; then - AC_DEFINE(ERL_NIF_DIRTY_SCHEDULER_SUPPORT, 1, [Dirty scheduler support]) - AC_DEFINE(ERL_DRV_DIRTY_SCHEDULER_SUPPORT, 1, [Dirty scheduler support]) - AC_MSG_RESULT(yes) - else - AC_MSG_RESULT(no) - fi else # Threads enabled for emulator EMU_THR_LIB_NAME=$ETHR_LIB_NAME @@ -1233,17 +1244,6 @@ else EMU_THR_DEFS="$EMU_THR_DEFS -DERTS_ENABLE_LOCK_COUNT" fi - AC_MSG_CHECKING(whether dirty schedulers should be enabled) - if test "x$enable_dirty_schedulers" != "xno"; then - EMU_THR_DEFS="$EMU_THR_DEFS -DERTS_DIRTY_SCHEDULERS" - AC_DEFINE(ERTS_DIRTY_SCHEDULERS, 1, [Define if the emulator supports dirty schedulers]) - AC_DEFINE(ERL_NIF_DIRTY_SCHEDULER_SUPPORT, 1, [Dirty scheduler support]) - AC_DEFINE(ERL_DRV_DIRTY_SCHEDULER_SUPPORT, 1, [Dirty scheduler support]) - AC_MSG_RESULT(yes) - else - AC_MSG_RESULT(no) - fi - case $host_os in linux*) AC_MSG_CHECKING([whether dlopen() needs to be called before first call to dlerror()]) @@ -3823,7 +3823,7 @@ if test "$enable_dtrace_test" = "yes" ; then [$RM -f $DTRACE_2STEP_TEST dtrace -G $DTRACE_CPP $DTRACE_BITS_FLAG -Iemulator/beam -o $DTRACE_2STEP_TEST -s emulator/beam/erlang_dtrace.d conftest.$OBJEXT 2>&AS_MESSAGE_LOG_FD if test -f $DTRACE_2STEP_TEST; then - rm $DTRACE_2STEP_TEST + $RM $DTRACE_2STEP_TEST DTRACE_ENABLED_2STEP=yes fi], []) @@ -4030,7 +4030,7 @@ ssl_done=yes # Default only one run # Remove all SKIP files from previous runs for a in ssl crypto ssh; do - /bin/rm -f $ERL_TOP/lib/$a/SKIP + $RM -f $ERL_TOP/lib/$a/SKIP done SSL_DYNAMIC_ONLY=$enable_dynamic_ssl @@ -4627,7 +4627,7 @@ need_java="jinterface ic/java_src" # Remove all SKIP files from previous runs for a in $need_java ; do - /bin/rm -f $ERL_TOP/lib/$a/SKIP + $RM -f $ERL_TOP/lib/$a/SKIP done if test "X$with_javac" = "Xno"; then @@ -4678,7 +4678,7 @@ dnl this deliberately does not believe that 'gcc' is a C++ compiler AC_CHECK_TOOLS(CXX, [$CCC c++ g++ CC cxx cc++ cl], false) # Remove SKIP file from previous run -/bin/rm -f $ERL_TOP/lib/orber/SKIP +$RM -f $ERL_TOP/lib/orber/SKIP if test "$CXX" = false; then echo "No C++ compiler found" > $ERL_TOP/lib/orber/SKIP diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index c499fc8081..1bbde7f1e0 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -627,6 +627,34 @@ <p>Sets the default binary virtual heap size of processes to the size <c><![CDATA[Size]]></c>.</p> </item> + <marker id="+hmax"/> + <tag><c><![CDATA[+hmax Size]]></c></tag> + <item> + <p>Sets the default maximum heap size of processes to the size + <c><![CDATA[Size]]></c>. If <c>+hmax</c> is not given, the default is <c>0</c> + which means that no maximum heap size is used. + For more information, see the documentation of + <seealso marker="erlang#process_flag_max_heap_size"> + <c>process_flag(max_heap_size, MaxHeapSize)</c></seealso>.</p> + </item> + <marker id="+hmaxel"/> + <tag><c><![CDATA[+hmaxel true|false]]></c></tag> + <item> + <p>Sets whether to send an error logger message for processes that reach + the maximum heap size or not. If <c>+hmaxel</c> is not given, the default is <c>true</c>. + For more information, see the documentation of + <seealso marker="erlang#process_flag_max_heap_size"> + <c>process_flag(max_heap_size, MaxHeapSize)</c></seealso>.</p> + </item> + <marker id="+hmaxk"/> + <tag><c><![CDATA[+hmaxk true|false]]></c></tag> + <item> + <p>Sets whether to kill processes that reach the maximum heap size or not. If + <c>+hmaxk</c> is not given, the default is <c>true</c>. For more information, + see the documentation of + <seealso marker="erlang#process_flag_max_heap_size"> + <c>process_flag(max_heap_size, MaxHeapSize)</c></seealso>.</p> + </item> <tag><c><![CDATA[+hpds Size]]></c></tag> <item> <p>Sets the initial process dictionary size of processes to the size diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml index 7546f7ef81..33a4fee182 100644 --- a/erts/doc/src/erl_nif.xml +++ b/erts/doc/src/erl_nif.xml @@ -138,29 +138,6 @@ ok automatically unloaded when the module code that it belongs to is purged by the code server.</p> - <p><marker id="lengthy_work"/> - As mentioned in the <seealso marker="#WARNING">warning</seealso> text at - the beginning of this document it is of vital importance that a native function - return relatively quickly. It is hard to give an exact maximum amount - of time that a native function is allowed to work, but as a rule of thumb - a well-behaving native function should return to its caller before a - millisecond has passed. This can be achieved using different approaches. - If you have full control over the code to execute in the native - function, the best approach is to divide the work into multiple chunks of - work and call the native function multiple times, either directly from Erlang code - or by having a native function schedule a future NIF call via the - <seealso marker="#enif_schedule_nif"> enif_schedule_nif</seealso> function. Function - <seealso marker="#enif_consume_timeslice">enif_consume_timeslice</seealso> can be - used to help with such work division. In some cases, however, this might not - be possible, e.g. when calling third-party libraries. Then you typically want - to dispatch the work to another thread, return - from the native function, and wait for the result. The thread can send - the result back to the calling thread using message passing. Information - about thread primitives can be found below. If you have built your system - with <em>the currently experimental</em> support for dirty schedulers, - you may want to try out this functionality by dispatching the work to a - <seealso marker="#dirty_nifs">dirty NIF</seealso>, - which does not have the same duration restriction as a normal NIF.</p> </description> <section> <title>FUNCTIONALITY</title> @@ -328,38 +305,161 @@ ok </list></p> </item> - <tag>Long-running NIFs</tag> - <item><p><marker id="dirty_nifs"/>Native functions - <seealso marker="#lengthy_work"> - must normally run quickly</seealso>, as explained earlier in this document. They - generally should execute for no more than a millisecond. But not all native functions - can execute so quickly; for example, functions that encrypt large blocks of data or - perform lengthy file system operations can often run for tens of seconds or more.</p> - <p>If the functionality of a long-running NIF can be split so that its work can be - achieved through a series of shorter NIF calls, the application can either make that series - of NIF calls from the Erlang level, or it can call a NIF that first performs a chunk of the - work, then invokes the <seealso marker="#enif_schedule_nif">enif_schedule_nif</seealso> - function to schedule another NIF call to perform the next chunk. The final call scheduled - in this manner can then return the overall result. Breaking up a long-running function in - this manner enables the VM to regain control between calls to the NIFs, thereby avoiding - degraded responsiveness, scheduler load balancing problems, and other strange behaviours.</p> - <p>A NIF that cannot be split and cannot execute in a millisecond or less is called a "dirty NIF" - because it performs work that the Erlang runtime cannot handle cleanly. - <em>Note that the dirty NIF functionality described here is experimental</em> and that you have to - enable support for dirty schedulers when building OTP in order to try the functionality out. - Applications that make use of such functions must indicate to the runtime that the functions are - dirty so they can be handled specially. To schedule a dirty NIF for execution, the - appropriate flags value can be set for the NIF in its <seealso marker="#ErlNifFunc">ErlNifFunc</seealso> - entry, or the application can call <seealso marker="#enif_schedule_nif">enif_schedule_nif</seealso>, - passing to it a pointer to the dirty NIF to be executed and indicating with the <c>flags</c> - argument whether it expects the operation to be CPU-bound or I/O-bound.</p> - <note><p>Dirty NIF support is available only when the emulator is configured with dirty - schedulers enabled. This feature is currently disabled by default. To determine whether - the dirty NIF API is available, native code can check to see if the C preprocessor macro - <c>ERL_NIF_DIRTY_SCHEDULER_SUPPORT</c> is defined. Also, if the Erlang runtime was built - without threading support, dirty schedulers are disabled. To check at runtime for the presence - of dirty scheduler threads, code can use the <seealso marker="#enif_system_info"><c> - enif_system_info()</c></seealso> API function.</p></note> + <tag><marker id="lengthy_work"/>Long-running NIFs</tag> + + <item><p> + As mentioned in the <seealso marker="#WARNING">warning</seealso> text at + the beginning of this document it is of <em>vital importance</em> that a + native function return relatively quickly. It is hard to give an exact + maximum amount of time that a native function is allowed to work, but as a + rule of thumb a well-behaving native function should return to its caller + before a millisecond has passed. This can be achieved using different + approaches. If you have full control over the code to execute in the + native function, the best approach is to divide the work into multiple + chunks of work and call the native function multiple times. In some + cases this might however not always be possible, e.g. when calling + third-party libraries.</p> + + <p>The + <seealso marker="#enif_consume_timeslice">enif_consume_timeslice()</seealso> + function can be used to inform the runtime system about the lenght of the + NIF call. It should typically always be used unless the NIF executes very + quickly.</p> + + <p>If the NIF call is too lenghty one needs to handle this in one of the + following ways in order to avoid degraded responsiveness, scheduler load + balancing problems, and other strange behaviours:</p> + + <taglist> + <tag>Yielding NIF</tag> + <item> + <p> + If the functionality of a long-running NIF can be split so that + its work can be achieved through a series of shorter NIF calls, + the application can either make that series of NIF calls from the + Erlang level, or it can call a NIF that first performs a chunk of + the work, then invokes the + <seealso marker="#enif_schedule_nif">enif_schedule_nif</seealso> + function to schedule another NIF call to perform the next chunk. + The final call scheduled in this manner can then return the + overall result. Breaking up a long-running function in + this manner enables the VM to regain control between calls to the + NIFs. + </p> + <p> + This approach is always preferred over the other alternatives + described below. This both from a performance perspective and + a system characteristics perspective. + </p> + </item> + + <tag>Threaded NIF</tag> + <item> + <p> + This is accomplished by dispatching the work to another thread + managed by the NIF library, return from the NIF, and wait for the + result. The thread can send the result back to the Erlang + process using <seealso marker="#enif_send">enif_send</seealso>. + Information about thread primitives can be found below. + </p> + </item> + + <tag><marker id="dirty_nifs"/>Dirty NIF</tag> + <item> + + <note> + <p> + <em>The dirty NIF functionality described here + is experimental</em>. Dirty NIF support is available only when + the emulator is configured with dirty schedulers enabled. This + feature is currently disabled by default. The Erlang runtime + without SMP support do not support dirty schedulers even when + the dirty scheduler support has been enabled. To check at + runtime for the presence of dirty scheduler threads, code can + use the + <seealso marker="#enif_system_info"><c>enif_system_info()</c></seealso> + API function. + </p> + </note> + + <p> + A NIF that cannot be split and cannot execute in a millisecond or + less is called a "dirty NIF" because it performs work that the + Erlang runtime cannot handle cleanly. Applications that make use + of such functions must indicate to the runtime that the functions + are dirty so they can be handled specially. To schedule a dirty + NIF for execution, the appropriate flags value can be set for the + NIF in its <seealso marker="#ErlNifFunc"><c>ErlNifFunc</c></seealso> + entry, or the application can call + <seealso marker="#enif_schedule_nif"><c>enif_schedule_nif</c></seealso>, + passing to it a pointer to the dirty NIF to be executed and + indicating with the <c>flags</c> argument whether it expects the + operation to be CPU-bound or I/O-bound. A dirty NIF executing + on a dirty scheduler does not have the same duration restriction + as a normal NIF. + </p> + + <p> + While a process is executing a dirty NIF some operations that + communicate with it may take a very long time to complete. + Suspend, or garbage collection of a process executing a dirty + NIF cannot be done until the dirty NIF has returned, so other + processes waiting for such operations to complete might have to + wait for a very long time. Blocking multi scheduling, i.e., + calling + <seealso marker="erlang#system_flag_multi_scheduling"><c>erlang:system_flag(multi_scheduling, + block)</c></seealso>, might also take a very long time to + complete. This since all ongoing dirty operations on all + dirty schedulers need to complete before the the block + operation can complete. + </p> + + <p> + A lot of operations communicating with a process executing a + dirty NIF can, however, complete while it is executing the + dirty NIF. For example, retreiving information about it via + <c>process_info()</c>, setting its group leader, + register/unregister its name, etc. + </p> + + <p> + Termination of a process executing a dirty NIF can only be + completed up to a certain point while it is executing the + dirty NIF. All Erlang resources such as registered names, + ETS tables, etc will be released. All links and monitors + will be triggered. The actual execution of the NIF will + however <em>not</em> be stopped. The NIF can safely contiue + execution, allocate heap memory, etc, but it is of course better + to stop executing as soon as possible. The NIF can check + whether current process is alive or not using + <seealso marker="#enif_is_current_process_alive"><c>enif_is_current_process_alive</c></seealso>. + Communication using + <seealso marker="#enif_send"><c>enif_send</c></seealso>, + and <seealso marker="#enif_port_command"><c>enif_port_command</c></seealso> + will also be dropped when the sending process is not alive. + Deallocation of certain internal resources such as process + heap, and process control block will be delayed until the + dirty NIF has completed. + </p> + + <p>Currently known issues that are planned to be fixed:</p> + <list> + <item> + <p> + Since purging of a module currently might need to garbage + collect a process in order to determine if it has + references to the module, a process executing a dirty + NIF might delay purging for a very long time. Delaying + a purge operatin implies delaying <em>all</em> code + loding operations which might cause severe problems for + the system as a whole. + </p> + </item> + </list> + + </item> + </taglist> + </item> </taglist> </section> @@ -508,6 +608,10 @@ typedef struct { CPU-bound, its <c>flags</c> field should be set to <c>ERL_NIF_DIRTY_JOB_CPU_BOUND</c>, or for I/O-bound jobs, <c>ERL_NIF_DIRTY_JOB_IO_BOUND</c>.</p> + <note><p>If one of the + <c>ERL_NIF_DIRTY_JOB_*_BOUND</c> flags is set, and the runtime + system has no support for dirty schedulers, the runtime system + will refuse to load the NIF library.</p></note> </item> <tag><marker id="ErlNifBinary"/>ErlNifBinary</tag> <item> @@ -963,6 +1067,13 @@ typedef enum { <fsummary>Determine if a term is a binary</fsummary> <desc><p>Return true if <c>term</c> is a binary</p></desc> </func> + <func><name><ret>int</ret><nametext>enif_is_current_process_alive(ErlNifEnv* env)</nametext></name> + <fsummary>Determine if currently executing process is alive or not.</fsummary> + <desc><p>Return true if currently executing process is currently alive; otherwise + false.</p> + <p>This function can only be used from a NIF-calling thread, and with an + environment corresponding to currently executing processes.</p></desc> + </func> <func><name><ret>int</ret><nametext>enif_is_empty_list(ErlNifEnv* env, ERL_NIF_TERM term)</nametext></name> <fsummary>Determine if a term is an empty list</fsummary> <desc><p>Return true if <c>term</c> is an empty list.</p></desc> @@ -993,15 +1104,10 @@ typedef enum { <func><name><ret>int</ret><nametext>enif_is_on_dirty_scheduler(ErlNifEnv* env)</nametext></name> <fsummary>Check to see if executing on a dirty scheduler thread</fsummary> <desc> - <p>Check to see if the current NIF is executing on a dirty scheduler thread. If the - emulator is built with threading support, calling <c>enif_is_on_dirty_scheduler</c> - from within a dirty NIF returns true. It returns false when the calling NIF is a regular - NIF running on a normal scheduler thread, or when the emulator is built without threading - support.</p> - <note><p>This function is available only when the emulator is configured with dirty - schedulers enabled. This feature is currently disabled by default. To determine whether - the dirty NIF API is available, native code can check to see if the C preprocessor macro - <c>ERL_NIF_DIRTY_SCHEDULER_SUPPORT</c> is defined.</p></note> + <p>Check to see if the current NIF is executing on a dirty scheduler thread. If + executing on a dirty scheduler thread true returned; otherwise false.</p> + <p>This function can only be used from a NIF-calling thread, and with an + environment corresponding to currently executing processes.</p> </desc> </func> <func><name><ret>int</ret><nametext>enif_is_pid(ErlNifEnv* env, ERL_NIF_TERM term)</nametext></name> @@ -1015,7 +1121,8 @@ typedef enum { <func><name><ret>int</ret><nametext>enif_is_port_alive(ErlNifEnv* env, ErlNifPort *port_id)</nametext></name> <fsummary>Determine if a local port is alive or not.</fsummary> <desc><p>Return true if <c>port_id</c> is currently alive.</p> - <p>This function can only be used in a from a NIF-calling thread.</p></desc> + <p>This function is only thread-safe when the emulator with SMP support is used. + It can only be used in a non-SMP emulator from a NIF-calling thread.</p></desc> </func> <func><name><ret>int</ret><nametext>enif_is_process_alive(ErlNifEnv* env, ErlNifPid *pid)</nametext></name> <fsummary>Determine if a local process is alive or not.</fsummary> @@ -1483,9 +1590,7 @@ enif_map_iterator_destroy(env, &iter); <fsummary>Send a port_command to to_port</fsummary> <desc> <p>This function works the same as <seealso marker="erlang#port_command-2">erlang:port_command/2</seealso> - except that it is always completely asynchronous. This call may return false - if it detects that the port is already dead, otherwise it will return true. - </p> + except that it is always completely asynchronous.</p> <taglist> <tag><c>env</c></tag> <item>The environment of the calling process. May not be NULL.</item> @@ -1504,7 +1609,10 @@ enif_map_iterator_destroy(env, &iter); calls to <c>enif_alloc_env</c>, <c>enif_make_copy</c>, <c>enif_port_command</c> and <c>enif_free_env</c> into one call. This optimization is only usefull when a majority of the terms are to be copied from <c>env</c> to the <c>msg_env</c>.</p> - <p>The call may return false if it detects that the command failed for some reason. Otherwise true is returned.</p> + <p>This function return true if the command was successfully sent; otherwise, + false. The call may return false if it detects that the command failed for some + reason. For example, <c>*to_port</c> does not refer to a local port, if currently + executing process, i.e. the sender, is not alive, or if <c>msg</c> is invalid.</p> <p>See also: <seealso marker="#enif_get_local_port"><c>enif_get_local_port</c></seealso>.</p> </desc> </func> @@ -1635,7 +1743,9 @@ enif_map_iterator_destroy(env, &iter); <tag><c>msg</c></tag> <item>The message term to send.</item> </taglist> - <p>Return true on success, or false if <c>*to_pid</c> does not refer to an alive local process.</p> + <p>Return true if the message was successfully sent; otherwise, false. The send + operation will fail if <c>*to_pid</c> does not refer to an alive local process, + or if currently executing process, i.e. the sender, is not alive.</p> <p>The message environment <c>msg_env</c> with all its terms (including <c>msg</c>) will be invalidated by a successful call to <c>enif_send</c>. The environment should either be freed with <seealso marker="#enif_free_env">enif_free_env</seealso> @@ -1653,6 +1763,15 @@ enif_map_iterator_destroy(env, &iter); <desc><p>Get the byte size of a resource object <c>obj</c> obtained by <seealso marker="#enif_alloc_resource">enif_alloc_resource</seealso>.</p></desc> </func> + + <func><name><ret>int</ret><nametext>enif_snprintf(char *str, size_t size, const char *format, ...)</nametext></name> + <fsummary>Format strings and Erlang terms</fsummary> + <desc> + <p>Similar to <c>snprintf</c> but this format string also accepts <c>"%T"</c> which formats Erlang terms. + </p> + </desc> + </func> + <func> <name><ret>void</ret><nametext>enif_system_info(ErlNifSysInfo *sys_info_ptr, size_t size)</nametext></name> <fsummary>Get information about the Erlang runtime system</fsummary> diff --git a/erts/doc/src/erl_tracer.xml b/erts/doc/src/erl_tracer.xml index 2075b962d8..d4c8bbad31 100644 --- a/erts/doc/src/erl_tracer.xml +++ b/erts/doc/src/erl_tracer.xml @@ -67,7 +67,7 @@ <desc> <p>The different trace tags that the tracer will be called with. Each trace tag is described in greater detail in - <seealso marker="#trace">Module:trace/6</seealso> + <seealso marker="#Module:trace/6">Module:trace/6</seealso> </p> </desc> </datatype> @@ -81,7 +81,7 @@ <datatype> <name name="trace_opts" /> <desc> - <p>The options for the tracee. + <p>The options for the tracee.</p> <taglist> <tag><c>timestamp</c></tag> <item>If not set to <c>undefined</c>, the tracer has been requested to @@ -90,10 +90,9 @@ <item>If not set to <c>true</c>, the tracer has been requested to include the output of a match specification that was run.</item> <tag><c>scheduler_id</c></tag> - <item>Set to a number of the scheduler id is to be included by the tracer. + <item>Set to a number if the scheduler id is to be included by the tracer. Otherwise it is set to <c>undefined</c>.</item> </taglist> - </p> </desc> </datatype> <datatype> @@ -114,30 +113,30 @@ <p>The following functions should be exported from a <c>erl_tracer</c> callback module.</p> <taglist> - <tag><seealso marker="#enabled"><c>Module:enabled/3</c></seealso></tag> + <tag><seealso marker="#Module:enabled/3"><c>Module:enabled/3</c></seealso></tag> <item>Mandatory</item> - <tag><seealso marker="#trace"><c>Module:trace/6</c></seealso></tag> + <tag><seealso marker="#Module:trace/6"><c>Module:trace/6</c></seealso></tag> <item>Mandatory</item> - <tag><seealso marker="#enabled_procs"><c>Module:enabled_procs/3</c></seealso></tag> + <tag><seealso marker="#Module:enabled_procs/3"><c>Module:enabled_procs/3</c></seealso></tag> <item>Optional</item> - <tag><seealso marker="#trace_procs"><c>Module:trace_procs/6</c></seealso></tag> + <tag><seealso marker="#Module:trace_procs/6"><c>Module:trace_procs/6</c></seealso></tag> <item>Optional</item> - <tag><seealso marker="#enabled_ports"><c>Module:enabled_ports/3</c></seealso></tag> + <tag><seealso marker="#Module:enabled_ports/3"><c>Module:enabled_ports/3</c></seealso></tag> <item>Optional</item> - <tag><seealso marker="#trace_ports"><c>Module:trace_ports/6</c></seealso></tag> + <tag><seealso marker="#Module:trace_ports/6"><c>Module:trace_ports/6</c></seealso></tag> <item>Optional</item> - <tag><seealso marker="#enabled_running_ports"><c>Module:enabled_running_ports/3</c></seealso></tag> + <tag><seealso marker="#Module:enabled_running_ports/3"><c>Module:enabled_running_ports/3</c></seealso></tag> <item>Optional</item> - <tag><seealso marker="#trace_running_ports"><c>Module:trace_running_ports/6</c></seealso></tag> + <tag><seealso marker="#Module:trace_running_ports/6"><c>Module:trace_running_ports/6</c></seealso></tag> <item>Optional</item> - <tag><seealso marker="#enabled_running_procs"><c>Module:enabled_running_procs/3</c></seealso></tag> + <tag><seealso marker="#Module:enabled_running_procs/3"><c>Module:enabled_running_procs/3</c></seealso></tag> <item>Optional</item> - <tag><seealso marker="#trace_running_procs"><c>Module:trace_running_procs/6</c></seealso></tag> + <tag><seealso marker="#Module:trace_running_procs/6"><c>Module:trace_running_procs/6</c></seealso></tag> <item>Optional</item> </taglist> </section> - <marker id="enabled"></marker> + <funcs> <func> <name>Module:enabled(TraceTag, TracerState, Tracee) -> Result</name> @@ -155,19 +154,17 @@ overhead associated with tracing. If <c>trace</c> is returned the necessary trace data will be created and the trace call-back of the tracer will be called. If <c>discard</c> is returned, this trace call - will be discarded and no call to trace will be done. If - <c>remove</c> is returned, the VM will attempt to remove this tracer - from the tracee, together with any trace flags set on the tracee. + will be discarded and no call to trace will be done. </p> <p><c>trace_status</c> is a special type of <c>TraceTag</c> which is used to check if the tracer should still be active. It is called in multiple scenarios, but most significantly it is used when tracing is started - using this tracer.</p> + using this tracer. If <c>remove</c> is returned when the <c>trace_status</c> + is checked, the tracer will be removed from the tracee.</p> <p>This function may be called multiple times per tracepoint, so it is important that it is both fast and side effect free.</p> </desc> </func> - <marker id="trace"></marker> <func> <name>Module:trace(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -182,7 +179,7 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled">Module:enabled/3</seealso> + the <seealso marker="#Module:enabled/3">Module:enabled/3</seealso> callback returned <c>trace</c>. In it any side effects needed by the tracer should be done. The tracepoint payload is located in the <c>FirstTraceTerm</c> and <c>SecondTraceTerm</c>. The content @@ -213,7 +210,6 @@ </desc> </func> - <marker id="enabled_procs"></marker> <func> <name>Module:enabled_procs(TraceTag, TracerState, Tracee) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -231,7 +227,6 @@ </desc> </func> - <marker id="trace_procs"></marker> <func> <name>Module:trace_procs(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -246,13 +241,12 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled_procs">Module:enabled_procs/3</seealso> + the <seealso marker="#Module:enabled_procs/3">Module:enabled_procs/3</seealso> callback returned <c>trace</c>.</p> <p>If <c>trace_procs/6</c> is not defined <c>trace/6</c> will be called instead.</p> </desc> </func> - <marker id="enabled_ports"></marker> <func> <name>Module:enabled_ports(TraceTag, TracerState, Tracee) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -270,7 +264,6 @@ </desc> </func> - <marker id="trace_ports"></marker> <func> <name>Module:trace_ports(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -285,13 +278,12 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled_ports">Module:enabled_ports/3</seealso> + the <seealso marker="#Module:enabled_ports/3">Module:enabled_ports/3</seealso> callback returned <c>trace</c>.</p> <p>If <c>trace_ports/6</c> is not defined <c>trace/6</c> will be called instead.</p> </desc> </func> - <marker id="enabled_running_procs"></marker> <func> <name>Module:enabled_running_procs(TraceTag, TracerState, Tracee) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -309,7 +301,6 @@ </desc> </func> - <marker id="trace_running_procs"></marker> <func> <name>Module:trace_running_procs(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -324,13 +315,12 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled_running_procs">Module:enabled_running_procs/3</seealso> + the <seealso marker="#Module:enabled_running_procs/3">Module:enabled_running_procs/3</seealso> callback returned <c>trace</c>.</p> <p>If <c>trace_running_procs/6</c> is not defined <c>trace/6</c> will be called instead.</p> </desc> </func> - <marker id="enabled_running_ports"></marker> <func> <name>Module:enabled_running_ports(TraceTag, TracerState, Tracee) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -348,7 +338,6 @@ </desc> </func> - <marker id="trace_running_ports"></marker> <func> <name>Module:trace_running_ports(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -363,13 +352,12 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled_running_ports">Module:enabled_running_ports/3</seealso> + the <seealso marker="#Module:enabled_running_ports/3">Module:enabled_running_ports/3</seealso> callback returned <c>trace</c>.</p> <p>If <c>trace_running_ports/6</c> is not defined <c>trace/6</c> will be called instead.</p> </desc> </func> - <marker id="enabled_call"></marker> <func> <name>Module:enabled_call(TraceTag, TracerState, Tracee) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -387,7 +375,6 @@ </desc> </func> - <marker id="trace_call"></marker> <func> <name>Module:trace_call(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -402,13 +389,12 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled_call">Module:enabled_call/3</seealso> + the <seealso marker="#Module:enabled_call/3">Module:enabled_call/3</seealso> callback returned <c>trace</c>.</p> <p>If <c>trace_call/6</c> is not defined <c>trace/6</c> will be called instead.</p> </desc> </func> - <marker id="enabled_send"></marker> <func> <name>Module:enabled_send(TraceTag, TracerState, Tracee) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -426,7 +412,6 @@ </desc> </func> - <marker id="trace_send"></marker> <func> <name>Module:trace_send(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -441,13 +426,12 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled_send">Module:enabled_send/3</seealso> + the <seealso marker="#Module:enabled_send/3">Module:enabled_send/3</seealso> callback returned <c>trace</c>.</p> <p>If <c>trace_send/6</c> is not defined <c>trace/6</c> will be called instead.</p> </desc> </func> - <marker id="enabled_receive"></marker> <func> <name>Module:enabled_receive(TraceTag, TracerState, Tracee) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -465,7 +449,6 @@ </desc> </func> - <marker id="trace_receive"></marker> <func> <name>Module:trace_receive(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -480,13 +463,12 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled_receive">Module:enabled_receive/3</seealso> + the <seealso marker="#Module:enabled_receive/3">Module:enabled_receive/3</seealso> callback returned <c>trace</c>.</p> <p>If <c>trace_receive/6</c> is not defined <c>trace/6</c> will be called instead.</p> </desc> </func> - <marker id="enabled_garbage_collection"></marker> <func> <name>Module:enabled_garbage_collection(TraceTag, TracerState, Tracee) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -504,7 +486,6 @@ </desc> </func> - <marker id="trace_garbage_collection"></marker> <func> <name>Module:trace_garbage_collection(TraceTag, TracerState, Tracee, FirstTraceTerm, SecondTraceTerm, Opts) -> Result</name> <fsummary>Check if a trace event should be generated.</fsummary> @@ -519,7 +500,7 @@ </type> <desc> <p>This callback will be called when a tracepoint is triggered and - the <seealso marker="#enabled_garbage_collection">Module:enabled_garbage_collection/3</seealso> + the <seealso marker="#Module:enabled_garbage_collection/3">Module:enabled_garbage_collection/3</seealso> callback returned <c>trace</c>.</p> <p>If <c>trace_garbage_collection/6</c> is not defined <c>trace/6</c> will be called instead.</p> </desc> @@ -617,7 +598,7 @@ static int upgrade(ErlNifEnv* env, void** priv_data, void** old_priv_data, } /* - * argv[0]: Trace Tag + * argv[0]: TraceTag * argv[1]: TracerState * argv[2]: Tracee */ @@ -626,8 +607,11 @@ static ERL_NIF_TERM enabled(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) ErlNifPid to_pid; if (enif_get_local_pid(env, argv[1], &to_pid)) if (!enif_is_process_alive(env, &to_pid)) - /* tracer is dead so we should remove this tracepoint */ - return enif_make_atom(env, "remove"); + if (enif_is_identical(enif_make_atom(env, "trace_status"), argv[0])) + /* tracer is dead so we should remove this tracepoint */ + return enif_make_atom(env, "remove"); + else + return enif_make_atom(env, "discard"); /* Only generate trace for when tracer != tracee */ if (enif_is_identical(argv[1], argv[2])) @@ -645,7 +629,7 @@ static ERL_NIF_TERM enabled(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) } /* - * argv[0]: Trace Tag, should only be 'send' + * argv[0]: TraceTag, should only be 'send' * argv[1]: TracerState, process to send {argv[2], argv[4]} to * argv[2]: Tracee * argv[3]: Message, ignored diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index bf30cc7928..e0723127f2 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -4322,6 +4322,7 @@ os_prompt% </pre> </desc> </func> + <marker id="process_flag_min_heap_size"/> <func> <name name="process_flag" arity="2" clause_i="3"/> <fsummary>Sets process flag <c>min_heap_size</c> for the calling process.</fsummary> @@ -4340,9 +4341,95 @@ os_prompt% </pre> <p>Returns the old value of the flag.</p> </desc> </func> - <marker id="process_flag_message_queue_data"/> + <marker id="process_flag_max_heap_size"/> <func> <name name="process_flag" arity="2" clause_i="5"/> + <type name="max_heap_size"/> + <fsummary>Sets process flag <c>max_heap_size</c> for the calling process.</fsummary> + <desc> + <p> + This flag sets the maximum heap size for the calling process. + If <c><anno>MaxHeapSize</anno></c> is an integer, the system default + values for <c>kill</c> and <c>error_logger</c> are used. + <taglist> + <tag><c>size</c></tag> + <item> + <p> + The maximum size in words of the process. If set to zero, the + heap size limit is disabled. Badarg will be thrown if the value is + smaller than + <seealso marker="#process_flag_min_heap_size"><c>min_heap_size</c></seealso>. + The size check is only done when a garbage collection is triggered. + </p> + <p> + <c>size</c> is the entire heap of the process when garbage collection + is triggered, this includes all generational heaps, the process stack, + any <seealso marker="#process_flag_message_queue_data"> + messages that are considered to be part of the heap</seealso> and any + extra memory that the garbage collector needs during collection. + </p> + <p> + <c>size</c> is the same as can be retrieved using + <seealso marker="#process_info_total_heap_size"> + <c>erlang:process_info(Pid, total_heap_size)</c></seealso>, + or by adding <c>heap_block_size</c>, <c>old_heap_block_size</c> + and <c>mbuf_size</c> from <seealso marker="#process_info_garbage_collection_info"> + <c>erlang:process_info(Pid, garbage_collection_info)</c></seealso>. + </p> + </item> + <tag><c>kill</c></tag> + <item> + <p> + When set to <c>true</c> the runtime system will send an + untrappable exit signal with reason <c>kill</c> to the process + if the maximum heap size is reached. The garbage collection + that triggered the <c>kill</c> will not be completed, instead the + process will exit as soon as is possible. When set to <c>false</c> + no exit signal will be sent to the process, instead it will + continue executing. + </p> + <p> + If <c>kill</c> is not defined in the map + the system default will be used. The default system default + is <c>true</c>. It can be changed by either the erl + <seealso marker="erl#+hmaxk">+hmaxk</seealso> option, + or <seealso marker="#system_flag_max_heap_size"><c> + erlang:system_flag(max_heap_size, MaxHeapSize)</c></seealso>. + </p> + </item> + <tag><c>error_logger</c></tag> + <item> + <p> + When set to <c>true</c> the runtime system will send a + message to the current <seealso marker="kernel:error_logger"><c>error_logger</c></seealso> + containing details about the process when the maximum + heap size is reached. One <c>error_logger</c> report will + be sent each time the limit is reached. + </p> + <p> + If <c>error_logger</c> is not defined in the map the system + default will be used. The default system default is <c>true</c>. + It can be changed by either the erl <seealso marker="erl#+hmaxel">+hmaxel</seealso> + option, or <seealso marker="#system_flag_max_heap_size"><c> + erlang:system_flag(max_heap_size, MaxHeapSize)</c></seealso>. + </p> + </item> + <p> + The heap size of a process is quite hard to predict, especially the + amount of memory that is used during the garbage collection. When + contemplating using this option, it is recommended to first run + it in production with <c>kill</c> set to <c>false</c> and inspect + the <c>error_logger</c> reports to see what the normal peak sizes + of the processes in the system is and then tune the value + accordingly. + </p> + </taglist> + </p> + </desc> + </func> + <marker id="process_flag_message_queue_data"/> + <func> + <name name="process_flag" arity="2" clause_i="6"/> <fsummary>Set process flag <c>message_queue_data</c> for the calling process</fsummary> <type name="message_queue_data"/> <desc> @@ -4392,7 +4479,7 @@ os_prompt% </pre> </desc> </func> <func> - <name name="process_flag" arity="2" clause_i="6"/> + <name name="process_flag" arity="2" clause_i="7"/> <fsummary>Sets process flag <c>priority</c> for the calling process.</fsummary> <type name="priority_level"/> <desc> @@ -4466,7 +4553,7 @@ os_prompt% </pre> </func> <func> - <name name="process_flag" arity="2" clause_i="7"/> + <name name="process_flag" arity="2" clause_i="8"/> <fsummary>Sets process flag <c>save_calls</c> for the calling process.</fsummary> <desc> <p><c><anno>N</anno></c> must be an integer in the interval 0..10000. @@ -4497,7 +4584,7 @@ os_prompt% </pre> </func> <func> - <name name="process_flag" arity="2" clause_i="8"/> + <name name="process_flag" arity="2" clause_i="9"/> <fsummary>Sets process flag <c>sensitive</c> for the calling process.</fsummary> <desc> <p>Sets or clears flag <c>sensitive</c> for the current process. @@ -4551,6 +4638,7 @@ os_prompt% </pre> <type name="process_info_result_item"/> <type name="priority_level"/> <type name="stack_item"/> + <type name="max_heap_size" /> <type name="message_queue_data" /> <desc> <p>Returns a list containing <c><anno>InfoTuple</anno></c>s with @@ -4604,6 +4692,7 @@ os_prompt% </pre> <type name="process_info_result_item"/> <type name="stack_item"/> <type name="priority_level"/> + <type name="max_heap_size" /> <type name="message_queue_data" /> <desc> <p>Returns information about the process identified by @@ -4696,6 +4785,7 @@ os_prompt% </pre> The content of <c><anno>GCInfo</anno></c> can be changed without prior notice.</p> </item> + <marker id="process_info_garbage_collection_info"/> <tag><c>{garbage_collection_info, <anno>GCInfo</anno>}</c></tag> <item> <p><c><anno>GCInfo</anno></c> is a list containing miscellaneous @@ -4875,6 +4965,7 @@ os_prompt% </pre> total suspend count on <c><anno>Suspendee</anno></c>, only the parts contributed by <c><anno>Pid</anno></c>.</p> </item> + <marker id="process_info_total_heap_size"/> <tag><c>{total_heap_size, <anno>Size</anno>}</c></tag> <item> <p><c><anno>Size</anno></c> is the total size, in words, of all heap @@ -5569,6 +5660,7 @@ true</pre> <name name="spawn_opt" arity="2"/> <fsummary>Creates a new process with a fun as entry point.</fsummary> <type name="priority_level"/> + <type name="max_heap_size" /> <type name="message_queue_data" /> <type name="spawn_opt_option" /> <desc> @@ -5586,6 +5678,7 @@ true</pre> <name name="spawn_opt" arity="3"/> <fsummary>Creates a new process with a fun as entry point on a given node.</fsummary> <type name="priority_level"/> + <type name="max_heap_size" /> <type name="message_queue_data" /> <type name="spawn_opt_option" /> <desc> @@ -5602,6 +5695,7 @@ true</pre> <name name="spawn_opt" arity="4"/> <fsummary>Creates a new process with a function as entry point.</fsummary> <type name="priority_level"/> + <type name="max_heap_size" /> <type name="message_queue_data" /> <type name="spawn_opt_option" /> <desc> @@ -5705,6 +5799,16 @@ true</pre> fine-tuning an application and to measure the execution time with various <c><anno>VSize</anno></c> values.</p> </item> + <tag><c>{max_heap_size, <anno>Size</anno>}</c></tag> + <item> + <p>Sets the <c>max_heap_size</c> process flag. The default + <c>max_heap_size</c> is determined by the + <seealso marker="erl#+hmax"><c>+hmax</c></seealso> <c>erl</c> + command line argument. For more information, see the + documentation of + <seealso marker="#process_flag_max_heap_size"><c>process_flag(max_heap_size, + <anno>Size</anno>)</c></seealso>.</p> + </item> <tag><c>{message_queue_data, <anno>MQD</anno>}</c></tag> <item> <p>Sets the state of the <c>message_queue_data</c> process @@ -5725,6 +5829,7 @@ true</pre> <name name="spawn_opt" arity="5"/> <fsummary>Creates a new process with a function as entry point on a given node.</fsummary> <type name="priority_level"/> + <type name="max_heap_size" /> <type name="message_queue_data" /> <type name="spawn_opt_option" /> <desc> @@ -6506,8 +6611,25 @@ ok </desc> </func> + <marker id="system_flag_max_heap_size"></marker> <func> <name name="system_flag" arity="2" clause_i="8"/> + <type name="max_heap_size"/> + <fsummary>Sets system flag <c>max_heap_size</c></fsummary> + <desc> + <p> + Sets the default maximum heap size settings for processes. + The size is given in words. The new <c>max_heap_size</c> + effects only processes spawned efter the change has been made. + <c>max_heap_size</c> can be set for individual processes using + <seealso marker="#spawn_opt/4">spawn_opt/N</seealso> or + <seealso marker="#process_flag_message_queue_data">process_flag/2</seealso>.</p> + <p>Returns the old value of the flag.</p> + </desc> + </func> + + <func> + <name name="system_flag" arity="2" clause_i="9"/> <fsummary>Sets system flag <c>multi_scheduling</c>.</fsummary> <desc> <p><marker id="system_flag_multi_scheduling"></marker> @@ -6557,7 +6679,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="9"/> + <name name="system_flag" arity="2" clause_i="10"/> <fsummary>Sets system flag <c>scheduler_bind_type</c>.</fsummary> <type name="scheduler_bind_type"/> <desc> @@ -6675,7 +6797,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="10"/> + <name name="system_flag" arity="2" clause_i="11"/> <fsummary>Sets system flag <c>scheduler_wall_time</c>.</fsummary> <desc><p><marker id="system_flag_scheduler_wall_time"></marker> Turns on or off scheduler wall time measurements.</p> @@ -6685,7 +6807,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="11"/> + <name name="system_flag" arity="2" clause_i="12"/> <fsummary>Sets system flag <c>schedulers_online</c>.</fsummary> <desc> <p><marker id="system_flag_schedulers_online"></marker> @@ -6710,7 +6832,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="12"/> + <name name="system_flag" arity="2" clause_i="13"/> <fsummary>Sets system flag <c>trace_control_word</c>.</fsummary> <desc> <p>Sets the value of the node trace control word to @@ -6724,7 +6846,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="12"/> + <name name="system_flag" arity="2" clause_i="14"/> <fsummary>Finalize the Time Offset</fsummary> <desc> <p><marker id="system_flag_time_offset"></marker> @@ -6991,6 +7113,81 @@ ok </func> <func> + <name name="system_info" arity="1" clause_i="27"/> + <name name="system_info" arity="1" clause_i="28"/> + <name name="system_info" arity="1" clause_i="36"/> + <name name="system_info" arity="1" clause_i="37"/> + <name name="system_info" arity="1" clause_i="38"/> + <name name="system_info" arity="1" clause_i="39"/> + <type name="message_queue_data"/> + <type name="max_heap_size"/> + <fsummary>Information about the default process heap settings.</fsummary> + <desc> + <taglist> + <tag><c>fullsweep_after</c></tag> + <item> + <p>Returns <c>{fullsweep_after, integer() >= 0}</c>, which is + the <c>fullsweep_after</c> garbage collection setting used + by default. For more information, see + <c>garbage_collection</c> described in the following.</p> + </item> + <tag><c>garbage_collection</c></tag> + <item> + <p>Returns a list describing the default garbage collection + settings. A process spawned on the local node by a + <c>spawn</c> or <c>spawn_link</c> uses these + garbage collection settings. The default settings can be + changed by using + <seealso marker="#system_flag/2">system_flag/2</seealso>. + <seealso marker="#spawn_opt/4">spawn_opt/4</seealso> + can spawn a process that does not use the default + settings.</p> + </item> + <tag><c>max_heap_size</c></tag> + <item> + <p>Returns <c>{max_heap_size, <anno>MaxHeapSize</anno>}</c>, + where <c><anno>MaxHeapSize</anno></c> is the current + system-wide max heap size settings for spawned processes. + This setting can be set using the <c>erl</c> command line + flags <seealso marker="erl#+hmax"><c>+hmax</c></seealso>, + <seealso marker="erl#+hmaxk"><c>+hmaxk</c></seealso> and + <seealso marker="erl#+hmaxel"><c>+hmaxel</c></seealso>. It can + also be changed at run-time using + <seealso marker="#system_flag_max_heap_size"> + <c>erlang:system_flag(max_heap_size, MaxHeapSize)</c></seealso>. + For more details about the <c>max_heap_size</c> process flag + see <seealso marker="#process_flag_max_heap_size"> + <c>process_flag(max_heap_size, MaxHeapSize)</c></seealso>. + </p> + </item> + <tag><c>min_heap_size</c></tag> + <item> + <p>Returns <c>{min_heap_size, <anno>MinHeapSize</anno>}</c>, + where <c><anno>MinHeapSize</anno></c> is the current + system-wide minimum heap size for spawned processes.</p> + </item> + <tag><marker id="system_info_message_queue_data"><c>message_queue_data</c></marker></tag> + <item> + <p>Returns the default value of the <c>message_queue_data</c> + process flag which is either <c>off_heap</c>, <c>on_heap</c>, or <c>mixed</c>. + This default is set by the <c>erl</c> command line argument + <seealso marker="erl#+hmqd"><c>+hmqd</c></seealso>. For more information on the + <c>message_queue_data</c> process flag, see documentation of + <seealso marker="#process_flag_message_queue_data"><c>process_flag(message_queue_data, + MQD)</c></seealso>.</p> + </item> + <tag><c>min_bin_vheap_size</c></tag> + <item> + <p>Returns <c>{min_bin_vheap_size, + <anno>MinBinVHeapSize</anno>}</c>, where + <c><anno>MinBinVHeapSize</anno></c> is the current system-wide + minimum binary virtual heap size for spawned processes.</p> + </item> + </taglist> + </desc> + </func> + + <func> <name name="system_info" arity="1" clause_i="6"/> <name name="system_info" arity="1" clause_i="7"/> <name name="system_info" arity="1" clause_i="8"/> @@ -7010,8 +7207,6 @@ ok <name name="system_info" arity="1" clause_i="24"/> <name name="system_info" arity="1" clause_i="25"/> <name name="system_info" arity="1" clause_i="26"/> - <name name="system_info" arity="1" clause_i="27"/> - <name name="system_info" arity="1" clause_i="28"/> <name name="system_info" arity="1" clause_i="29"/> <name name="system_info" arity="1" clause_i="30"/> <name name="system_info" arity="1" clause_i="31"/> @@ -7019,10 +7214,6 @@ ok <name name="system_info" arity="1" clause_i="33"/> <name name="system_info" arity="1" clause_i="34"/> <name name="system_info" arity="1" clause_i="35"/> - <name name="system_info" arity="1" clause_i="36"/> - <name name="system_info" arity="1" clause_i="37"/> - <name name="system_info" arity="1" clause_i="38"/> - <name name="system_info" arity="1" clause_i="39"/> <name name="system_info" arity="1" clause_i="40"/> <name name="system_info" arity="1" clause_i="41"/> <name name="system_info" arity="1" clause_i="42"/> @@ -7052,6 +7243,7 @@ ok <name name="system_info" arity="1" clause_i="66"/> <name name="system_info" arity="1" clause_i="67"/> <name name="system_info" arity="1" clause_i="68"/> + <name name="system_info" arity="1" clause_i="69"/> <fsummary>Information about the system.</fsummary> <desc> <p>Returns various information about the current system @@ -7288,25 +7480,6 @@ ok <c>ERL_MAX_ETS_TABLES</c> before starting the Erlang runtime system.</p> </item> - <tag><c>fullsweep_after</c></tag> - <item> - <p>Returns <c>{fullsweep_after, integer() >= 0}</c>, which is - the <c>fullsweep_after</c> garbage collection setting used - by default. For more information, see - <c>garbage_collection</c> described in the following.</p> - </item> - <tag><c>garbage_collection</c></tag> - <item> - <p>Returns a list describing the default garbage collection - settings. A process spawned on the local node by a - <c>spawn</c> or <c>spawn_link</c> uses these - garbage collection settings. The default settings can be - changed by using - <seealso marker="#system_flag/2">system_flag/2</seealso>. - <seealso marker="#spawn_opt/4">spawn_opt/4</seealso> - can spawn a process that does not use the default - settings.</p> - </item> <tag><c>heap_sizes</c></tag> <item> <p>Returns a list of integers representing valid heap sizes @@ -7381,29 +7554,6 @@ ok <item> <p>Returns a string containing the Erlang machine name.</p> </item> - <tag><c>min_heap_size</c></tag> - <item> - <p>Returns <c>{min_heap_size, <anno>MinHeapSize</anno>}</c>, - where <c><anno>MinHeapSize</anno></c> is the current - system-wide minimum heap size for spawned processes.</p> - </item> - <tag><marker id="system_info_message_queue_data"><c>message_queue_data</c></marker></tag> - <item> - <p>Returns the default value of the <c>message_queue_data</c> - process flag which is either <c>off_heap</c>, <c>on_heap</c>, or <c>mixed</c>. - This default is set by the <c>erl</c> command line argument - <seealso marker="erl#+hmqd"><c>+hmqd</c></seealso>. For more information on the - <c>message_queue_data</c> process flag, see documentation of - <seealso marker="#process_flag_message_queue_data"><c>process_flag(message_queue_data, - MQD)</c></seealso>.</p> - </item> - <tag><c>min_bin_vheap_size</c></tag> - <item> - <p>Returns <c>{min_bin_vheap_size, - <anno>MinBinVHeapSize</anno>}</c>, where - <c><anno>MinBinVHeapSize</anno></c> is the current system-wide - minimum binary virtual heap size for spawned processes.</p> - </item> <tag><c>modified_timing_level</c></tag> <item> <p>Returns the modified timing-level (an integer) if @@ -8028,12 +8178,13 @@ ok <c>GcPid</c> and <c>Info</c> are the same as for <c>long_gc</c> earlier, except that the tuple tagged with <c>timeout</c> is not present.</p> - <p>As of <c>ERTS</c> 5.6, the monitor message is sent - if the sum of the sizes of all memory blocks allocated - for all heap generations is equal to or higher than <c>Size</c>. - Previously the monitor message was sent if the memory block - allocated for the youngest generation was equal to or higher - than <c>Size</c>.</p> + <p>The monitor message is sent if the sum of the sizes of + all memory blocks allocated for all heap generations after + a garbage collection is equal to or higher than <c>Size</c>.</p> + <p>When a process is killed by <seealso marker="#process_flag_max_heap_size"> + <c>max_heap_size</c></seealso>, it is killed before the + garbage collection is complete and thus no large heap message + will be sent.</p> </item> <tag><c>busy_port</c></tag> <item> @@ -8560,7 +8711,9 @@ timestamp() -> <tag><c>garbage_collection</c></tag> <item> <p>Traces garbage collections of processes.</p> - <p>Message tags: <c><seealso marker="#trace_3_trace_messages_gc_minor_start">gc_minor_start</seealso></c> and <c><seealso marker="#trace_3_trace_messages_gc_minor_end">gc_minor_end</seealso></c>.</p> + <p>Message tags: <c><seealso marker="#trace_3_trace_messages_gc_minor_start">gc_minor_start</seealso></c>, + <c><seealso marker="#trace_3_trace_messages_gc_max_heap_size">gc_max_heap_size</seealso></c> and + <c><seealso marker="#trace_3_trace_messages_gc_minor_end">gc_minor_end</seealso></c>.</p> </item> <tag><c>timestamp</c></tag> <item> @@ -8929,6 +9082,19 @@ timestamp() -> <p>All sizes are in words.</p> </item> <tag> + <marker id="trace_3_trace_messages_gc_max_heap_size"></marker> + <c>{trace, Pid, gc_max_heap_size, Info}</c> + </tag> + <item> + <p> + Sent when the <seealso marker="#process_flag_max_heap_size"><c>max_heap_size</c></seealso> + is reached during garbage collection. <c>Info</c> contains the + same kind of list as in message <c>gc_start</c>, + but the sizes reflect the sizes that triggered max_heap_size to + be reached. + </p> + </item> + <tag> <marker id="trace_3_trace_messages_gc_minor_end"></marker> <c>{trace, Pid, gc_minor_end, Info}</c> </tag> diff --git a/erts/doc/src/erts_alloc.xml b/erts/doc/src/erts_alloc.xml index 70775b9f0f..9aef1c0b1f 100644 --- a/erts/doc/src/erts_alloc.xml +++ b/erts/doc/src/erts_alloc.xml @@ -583,7 +583,7 @@ </taglist> <p>The following flag is special for <c>exec_alloc</c>:</p> <taglist> - <tag><marker id="MIscs"/><c><![CDATA[+MXscs <size in MB>]]></c></tag> + <tag><marker id="MXscs"/><c><![CDATA[+MXscs <size in MB>]]></c></tag> <item> <c>exec_alloc</c> super carrier size (in MB). The amount of <em>virtual</em> address space reserved for native executable code diff --git a/erts/doc/src/match_spec.xml b/erts/doc/src/match_spec.xml index 3944f24f84..7be3d15de6 100644 --- a/erts/doc/src/match_spec.xml +++ b/erts/doc/src/match_spec.xml @@ -33,21 +33,15 @@ <file>match_spec.xml</file> </header> <p>A "match specification" (match_spec) is an Erlang term describing a - small "program" that will try to match something (either the - parameters to a function as used in the <c><![CDATA[erlang:trace_pattern/2]]></c> - BIF, or the objects in an ETS table.). + small "program" that will try to match something. It can be used + to either control tracing with + <seealso marker="erlang#trace_pattern/3">erlang:trace_pattern/3</seealso> + or to search for objects in an ETS table with for example + <seealso marker="stdlib:ets#select/2">ets:select/2</seealso>. The match_spec in many ways works like a small function in Erlang, but is interpreted/compiled by the Erlang runtime system to something much more efficient than calling an Erlang function. The match_spec is also very limited compared to the expressiveness of real Erlang functions.</p> - <p>Match specifications are given to the BIF <c><![CDATA[erlang:trace_pattern/2]]></c> to - execute matching of function arguments as well as to define some actions - to be taken when the match succeeds (the <c><![CDATA[MatchBody]]></c> part). Match - specifications can also be used in ETS, to specify objects to be - returned from an <c><![CDATA[ets:select/2]]></c> call (or other select - calls). The semantics and restrictions differ slightly when using - match specifications for tracing and in ETS, the differences are - defined in a separate paragraph below.</p> <p>The most notable difference between a match_spec and an Erlang fun is of course the syntax. Match specifications are Erlang terms, not Erlang code. A match_spec also has a somewhat strange concept of @@ -382,6 +376,51 @@ the pid() of the current process.</p> </section> + <marker id="match_target"></marker> + <section> + <title>Match target</title> + <p>Each execution of a match specification is done against + a match target term. The format and content of the target term + depends on the context in which the match is done. The match + target for ETS is always a full table tuple. The match target + for call trace is always a list of all function arguments. The + match target for event trace depends on the event type, see + table below.</p> + <table> + <row> + <cell align="left" valign="middle">Context</cell> + <cell align="left" valign="middle">Type</cell> + <cell align="left" valign="middle">Match target</cell> + <cell align="left" valign="middle">Description</cell> + </row> + <row> + <cell align="left" valign="middle">ETS</cell> + <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle">{Key, Value1, Value2, ...}</cell> + <cell align="left" valign="middle">A table object</cell> + </row> + <row> + <cell align="left" valign="middle">Trace</cell> + <cell align="left" valign="middle">call</cell> + <cell align="left" valign="middle">[Arg1, Arg2, ...]</cell> + <cell align="left" valign="middle">Function arguments</cell> + </row> + <row> + <cell align="left" valign="middle">Trace</cell> + <cell align="left" valign="middle">send</cell> + <cell align="left" valign="middle">[Receiver, Message]</cell> + <cell align="left" valign="middle">Receiving process/port and message term</cell> + </row> + <row> + <cell align="left" valign="middle">Trace</cell> + <cell align="left" valign="middle">'receive'</cell> + <cell align="left" valign="middle">[Node, Sender, Message]</cell> + <cell align="left" valign="middle">Sending node, process/port and message term</cell> + </row> + <tcaption>Match target depending on context</tcaption> + </table> + </section> + <section> <title>Variables and literals</title> <p>Variables take the form <c><![CDATA['$<number>']]></c> where @@ -396,10 +435,8 @@ <c><![CDATA[MatchCondition]]></c> parts, only variables bound previously may be used. As a special case, in the <c><![CDATA[MatchCondition/MatchBody]]></c> parts, the variable <c><![CDATA['$_']]></c> - expands to the whole expression which matched the - <c><![CDATA[MatchHead]]></c> (i.e., the whole parameter list to the possibly - traced function or the whole matching object in the ets table) - and the variable <c><![CDATA['$$']]></c> expands to a list + expands to the whole <seealso marker="#match_target">match target</seealso> + term and the variable <c><![CDATA['$$']]></c> expands to a list of the values of all bound variables in order (i.e. <c><![CDATA[['$1','$2', ...]]]></c>). </p> @@ -480,8 +517,8 @@ <p>For each tuple in the <c><![CDATA[MatchExpression]]></c> list and while no match has succeeded:</p> <list type="bulleted"> - <item>Match the <c><![CDATA[MatchHead]]></c> part against the arguments to the - function, + <item>Match the <c><![CDATA[MatchHead]]></c> part against the + match target term, binding the <c><![CDATA['$<number>']]></c> variables (much like in <c><![CDATA[ets:match/2]]></c>). If the <c><![CDATA[MatchHead]]></c> cannot match the arguments, the match fails. @@ -522,13 +559,10 @@ term. The <c><![CDATA[ActionTerm]]></c>'s are executed as in an imperative language, i.e. for their side effects. Functions with side effects are also allowed when tracing.</p> - <p>In ETS the match head is a <c><![CDATA[tuple()]]></c> (or a single match - variable) while it is a list (or a single match variable) when - tracing.</p> </section> <section> - <title>ETS Examples</title> + <title>Tracing Examples</title> <p>Match an argument list of three where the first and third arguments are equal:</p> <code type="none"><![CDATA[ @@ -585,42 +619,6 @@ parameter list with a single variable is a special case. In all other cases the <c><![CDATA[MatchHead]]></c> has to be a <em>proper</em> list. </p> - <p>Match all objects in an ets table where the first element is - the atom 'strider' and the tuple arity is 3 and return the whole - object.</p> - <code type="none"><![CDATA[ -[{{strider,'_','_'}, - [], - ['$_']}] - ]]></code> - <p>Match all objects in an ets table with arity > 1 and the first - element is 'gandalf', return element 2.</p> - <code type="none"><![CDATA[ -[{'$1', - [{'==', gandalf, {element, 1, '$1'}},{'>=',{size, '$1'},2}], - [{element,2,'$1'}]}] - ]]></code> - <p>In the above example, if the first element had been the key, - it's much more efficient to match that key in the <c><![CDATA[MatchHead]]></c> - part than in the <c><![CDATA[MatchConditions]]></c> part. The search space of - the tables is restricted with regards to the <c><![CDATA[MatchHead]]></c> so - that only objects with the matching key are searched. - </p> - <p>Match tuples of 3 elements where the second element is either - 'merry' or 'pippin', return the whole objects.</p> - <code type="none"><![CDATA[ -[{{'_',merry,'_'}, - [], - ['$_']}, - {{'_',pippin,'_'}, - [], - ['$_']}] - ]]></code> - <p>The function <c><![CDATA[ets:test_ms/2]]></c> can be useful for testing - complicated ets matches.</p> - </section> - <section> - <title>Tracing Examples</title> <p>Only generate trace message if trace control word is set to 1:</p> <code type="none"><![CDATA[ [{'_', @@ -658,5 +656,42 @@ {'_',[],[]}] ]]></code> </section> + + <section> + <title>ETS Examples</title> + <p>Match all objects in an ets table where the first element is + the atom 'strider' and the tuple arity is 3 and return the whole + object.</p> + <code type="none"><![CDATA[ +[{{strider,'_','_'}, + [], + ['$_']}] + ]]></code> + <p>Match all objects in an ets table with arity > 1 and the first + element is 'gandalf', return element 2.</p> + <code type="none"><![CDATA[ +[{'$1', + [{'==', gandalf, {element, 1, '$1'}},{'>=',{size, '$1'},2}], + [{element,2,'$1'}]}] + ]]></code> + <p>In the above example, if the first element had been the key, + it's much more efficient to match that key in the <c><![CDATA[MatchHead]]></c> + part than in the <c><![CDATA[MatchConditions]]></c> part. The search space of + the tables is restricted with regards to the <c><![CDATA[MatchHead]]></c> so + that only objects with the matching key are searched. + </p> + <p>Match tuples of 3 elements where the second element is either + 'merry' or 'pippin', return the whole objects.</p> + <code type="none"><![CDATA[ +[{{'_',merry,'_'}, + [], + ['$_']}, + {{'_',pippin,'_'}, + [], + ['$_']}] + ]]></code> + <p>The function <c><![CDATA[ets:test_ms/2]]></c> can be useful for testing + complicated ets matches.</p> + </section> </chapter> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index fb486c917f..2212aed5e0 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -50,6 +50,8 @@ LDFLAGS=@LDFLAGS@ ARFLAGS=rc OMIT_OMIT_FP=no +DIRTY_SCHEDULER_SUPPORT=@DIRTY_SCHEDULER_SUPPORT@ + ifeq ($(TYPE),debug) PURIFY = TYPEMARKER = .debug @@ -174,6 +176,10 @@ FLAVOR_MARKER=.smp FLAVOR_FLAGS=-DERTS_SMP ENABLE_ALLOC_TYPE_VARS += smp nofrag M4FLAGS += -DERTS_SMP=1 +ifeq ($(DIRTY_SCHEDULER_SUPPORT),yes) +THR_DEFS += -DERTS_DIRTY_SCHEDULERS +endif + else # If flavor isn't one of the above, it *is* plain flavor... @@ -182,7 +188,6 @@ FLAVOR_MARKER= FLAVOR_FLAGS= ENABLE_ALLOC_TYPE_VARS += nofrag M4FLAGS += - endif TF_MARKER=$(TYPEMARKER)$(FLAVOR_MARKER) diff --git a/erts/emulator/beam/atom.names b/erts/emulator/beam/atom.names index 3022c0a99a..8f65e71531 100644 --- a/erts/emulator/beam/atom.names +++ b/erts/emulator/beam/atom.names @@ -275,6 +275,7 @@ atom garbage_collection_info atom gc_end atom gc_major_end atom gc_major_start +atom gc_max_heap_size atom gc_minor_end atom gc_minor_start atom gc_start @@ -366,6 +367,7 @@ atom match_spec atom match_spec_result atom max atom maximum +atom max_heap_size atom max_tables max_processes atom mbuf_size atom md5 diff --git a/erts/emulator/beam/beam_bp.c b/erts/emulator/beam/beam_bp.c index 1e30e8d8d1..8489897d3a 100644 --- a/erts/emulator/beam/beam_bp.c +++ b/erts/emulator/beam/beam_bp.c @@ -82,7 +82,7 @@ erts_smp_atomic32_t erts_staging_bp_index; static ERTS_INLINE ErtsMonotonicTime get_mtime(Process *c_p) { - return erts_get_monotonic_time(ERTS_PROC_GET_SCHDATA(c_p)); + return erts_get_monotonic_time(erts_proc_sched_data(c_p)); } /* ************************************************************************* @@ -655,8 +655,7 @@ erts_generic_breakpoint(Process* c_p, BeamInstr* I, Eterm* reg) erts_smp_atomic_inc_nob(&bp->count->acount); } - if (bp_flags & ERTS_BPF_TIME_TRACE_ACTIVE - && ERTS_TRACER_PROC_IS_ENABLED(c_p)) { + if (bp_flags & ERTS_BPF_TIME_TRACE_ACTIVE) { Eterm w; erts_trace_time_call(c_p, I, bp->time); w = (BeamInstr) *c_p->cp; @@ -753,8 +752,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) } } if (bp_flags & ERTS_BPF_TIME_TRACE_ACTIVE && - IS_TRACED_FL(p, F_TRACE_CALLS) && - ERTS_TRACER_PROC_IS_ENABLED(p)) { + IS_TRACED_FL(p, F_TRACE_CALLS)) { BeamInstr *pc = (BeamInstr *)ep->code+3; erts_trace_time_call(p, pc, bp->time); } @@ -976,7 +974,8 @@ erts_trace_time_call(Process* c_p, BeamInstr* I, BpDataTime* bdt) BpDataTime *pbdt = NULL; ASSERT(c_p); - ASSERT(erts_smp_atomic32_read_acqb(&c_p->state) & ERTS_PSFLG_RUNNING); + ASSERT(erts_smp_atomic32_read_acqb(&c_p->state) & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING)); /* get previous timestamp and breakpoint * from the process psd */ @@ -1053,7 +1052,8 @@ erts_trace_time_return(Process *p, BeamInstr *pc) BpDataTime *pbdt = NULL; ASSERT(p); - ASSERT(erts_smp_atomic32_read_acqb(&p->state) & ERTS_PSFLG_RUNNING); + ASSERT(erts_smp_atomic32_read_acqb(&p->state) & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING)); /* get previous timestamp and breakpoint * from the process psd */ diff --git a/erts/emulator/beam/beam_bp.h b/erts/emulator/beam/beam_bp.h index 9c2fc007a2..541af77211 100644 --- a/erts/emulator/beam/beam_bp.h +++ b/erts/emulator/beam/beam_bp.h @@ -80,7 +80,7 @@ typedef struct generic_bp { #define ERTS_BP_CALL_TIME_SCHEDULE_EXITING (2) #ifdef ERTS_SMP -#define bp_sched2ix_proc(p) ((p)->scheduler_data->no - 1) +#define bp_sched2ix_proc(p) (erts_proc_sched_data(p)->no - 1) #else #define bp_sched2ix_proc(p) (0) #endif diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index 99eb26bec5..f8f2e29c95 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -64,18 +64,21 @@ # ifdef ERTS_SMP # define PROCESS_MAIN_CHK_LOCKS(P) \ do { \ - if ((P)) { \ + if ((P)) \ erts_proc_lc_chk_only_proc_main((P)); \ - } \ - else \ - erts_lc_check_exact(NULL, 0); \ - ERTS_SMP_LC_ASSERT(!erts_thr_progress_is_blocking()); \ + ERTS_SMP_LC_ASSERT(!erts_thr_progress_is_blocking()); \ +} while (0) +# define ERTS_SMP_REQ_PROC_MAIN_LOCK(P) \ +do { \ + if ((P)) \ + erts_proc_lc_require_lock((P), ERTS_PROC_LOCK_MAIN, \ + __FILE__, __LINE__); \ +} while (0) +# define ERTS_SMP_UNREQ_PROC_MAIN_LOCK(P) \ +do { \ + if ((P)) \ + erts_proc_lc_unrequire_lock((P), ERTS_PROC_LOCK_MAIN); \ } while (0) -# define ERTS_SMP_REQ_PROC_MAIN_LOCK(P) \ - if ((P)) erts_proc_lc_require_lock((P), ERTS_PROC_LOCK_MAIN,\ - __FILE__, __LINE__) -# define ERTS_SMP_UNREQ_PROC_MAIN_LOCK(P) \ - if ((P)) erts_proc_lc_unrequire_lock((P), ERTS_PROC_LOCK_MAIN) # else # define ERTS_SMP_REQ_PROC_MAIN_LOCK(P) # define ERTS_SMP_UNREQ_PROC_MAIN_LOCK(P) @@ -1202,12 +1205,12 @@ init_emulator(void) do { \ if (ERTS_PROC_GET_SAVED_CALLS_BUF((P))) { \ ASSERT(FC <= 0); \ - ASSERT(ERTS_PROC_GET_SCHDATA(c_p)->virtual_reds \ + ASSERT(erts_proc_sched_data(c_p)->virtual_reds \ <= 0 - (FC)); \ } \ else { \ ASSERT(FC <= CONTEXT_REDS); \ - ASSERT(ERTS_PROC_GET_SCHDATA(c_p)->virtual_reds \ + ASSERT(erts_proc_sched_data(c_p)->virtual_reds \ <= CONTEXT_REDS - (FC)); \ } \ } while (0) @@ -1321,8 +1324,8 @@ void process_main(void) if (start_time != 0) { Sint64 diff = erts_timestamp_millis() - start_time; if (diff > 0 && (Uint) diff > erts_system_monitor_long_schedule -#ifdef ERTS_DIRTY_SCHEDULERS - && !ERTS_SCHEDULER_IS_DIRTY(c_p->scheduler_data) +#if defined(ERTS_SMP) && defined(ERTS_DIRTY_SCHEDULERS) + && !ERTS_SCHEDULER_IS_DIRTY(erts_proc_sched_data(c_p)) #endif ) { BeamInstr *inptr = find_function_from_pc(start_time_i); @@ -1351,8 +1354,8 @@ void process_main(void) start_time_i = c_p->i; } - reg = ERTS_PROC_GET_SCHDATA(c_p)->x_reg_array; - freg = ERTS_PROC_GET_SCHDATA(c_p)->f_reg_array; + reg = erts_proc_sched_data(c_p)->x_reg_array; + freg = erts_proc_sched_data(c_p)->f_reg_array; ERL_BITS_RELOAD_STATEP(c_p); { int reds; @@ -1704,6 +1707,14 @@ void process_main(void) BeamInstr *next; Eterm result; + if (!(FCALLS > 0 || FCALLS > neg_o_reds)) { + /* If we have run out of reductions, we do a context + switch before calling the bif */ + c_p->arity = 2; + c_p->current = NULL; + goto context_switch3; + } + PRE_BIF_SWAPOUT(c_p); c_p->fcalls = FCALLS - 1; result = erl_send(c_p, r(0), x(1)); @@ -2810,6 +2821,15 @@ do { \ BeamInstr *next; ErlHeapFragment *live_hf_end; + + if (!((FCALLS - 1) > 0 || (FCALLS-1) > neg_o_reds)) { + /* If we have run out of reductions, we do a context + switch before calling the bif */ + c_p->arity = ((Export *)Arg(0))->code[2]; + c_p->current = ((Export *)Arg(0))->code; + goto context_switch3; + } + if (ERTS_MSACC_IS_ENABLED_CACHED_X()) { if (GET_BIF_MODULE(Arg(0)) == am_ets) { ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_ETS); @@ -3338,10 +3358,19 @@ do { \ context_switch2: /* Entry for fun calls. */ c_p->current = I-3; /* Pointer to Mod, Func, Arity */ + context_switch3: + { Eterm* argp; int i; + if (erts_smp_atomic32_read_nob(&c_p->state) & ERTS_PSFLG_EXITING) { + c_p->i = beam_exit; + c_p->arity = 0; + c_p->current = NULL; + goto do_schedule; + } + /* * Make sure that there is enough room for the argument registers to be saved. */ @@ -3509,6 +3538,12 @@ do { \ BifFunction vbf; ErlHeapFragment *live_hf_end; + if (!((FCALLS - 1) > 0 || (FCALLS - 1) > neg_o_reds)) { + /* If we have run out of reductions, we do a context + switch before calling the nif */ + goto context_switch; + } + ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_NIF); DTRACE_NIF_ENTRY(c_p, (Eterm)I[-3], (Eterm)I[-2], (Uint)I[-1]); @@ -3524,18 +3559,27 @@ do { \ typedef Eterm NifF(struct enif_environment_t*, int argc, Eterm argv[]); NifF* fp = vbf = (NifF*) I[1]; struct enif_environment_t env; +#ifdef ERTS_DIRTY_SCHEDULERS + if (!c_p->scheduler_data) + live_hf_end = ERTS_INVALID_HFRAG_PTR; /* On dirty scheduler */ + else +#endif + live_hf_end = c_p->mbuf; erts_pre_nif(&env, c_p, (struct erl_module_nif*)I[2], NULL); - live_hf_end = c_p->mbuf; nif_bif_result = (*fp)(&env, bif_nif_arity, reg); if (env.exception_thrown) nif_bif_result = THE_NON_VALUE; erts_post_nif(&env); - } - ASSERT(!ERTS_PROC_IS_EXITING(c_p) || is_non_value(nif_bif_result)); - PROCESS_MAIN_CHK_LOCKS(c_p); - ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); - ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_EMULATOR); + PROCESS_MAIN_CHK_LOCKS(c_p); + ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); + ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_EMULATOR); + if (env.exiting) { + ERTS_SMP_REQ_PROC_MAIN_LOCK(c_p); + goto do_schedule; + } + ASSERT(!ERTS_PROC_IS_EXITING(c_p)); + } DTRACE_NIF_RETURN(c_p, (Eterm)I[-3], (Eterm)I[-2], (Uint)I[-1]); goto apply_bif_or_nif_epilogue; @@ -3551,6 +3595,13 @@ do { \ * code[3]: &&apply_bif * code[4]: Function pointer to BIF function */ + + if (!((FCALLS - 1) > 0 || (FCALLS - 1) > neg_o_reds)) { + /* If we have run out of reductions, we do a context + switch before calling the bif */ + goto context_switch; + } + if (ERTS_MSACC_IS_ENABLED_CACHED_X()) { if ((Eterm)I[-3] == am_ets) { ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_ETS); @@ -4887,8 +4938,8 @@ do { \ #ifdef DEBUG pid = c_p->common.id; /* may have switched process... */ #endif - reg = ERTS_PROC_GET_SCHDATA(c_p)->x_reg_array; - freg = ERTS_PROC_GET_SCHDATA(c_p)->f_reg_array; + reg = erts_proc_sched_data(c_p)->x_reg_array; + freg = erts_proc_sched_data(c_p)->f_reg_array; ERL_BITS_RELOAD_STATEP(c_p); /* XXX: this abuse of def_arg_reg[] is horrid! */ neg_o_reds = -c_p->def_arg_reg[4]; diff --git a/erts/emulator/beam/beam_load.c b/erts/emulator/beam/beam_load.c index 95489d9a63..0c2743beb2 100644 --- a/erts/emulator/beam/beam_load.c +++ b/erts/emulator/beam/beam_load.c @@ -4424,6 +4424,7 @@ gen_get_map_elements(LoaderState* stp, GenOpArg Fail, GenOpArg Src, int good_hash; #endif + ERTS_UNDEF(hx, 0); ASSERT(Size.type == TAG_u); NEW_GENOP(stp, op); diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index eed0702688..2a3bd4afe5 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -44,6 +44,7 @@ #include "erl_ptab.h" #include "erl_bits.h" #include "erl_bif_unique.h" +#include "erl_map.h" #include "erl_msacc.h" Export *erts_await_result; @@ -880,6 +881,8 @@ BIF_RETTYPE spawn_opt_1(BIF_ALIST_1) so.flags = erts_default_spo_flags|SPO_USE_ARGS; so.min_heap_size = H_MIN_SIZE; so.min_vheap_size = BIN_VH_MIN_SIZE; + so.max_heap_size = H_MAX_SIZE; + so.max_heap_flags = H_MAX_FLAGS; so.priority = PRIORITY_NORMAL; so.max_gen_gcs = (Uint16) erts_smp_atomic32_read_nob(&erts_max_gen_gcs); so.scheduler = 0; @@ -937,6 +940,9 @@ BIF_RETTYPE spawn_opt_1(BIF_ALIST_1) } else { so.min_heap_size = erts_next_heap_size(min_heap_size, 0); } + } else if (arg == am_max_heap_size) { + if (!erts_max_heap_size(val, &so.max_heap_size, &so.max_heap_flags)) + goto error; } else if (arg == am_min_bin_vheap_size && is_small(val)) { Sint min_vheap_size = signed_val(val); if (min_vheap_size < 0) { @@ -970,6 +976,10 @@ BIF_RETTYPE spawn_opt_1(BIF_ALIST_1) goto error; } + if (so.max_heap_size != 0 && so.max_heap_size < so.min_heap_size) { + goto error; + } + /* * Spawn the process. */ @@ -1686,7 +1696,7 @@ BIF_RETTYPE process_flag_2(BIF_ALIST_2) ERTS_PSFLG_BOUND); } - curr = ERTS_GET_SCHEDULER_DATA_FROM_PROC(BIF_P)->run_queue; + curr = erts_proc_sched_data(BIF_P)->run_queue; old = (ERTS_PSFLG_BOUND & state) ? curr : NULL; ASSERT(!old || old == curr); @@ -1731,6 +1741,23 @@ BIF_RETTYPE process_flag_2(BIF_ALIST_2) } BIF_RET(old_value); } + else if (BIF_ARG_1 == am_max_heap_size) { + Eterm *hp; + Uint sz = 0, max_heap_size, max_heap_flags; + + if (!erts_max_heap_size(BIF_ARG_2, &max_heap_size, &max_heap_flags)) + goto error; + + if ((max_heap_size < MIN_HEAP_SIZE(BIF_P) && max_heap_size != 0)) + goto error; + + erts_max_heap_size_map(MAX_HEAP_SIZE_GET(BIF_P), MAX_HEAP_SIZE_FLAGS_GET(BIF_P), NULL, &sz); + hp = HAlloc(BIF_P, sz); + old_value = erts_max_heap_size_map(MAX_HEAP_SIZE_GET(BIF_P), MAX_HEAP_SIZE_FLAGS_GET(BIF_P), &hp, NULL); + MAX_HEAP_SIZE_SET(BIF_P, max_heap_size); + MAX_HEAP_SIZE_FLAGS_SET(BIF_P, max_heap_flags); + BIF_RET(old_value); + } else if (BIF_ARG_1 == am_message_queue_data) { old_value = erts_change_message_queue_management(BIF_P, BIF_ARG_2); if (is_non_value(old_value)) @@ -4198,8 +4225,28 @@ BIF_RETTYPE group_leader_2(BIF_ALIST_2) else { locks &= ~ERTS_PROC_LOCK_STATUS; erts_smp_proc_unlock(new_member, ERTS_PROC_LOCK_STATUS); - new_member->group_leader = STORE_NC_IN_PROC(new_member, - BIF_ARG_1); + if (erts_smp_atomic32_read_nob(&new_member->state) + & !(ERTS_PSFLG_DIRTY_RUNNING|ERTS_PSFLG_DIRTY_RUNNING_SYS)) { + new_member->group_leader = STORE_NC_IN_PROC(new_member, + BIF_ARG_1); + } + else { + ErlHeapFragment *bp; + Eterm *hp; + /* + * Other process executing on a dirty scheduler, + * so we are not allowed to write to its heap. + * Store in heap fragment. + */ + + bp = new_message_buffer(NC_HEAP_SIZE(BIF_ARG_1)); + hp = bp->mem; + new_member->group_leader = STORE_NC(&hp, + &new_member->off_heap, + BIF_ARG_1); + bp->next = new_member->mbuf; + new_member->mbuf = bp; + } } } @@ -4350,6 +4397,31 @@ BIF_RETTYPE system_flag_2(BIF_ALIST_2) erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); BIF_RET(make_small(oval)); + } else if (BIF_ARG_1 == am_max_heap_size) { + + Eterm *hp, old_value; + Uint sz = 0, max_heap_size, max_heap_flags; + + if (!erts_max_heap_size(BIF_ARG_2, &max_heap_size, &max_heap_flags)) + goto error; + + if (max_heap_size < H_MIN_SIZE && max_heap_size != 0) + goto error; + + erts_max_heap_size_map(H_MAX_SIZE, H_MAX_FLAGS, NULL, &sz); + hp = HAlloc(BIF_P, sz); + old_value = erts_max_heap_size_map(H_MAX_SIZE, H_MAX_FLAGS, &hp, NULL); + + erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); + erts_smp_thr_progress_block(); + + H_MAX_SIZE = max_heap_size; + H_MAX_FLAGS = max_heap_flags; + + erts_smp_thr_progress_unblock(); + erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); + + BIF_RET(old_value); } else if (BIF_ARG_1 == am_display_items) { int oval = display_items; if (!is_small(BIF_ARG_2) || (n = signed_val(BIF_ARG_2)) < 0) { diff --git a/erts/emulator/beam/bif.h b/erts/emulator/beam/bif.h index 5d751dd67d..2203182a0d 100644 --- a/erts/emulator/beam/bif.h +++ b/erts/emulator/beam/bif.h @@ -59,12 +59,12 @@ extern Export *erts_convert_time_unit_trap; do { \ if (!ERTS_PROC_GET_SAVED_CALLS_BUF((p))) { \ if ((fcalls) > 0) \ - ERTS_PROC_GET_SCHDATA((p))->virtual_reds += (fcalls); \ + erts_proc_sched_data((p))->virtual_reds += (fcalls); \ (fcalls) = 0; \ } \ else { \ if ((fcalls) > -CONTEXT_REDS) \ - ERTS_PROC_GET_SCHDATA((p))->virtual_reds \ + erts_proc_sched_data((p))->virtual_reds \ += ((fcalls) - (-CONTEXT_REDS)); \ (fcalls) = -CONTEXT_REDS; \ } \ @@ -91,22 +91,22 @@ do { \ if (!ERTS_PROC_GET_SAVED_CALLS_BUF((p))) { \ if ((p)->fcalls >= reds) { \ (p)->fcalls -= reds; \ - ERTS_PROC_GET_SCHDATA((p))->virtual_reds += reds; \ + erts_proc_sched_data((p))->virtual_reds += reds; \ } \ else { \ if ((p)->fcalls > 0) \ - ERTS_PROC_GET_SCHDATA((p))->virtual_reds += (p)->fcalls;\ + erts_proc_sched_data((p))->virtual_reds += (p)->fcalls; \ (p)->fcalls = 0; \ } \ } \ else { \ if ((p)->fcalls >= reds - CONTEXT_REDS) { \ (p)->fcalls -= reds; \ - ERTS_PROC_GET_SCHDATA((p))->virtual_reds += reds; \ + erts_proc_sched_data((p))->virtual_reds += reds; \ } \ else { \ if ((p)->fcalls > -CONTEXT_REDS) \ - ERTS_PROC_GET_SCHDATA((p))->virtual_reds \ + erts_proc_sched_data((p))->virtual_reds \ += (p)->fcalls - (-CONTEXT_REDS); \ (p)->fcalls = -CONTEXT_REDS; \ } \ @@ -118,14 +118,14 @@ do { \ if (ERTS_PROC_GET_SAVED_CALLS_BUF((P))) { \ int nreds__ = ((int)(Reds)) - CONTEXT_REDS; \ if ((FCalls) > nreds__) { \ - ERTS_PROC_GET_SCHDATA((P))->virtual_reds \ + erts_proc_sched_data((P))->virtual_reds \ += (FCalls) - nreds__; \ (FCalls) = nreds__; \ } \ } \ else { \ if ((FCalls) > (Reds)) { \ - ERTS_PROC_GET_SCHDATA((P))->virtual_reds \ + erts_proc_sched_data((P))->virtual_reds \ += (FCalls) - (Reds); \ (FCalls) = (Reds); \ } \ @@ -165,7 +165,7 @@ do { \ #define ERTS_BIF_ERROR_TRAPPED1(Proc, Reason, Bif, A0) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->freason = (Reason); \ (Proc)->current = (Bif)->code; \ reg[0] = (Eterm) (A0); \ @@ -174,7 +174,7 @@ do { \ #define ERTS_BIF_ERROR_TRAPPED2(Proc, Reason, Bif, A0, A1) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->freason = (Reason); \ (Proc)->current = (Bif)->code; \ reg[0] = (Eterm) (A0); \ @@ -184,7 +184,7 @@ do { \ #define ERTS_BIF_ERROR_TRAPPED3(Proc, Reason, Bif, A0, A1, A2) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->freason = (Reason); \ (Proc)->current = (Bif)->code; \ reg[0] = (Eterm) (A0); \ @@ -208,7 +208,7 @@ do { \ #define ERTS_BIF_PREP_ERROR_TRAPPED1(Ret, Proc, Reason, Bif, A0) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->freason = (Reason); \ (Proc)->current = (Bif)->code; \ reg[0] = (Eterm) (A0); \ @@ -217,7 +217,7 @@ do { \ #define ERTS_BIF_PREP_ERROR_TRAPPED2(Ret, Proc, Reason, Bif, A0, A1) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->freason = (Reason); \ (Proc)->current = (Bif)->code; \ reg[0] = (Eterm) (A0); \ @@ -227,7 +227,7 @@ do { \ #define ERTS_BIF_PREP_ERROR_TRAPPED3(Ret, Proc, Reason, Bif, A0, A1, A2) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->freason = (Reason); \ (Proc)->current = (Bif)->code; \ reg[0] = (Eterm) (A0); \ @@ -246,7 +246,7 @@ do { \ #define ERTS_BIF_PREP_TRAP1(Ret, Trap, Proc, A0) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->arity = 1; \ reg[0] = (Eterm) (A0); \ (Proc)->i = (BeamInstr*) ((Trap)->addressv[erts_active_code_ix()]); \ @@ -256,7 +256,7 @@ do { \ #define ERTS_BIF_PREP_TRAP2(Ret, Trap, Proc, A0, A1) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->arity = 2; \ reg[0] = (Eterm) (A0); \ reg[1] = (Eterm) (A1); \ @@ -267,7 +267,7 @@ do { \ #define ERTS_BIF_PREP_TRAP3(Ret, Trap, Proc, A0, A1, A2) \ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->arity = 3; \ reg[0] = (Eterm) (A0); \ reg[1] = (Eterm) (A1); \ @@ -279,7 +279,7 @@ do { \ #define ERTS_BIF_PREP_TRAP3_NO_RET(Trap, Proc, A0, A1, A2)\ do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((Proc))->x_reg_array; \ (Proc)->arity = 3; \ reg[0] = (Eterm) (A0); \ reg[1] = (Eterm) (A1); \ @@ -296,7 +296,7 @@ do { \ } while(0) #define BIF_TRAP1(Trap_, p, A0) do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((p))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((p))->x_reg_array; \ (p)->arity = 1; \ reg[0] = (A0); \ (p)->i = (BeamInstr*) ((Trap_)->addressv[erts_active_code_ix()]); \ @@ -305,7 +305,7 @@ do { \ } while(0) #define BIF_TRAP2(Trap_, p, A0, A1) do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((p))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((p))->x_reg_array; \ (p)->arity = 2; \ reg[0] = (A0); \ reg[1] = (A1); \ @@ -315,7 +315,7 @@ do { \ } while(0) #define BIF_TRAP3(Trap_, p, A0, A1, A2) do { \ - Eterm* reg = ERTS_PROC_GET_SCHDATA((p))->x_reg_array; \ + Eterm* reg = erts_proc_sched_data((p))->x_reg_array; \ (p)->arity = 3; \ reg[0] = (A0); \ reg[1] = (A1); \ diff --git a/erts/emulator/beam/binary.c b/erts/emulator/beam/binary.c index cfd8fbe2f5..071a356260 100644 --- a/erts/emulator/beam/binary.c +++ b/erts/emulator/beam/binary.c @@ -117,7 +117,7 @@ new_binary(Process *p, byte *buf, Uint len) * When heap binary is not desired... */ -Eterm erts_new_mso_binary(Process *p, byte *buf, int len) +Eterm erts_new_mso_binary(Process *p, byte *buf, Uint len) { ProcBin* pb; Binary* bptr; diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index d02c6828f9..3c19e82b66 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -118,7 +118,9 @@ process_killer(void) | ERTS_PSFLG_ACTIVE_SYS | ERTS_PSFLG_IN_RUNQ | ERTS_PSFLG_RUNNING - | ERTS_PSFLG_RUNNING_SYS)) { + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS)) { erts_printf("Can only kill WAITING processes this way\n"); } else { @@ -214,7 +216,8 @@ print_process_info(int to, void *to_arg, Process *p) if (state & ERTS_PSFLG_GC) { garbing = 1; running = 1; - } else if (state & ERTS_PSFLG_RUNNING) + } else if (state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING)) running = 1; /* diff --git a/erts/emulator/beam/erl_alloc.c b/erts/emulator/beam/erl_alloc.c index c367d4162c..3c2c9def3b 100644 --- a/erts/emulator/beam/erl_alloc.c +++ b/erts/emulator/beam/erl_alloc.c @@ -137,10 +137,13 @@ static ErtsAllocatorState_t exec_alloc_state; #endif static ErtsAllocatorState_t test_alloc_state; -#define ERTS_ALC_INFO_A_ALLOC_UTIL (ERTS_ALC_A_MAX + 1) -#define ERTS_ALC_INFO_A_MSEG_ALLOC (ERTS_ALC_A_MAX + 2) -#define ERTS_ALC_INFO_A_ERTS_MMAP (ERTS_ALC_A_MAX + 3) -#define ERTS_ALC_INFO_A_MAX ERTS_ALC_INFO_A_ERTS_MMAP +enum { + ERTS_ALC_INFO_A_ALLOC_UTIL = ERTS_ALC_A_MAX + 1, + ERTS_ALC_INFO_A_MSEG_ALLOC, + ERTS_ALC_INFO_A_ERTS_MMAP, + ERTS_ALC_INFO_A_DISABLED_EXEC, /* fake a disabled "exec_alloc" */ + ERTS_ALC_INFO_A_END +}; typedef struct { erts_smp_atomic32_t refc; @@ -150,7 +153,7 @@ typedef struct { Process *proc; Eterm ref; Eterm ref_heap[REF_THING_SIZE]; - int allocs[ERTS_ALC_INFO_A_MAX - ERTS_ALC_A_MIN + 1 + 1]; + int allocs[ERTS_ALC_INFO_A_END - ERTS_ALC_A_MIN + 1]; } ErtsAllocInfoReq; ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(aireq, @@ -3260,6 +3263,12 @@ reply_alloc_info(void *vair) am_false); #endif break; +#ifndef ERTS_ALC_A_EXEC + case ERTS_ALC_INFO_A_DISABLED_EXEC: + alloc_atom = erts_bld_atom(hpp, szp, "exec_alloc"); + ainfo = erts_bld_tuple2(hpp, szp, alloc_atom, am_false); + break; +#endif default: alloc_atom = erts_bld_atom(hpp, szp, (char *) ERTS_ALC_A2AD(ai)); @@ -3287,7 +3296,8 @@ reply_alloc_info(void *vair) switch (ai) { case ERTS_ALC_A_SYSTEM: case ERTS_ALC_INFO_A_ALLOC_UTIL: - case ERTS_ALC_INFO_A_ERTS_MMAP: + case ERTS_ALC_INFO_A_ERTS_MMAP: + case ERTS_ALC_INFO_A_DISABLED_EXEC: break; case ERTS_ALC_INFO_A_MSEG_ALLOC: #if HAVE_ERTS_MSEG && defined(ERTS_SMP) @@ -3359,7 +3369,7 @@ erts_request_alloc_info(struct process *c_p, int internal) { ErtsAllocInfoReq *air = aireq_alloc(); - Eterm req_ai[ERTS_ALC_INFO_A_MAX+1] = {0}; + Eterm req_ai[ERTS_ALC_INFO_A_END] = {0}; Eterm alist; Eterm *hp; int airix = 0, ai; @@ -3399,6 +3409,12 @@ erts_request_alloc_info(struct process *c_p, ai = ERTS_ALC_INFO_A_ERTS_MMAP; goto save_alloc; } +#ifndef ERTS_ALC_A_EXEC + if (erts_is_atom_str("exec_alloc", alloc, 0)) { + ai = ERTS_ALC_INFO_A_DISABLED_EXEC; + goto save_alloc; + } +#endif if (erts_is_atom_str("alloc_util", alloc, 0)) { ai = ERTS_ALC_INFO_A_ALLOC_UTIL; save_alloc: diff --git a/erts/emulator/beam/erl_alloc.types b/erts/emulator/beam/erl_alloc.types index ba216c7eb4..227fedfb69 100644 --- a/erts/emulator/beam/erl_alloc.types +++ b/erts/emulator/beam/erl_alloc.types @@ -278,6 +278,7 @@ type IOB_REQ SHORT_LIVED SYSTEM io_bytes_request type TRACER_NIF LONG_LIVED SYSTEM tracer_nif type TRACE_MSG_QUEUE SHORT_LIVED SYSTEM trace_message_queue type SCHED_ASYNC_JOB SHORT_LIVED SYSTEM async_calls +type DIRTY_START STANDARD PROCESSES dirty_start +if threads_no_smp # Need thread safe allocs, but std_alloc and fix_alloc are not; diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 85cadafebc..2e195db0ee 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -45,6 +45,7 @@ #include "erl_async.h" #include "erl_thr_progress.h" #include "erl_bif_unique.h" +#include "erl_map.h" #define ERTS_PTAB_WANT_DEBUG_FUNCS__ #include "erl_ptab.h" #ifdef HIPE @@ -594,6 +595,7 @@ static Eterm pi_args[] = { am_suspending, am_min_heap_size, am_min_bin_vheap_size, + am_max_heap_size, am_current_location, am_current_stacktrace, am_message_queue_data, @@ -643,10 +645,11 @@ pi_arg2ix(Eterm arg) case am_suspending: return 26; case am_min_heap_size: return 27; case am_min_bin_vheap_size: return 28; - case am_current_location: return 29; - case am_current_stacktrace: return 30; - case am_message_queue_data: return 31; - case am_garbage_collection_info: return 32; + case am_max_heap_size: return 29; + case am_current_location: return 30; + case am_current_stacktrace: return 31; + case am_message_queue_data: return 32; + case am_garbage_collection_info: return 33; default: return -1; } } @@ -1107,7 +1110,7 @@ process_info_aux(Process *BIF_P, break; case am_status: - res = erts_process_status(BIF_P, ERTS_PROC_LOCK_MAIN, rp, rpid); + res = erts_process_status(rp, rpid); ASSERT(res != am_undefined); hp = HAlloc(BIF_P, 3); break; @@ -1348,6 +1351,18 @@ process_info_aux(Process *BIF_P, break; } + case am_max_heap_size: { + Uint hsz = 3; + (void) erts_max_heap_size_map(MAX_HEAP_SIZE_GET(rp), + MAX_HEAP_SIZE_FLAGS_GET(rp), + NULL, &hsz); + hp = HAlloc(BIF_P, hsz); + res = erts_max_heap_size_map(MAX_HEAP_SIZE_GET(rp), + MAX_HEAP_SIZE_FLAGS_GET(rp), + &hp, NULL); + break; + } + case am_total_heap_size: { ErtsMessage *mp; Uint total_heap_size; @@ -1391,8 +1406,12 @@ process_info_aux(Process *BIF_P, case am_garbage_collection: { DECL_AM(minor_gcs); Eterm t; + Uint map_sz = 0; + + erts_max_heap_size_map(MAX_HEAP_SIZE_GET(rp), MAX_HEAP_SIZE_FLAGS_GET(rp), NULL, &map_sz); - hp = HAlloc(BIF_P, 3+2 + 3+2 + 3+2 + 3+2 + 3); /* last "3" is for outside tuple */ + hp = HAlloc(BIF_P, 3+2 + 3+2 + 3+2 + 3+2 + 3+2 + map_sz + 3); + /* last "3" is for outside tuple */ t = TUPLE2(hp, AM_minor_gcs, make_small(GEN_GCS(rp))); hp += 3; res = CONS(hp, t, NIL); hp += 2; @@ -1403,6 +1422,11 @@ process_info_aux(Process *BIF_P, res = CONS(hp, t, res); hp += 2; t = TUPLE2(hp, am_min_bin_vheap_size, make_small(MIN_VHEAP_SIZE(rp))); hp += 3; res = CONS(hp, t, res); hp += 2; + + t = erts_max_heap_size_map(MAX_HEAP_SIZE_GET(rp), MAX_HEAP_SIZE_FLAGS_GET(rp), &hp, NULL); + + t = TUPLE2(hp, am_max_heap_size, t); hp += 3; + res = CONS(hp, t, res); hp += 2; break; } @@ -1412,12 +1436,12 @@ process_info_aux(Process *BIF_P, if (rp == BIF_P) { sz += ERTS_PROCESS_GC_INFO_MAX_SIZE; } else { - erts_process_gc_info(rp, &sz, NULL); + erts_process_gc_info(rp, &sz, NULL, 0, 0); sz += 3; } hp = HAlloc(BIF_P, sz); - res = erts_process_gc_info(rp, &actual_sz, &hp); + res = erts_process_gc_info(rp, &actual_sz, &hp, 0, 0); /* We may have some extra space, fill with 0 tuples */ if (actual_sz <= sz - 3) { @@ -2035,12 +2059,8 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) Uint arity = *tp++; return info_1_tuple(BIF_P, tp, arityval(arity)); } else if (BIF_ARG_1 == am_scheduler_id) { -#ifdef ERTS_SMP - ASSERT(BIF_P->scheduler_data); - BIF_RET(make_small(BIF_P->scheduler_data->no)); -#else - BIF_RET(make_small(1)); -#endif + ErtsSchedulerData *esdp = erts_proc_sched_data(BIF_P); + BIF_RET(make_small(esdp->no)); } else if (BIF_ARG_1 == am_compat_rel) { ASSERT(erts_compat_rel > 0); BIF_RET(make_small(erts_compat_rel)); @@ -2173,7 +2193,7 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) } else if (BIF_ARG_1 == am_garbage_collection){ Uint val = (Uint) erts_smp_atomic32_read_nob(&erts_max_gen_gcs); Eterm tup; - hp = HAlloc(BIF_P, 3+2 + 3+2 + 3+2); + hp = HAlloc(BIF_P, 3+2 + 3+2 + 3+2 + 3+2); tup = TUPLE2(hp, am_fullsweep_after, make_small(val)); hp += 3; res = CONS(hp, tup, NIL); hp += 2; @@ -2184,6 +2204,9 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) tup = TUPLE2(hp, am_min_bin_vheap_size, make_small(BIN_VH_MIN_SIZE)); hp += 3; res = CONS(hp, tup, res); hp += 2; + tup = TUPLE2(hp, am_max_heap_size, make_small(H_MAX_SIZE)); hp += 3; + res = CONS(hp, tup, res); hp += 2; + BIF_RET(res); } else if (BIF_ARG_1 == am_fullsweep_after){ Uint val = (Uint) erts_smp_atomic32_read_nob(&erts_max_gen_gcs); @@ -2194,6 +2217,12 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) hp = HAlloc(BIF_P, 3); res = TUPLE2(hp, am_min_heap_size,make_small(H_MIN_SIZE)); BIF_RET(res); + } else if (BIF_ARG_1 == am_max_heap_size) { + Uint sz = 0; + erts_max_heap_size_map(H_MAX_SIZE, H_MAX_FLAGS, NULL, &sz); + hp = HAlloc(BIF_P, sz); + res = erts_max_heap_size_map(H_MAX_SIZE, H_MAX_FLAGS, &hp, NULL); + BIF_RET(res); } else if (BIF_ARG_1 == am_min_bin_vheap_size) { hp = HAlloc(BIF_P, 3); res = TUPLE2(hp, am_min_bin_vheap_size,make_small(BIN_VH_MIN_SIZE)); @@ -3589,10 +3618,7 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) /* Used by timer process_SUITE, timer_bif_SUITE, and node_container_SUITE (emulator) */ if (is_internal_pid(tp[2])) { - BIF_RET(erts_process_status(BIF_P, - ERTS_PROC_LOCK_MAIN, - NULL, - tp[2])); + BIF_RET(erts_process_status(NULL, tp[2])); } } else if (ERTS_IS_ATOM_STR("link_list", tp[1])) { diff --git a/erts/emulator/beam/erl_bif_re.c b/erts/emulator/beam/erl_bif_re.c index f4daecd2a4..ff7746ce1d 100644 --- a/erts/emulator/beam/erl_bif_re.c +++ b/erts/emulator/beam/erl_bif_re.c @@ -630,9 +630,15 @@ static Eterm build_exec_return(Process *p, int rc, RestartContext *restartp, Ete } } else { ReturnInfo *ri; - ReturnInfo defri = {RetIndex,0,{0}}; + ReturnInfo defri; if (restartp->ret_info == NULL) { + /* OpenBSD 5.8 gcc compiler for some reason creates + bad code if the above initialization is done + inline with the struct. So don't do that. */ + defri.type = RetIndex; + defri.num_spec = 0; + defri.v[0] = 0; ri = &defri; } else { ri = restartp->ret_info; diff --git a/erts/emulator/beam/erl_bif_trace.c b/erts/emulator/beam/erl_bif_trace.c index b65c0e303f..66e5146da0 100644 --- a/erts/emulator/beam/erl_bif_trace.c +++ b/erts/emulator/beam/erl_bif_trace.c @@ -512,8 +512,7 @@ start_trace(Process *c_p, ErtsTracer tracer, && !ERTS_TRACER_COMPARE(ERTS_TRACER(port), tracer)) { /* This tracee is already being traced, and not by the * tracer to be */ - if (erts_is_tracer_proc_enabled(c_p, ERTS_PROC_LOCKS_ALL, - common, am_trace_status)) { + if (erts_is_tracer_enabled(tracer, common)) { /* The tracer is still in use */ return 1; } @@ -856,7 +855,7 @@ trace_info_pid(Process* p, Eterm pid_spec, Eterm key) return am_undefined; if (!ERTS_TRACER_IS_NIL(ERTS_TRACER(tracee))) - erts_is_tracer_proc_enabled(NULL, 0, &tracee->common, am_trace_status); + erts_is_tracer_proc_enabled(NULL, 0, &tracee->common); tracer = erts_tracer_to_term(p, ERTS_TRACER(tracee)); trace_flags = ERTS_TRACE_FLAGS(tracee); @@ -864,22 +863,24 @@ trace_info_pid(Process* p, Eterm pid_spec, Eterm key) erts_port_release(tracee); } else if (is_internal_pid(pid_spec)) { - Process *tracee; - tracee = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, - pid_spec, ERTS_PROC_LOCK_MAIN); + Process *tracee = erts_pid2proc_not_running(p, ERTS_PROC_LOCK_MAIN, + pid_spec, ERTS_PROC_LOCK_MAIN); + + if (tracee == ERTS_PROC_LOCK_BUSY) + ERTS_BIF_YIELD2(bif_export[BIF_trace_info_2], p, pid_spec, key); if (!tracee) return am_undefined; if (!ERTS_TRACER_IS_NIL(ERTS_TRACER(tracee))) erts_is_tracer_proc_enabled(tracee, ERTS_PROC_LOCK_MAIN, - &tracee->common, am_trace_status); + &tracee->common); tracer = erts_tracer_to_term(p, ERTS_TRACER(tracee)); trace_flags = ERTS_TRACE_FLAGS(tracee); - if (tracee != p) - erts_smp_proc_unlock(tracee, ERTS_PROC_LOCK_MAIN); + if (tracee != p) + erts_smp_proc_unlock(tracee, ERTS_PROC_LOCK_MAIN); } else if (is_external_pid(pid_spec) && external_pid_dist_entry(pid_spec) == erts_this_dist_entry) { return am_undefined; diff --git a/erts/emulator/beam/erl_bif_unique.c b/erts/emulator/beam/erl_bif_unique.c index 1e57e9fa53..7c70217d8d 100644 --- a/erts/emulator/beam/erl_bif_unique.c +++ b/erts/emulator/beam/erl_bif_unique.c @@ -257,7 +257,7 @@ static ERTS_INLINE Eterm unique_integer_bif(Process *c_p, int positive) Uint hsz; Eterm *hp; - esdp = ERTS_PROC_GET_SCHDATA(c_p); + esdp = erts_proc_sched_data(c_p); thr_id = (Uint64) esdp->thr_id; unique = esdp->unique++; bld_unique_integer_term(NULL, &hsz, thr_id, unique, positive); @@ -515,7 +515,7 @@ BIF_RETTYPE make_ref_0(BIF_ALIST_0) hp = HAlloc(BIF_P, REF_THING_SIZE); - res = erts_sched_make_ref_in_buffer(ERTS_PROC_GET_SCHDATA(BIF_P), hp); + res = erts_sched_make_ref_in_buffer(erts_proc_sched_data(BIF_P), hp); BIF_RET(res); } diff --git a/erts/emulator/beam/erl_bits.h b/erts/emulator/beam/erl_bits.h index 1c2a090f07..4bd5b24157 100644 --- a/erts/emulator/beam/erl_bits.h +++ b/erts/emulator/beam/erl_bits.h @@ -83,8 +83,8 @@ typedef struct erl_bin_match_struct{ #ifdef ERTS_SMP /* the state resides in the current process' scheduler data */ #define ERL_BITS_DECLARE_STATEP struct erl_bits_state *EBS -#define ERL_BITS_RELOAD_STATEP(P) do{EBS = &(P)->scheduler_data->erl_bits_state;}while(0) -#define ERL_BITS_DEFINE_STATEP(P) struct erl_bits_state *EBS = &(P)->scheduler_data->erl_bits_state +#define ERL_BITS_RELOAD_STATEP(P) do{EBS = &erts_proc_sched_data((P))->erl_bits_state;}while(0) +#define ERL_BITS_DEFINE_STATEP(P) struct erl_bits_state *EBS = &erts_proc_sched_data((P))->erl_bits_state #else /* reentrant API but with a hidden single global state, for testing only */ extern struct erl_bits_state ErlBitsState_; diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index acaca54e9a..bad34211a5 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -3462,7 +3462,7 @@ static void fix_table_locked(Process* p, DbTable* tb) fix = tb->common.fixations; if (fix == NULL) { tb->common.time.monotonic - = erts_get_monotonic_time(ERTS_PROC_GET_SCHDATA(p)); + = erts_get_monotonic_time(erts_proc_sched_data(p)); tb->common.time.offset = erts_get_time_offset(); } else { diff --git a/erts/emulator/beam/erl_db_util.c b/erts/emulator/beam/erl_db_util.c index 95b1cd0148..6732b708a8 100644 --- a/erts/emulator/beam/erl_db_util.c +++ b/erts/emulator/beam/erl_db_util.c @@ -174,7 +174,8 @@ set_match_trace(Process *tracee_p, Eterm fail_term, ErtsTracer tracer, ERTS_PROC_LOCKS_ALL == erts_proc_lc_my_proc_locks(tracee_p) || erts_thr_progress_is_blocking()); - if (ERTS_TRACER_IS_NIL(tracer) || erts_is_tracer_enabled(tracee_p, tracer)) + if (ERTS_TRACER_IS_NIL(tracer) + || erts_is_tracer_enabled(tracer, &tracee_p->common)) return set_tracee_flags(tracee_p, tracer, d_flags, e_flags); return fail_term; } @@ -1801,7 +1802,7 @@ Eterm db_prog_match(Process *c_p, /* We need to lure the scheduler into believing in the pseudo process, because of floating point exceptions. Do *after* mpsp is set!!! */ - esdp = ERTS_GET_SCHEDULER_DATA_FROM_PROC(psp); + esdp = erts_get_scheduler_data(); if (esdp) current_scheduled = esdp->current_process; /* SMP: psp->scheduler_data is set by get_match_pseudo_process */ diff --git a/erts/emulator/beam/erl_drv_nif.h b/erts/emulator/beam/erl_drv_nif.h index 2700b62854..6ec5fbb895 100644 --- a/erts/emulator/beam/erl_drv_nif.h +++ b/erts/emulator/beam/erl_drv_nif.h @@ -43,12 +43,11 @@ typedef struct { int suggested_stack_size; } ErlDrvThreadOpts; -#if defined(ERL_DRV_DIRTY_SCHEDULER_SUPPORT) || defined(ERL_NIF_DIRTY_SCHEDULER_SUPPORT) + typedef enum { - ERL_DRV_DIRTY_JOB_CPU_BOUND = 1, - ERL_DRV_DIRTY_JOB_IO_BOUND = 2 -} ErlDrvDirtyJobFlags; -#endif + ERL_DIRTY_JOB_CPU_BOUND = 1, + ERL_DIRTY_JOB_IO_BOUND = 2 +} ErlDirtyJobFlags; #ifdef SIZEOF_CHAR # define SIZEOF_CHAR_SAVED__ SIZEOF_CHAR diff --git a/erts/emulator/beam/erl_gc.c b/erts/emulator/beam/erl_gc.c index bed7e668d7..d740b2baec 100644 --- a/erts/emulator/beam/erl_gc.c +++ b/erts/emulator/beam/erl_gc.c @@ -41,6 +41,7 @@ #endif #include "dtrace-wrapper.h" #include "erl_bif_unique.h" +#include "dist.h" #define ERTS_INACT_WR_PB_LEAVE_MUCH_LIMIT 1 #define ERTS_INACT_WR_PB_LEAVE_MUCH_PERCENTAGE 20 @@ -146,7 +147,8 @@ static void offset_rootset(Process *p, Sint offs, char* area, Uint area_size, static void offset_off_heap(Process* p, Sint offs, char* area, Uint area_size); static void offset_mqueue(Process *p, Sint offs, char* area, Uint area_size); static void move_msgq_to_heap(Process *p); - +static int reached_max_heap_size(Process *p, Uint total_heap_size, + Uint extra_heap_size, Uint extra_old_heap_size); static void init_gc_info(ErtsGCInfo *gcip); #ifdef HARDDEBUG @@ -389,7 +391,7 @@ erts_gc_after_bif_call_lhf(Process* p, ErlHeapFragment *live_hf_end, if (p->freason == TRAP) { #if HIPE if (regs == NULL) { - regs = ERTS_PROC_GET_SCHDATA(p)->x_reg_array; + regs = erts_proc_sched_data(p)->x_reg_array; } #endif cost = garbage_collect(p, live_hf_end, 0, regs, p->arity, p->fcalls); @@ -404,6 +406,7 @@ erts_gc_after_bif_call_lhf(Process* p, ErlHeapFragment *live_hf_end, result = val[0]; } BUMP_REDS(p, cost); + return result; } @@ -507,14 +510,14 @@ delay_garbage_collection(Process *p, ErlHeapFragment *live_hf_end, int need, int /* Make sure that we do a proper GC as soon as possible... */ p->flags |= F_FORCE_GC; reds_left = ERTS_REDS_LEFT(p, fcalls); - ASSERT(CONTEXT_REDS - reds_left >= ERTS_PROC_GET_SCHDATA(p)->virtual_reds); + ASSERT(CONTEXT_REDS - reds_left >= erts_proc_sched_data(p)->virtual_reds); if (reds_left > ERTS_ABANDON_HEAP_COST) { int vreds = reds_left - ERTS_ABANDON_HEAP_COST; - ERTS_PROC_GET_SCHDATA((p))->virtual_reds += vreds; + erts_proc_sched_data((p))->virtual_reds += vreds; } - ASSERT(CONTEXT_REDS >= ERTS_PROC_GET_SCHDATA(p)->virtual_reds); + ASSERT(CONTEXT_REDS >= erts_proc_sched_data(p)->virtual_reds); return reds_left; } @@ -577,18 +580,22 @@ garbage_collect(Process* p, ErlHeapFragment *live_hf_end, int need, Eterm* objv, int nobj, int fcalls) { Uint reclaimed_now = 0; + Eterm gc_trace_end_tag; int reds; ErtsMonotonicTime start_time = 0; /* Shut up faulty warning... */ ErtsSchedulerData *esdp; + erts_aint32_t state; ERTS_MSACC_PUSH_STATE_M(); #ifdef USE_VM_PROBES DTRACE_CHARBUF(pidbuf, DTRACE_TERM_BUF_SIZE); #endif ASSERT(CONTEXT_REDS - ERTS_REDS_LEFT(p, fcalls) - >= ERTS_PROC_GET_SCHDATA(p)->virtual_reds); + >= erts_proc_sched_data(p)->virtual_reds); - if (p->flags & (F_DISABLE_GC|F_DELAY_GC)) + state = erts_smp_atomic32_read_nob(&p->state); + + if (p->flags & (F_DISABLE_GC|F_DELAY_GC) || state & ERTS_PSFLG_EXITING) return delay_garbage_collection(p, live_hf_end, need, fcalls); if (p->abandoned_heap) @@ -623,28 +630,28 @@ garbage_collect(Process* p, ErlHeapFragment *live_hf_end, if (GEN_GCS(p) < MAX_GEN_GCS(p) && !(FLAGS(p) & F_NEED_FULLSWEEP)) { if (IS_TRACED_FL(p, F_TRACE_GC)) { - trace_gc(p, am_gc_minor_start, need); + trace_gc(p, am_gc_minor_start, need, THE_NON_VALUE); } DTRACE2(gc_minor_start, pidbuf, need); reds = minor_collection(p, live_hf_end, need, objv, nobj, &reclaimed_now); DTRACE2(gc_minor_end, pidbuf, reclaimed_now); - if (IS_TRACED_FL(p, F_TRACE_GC)) { - trace_gc(p, am_gc_minor_end, reclaimed_now); - } - if (reds < 0) + if (reds == -1) { + if (IS_TRACED_FL(p, F_TRACE_GC)) { + trace_gc(p, am_gc_minor_end, reclaimed_now, THE_NON_VALUE); + } goto do_major_collection; + } + gc_trace_end_tag = am_gc_minor_end; } else { do_major_collection: ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_GC_FULL); if (IS_TRACED_FL(p, F_TRACE_GC)) { - trace_gc(p, am_gc_major_start, need); + trace_gc(p, am_gc_major_start, need, THE_NON_VALUE); } DTRACE2(gc_major_start, pidbuf, need); reds = major_collection(p, live_hf_end, need, objv, nobj, &reclaimed_now); DTRACE2(gc_major_end, pidbuf, reclaimed_now); - if (IS_TRACED_FL(p, F_TRACE_GC)) { - trace_gc(p, am_gc_major_end, reclaimed_now); - } + gc_trace_end_tag = am_gc_major_end; ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_GC); } @@ -658,8 +665,33 @@ do_major_collection: ErtsGcQuickSanityCheck(p); + /* Max heap size has been reached and the process was configured + to be killed, so we kill it and set it in a delayed garbage + collecting state. There should be no gc_end trace or + long_gc/large_gc triggers when this happens as process was + killed before a GC could be done. */ + if (reds == -2) { + ErtsProcLocks locks = ERTS_PROC_LOCKS_ALL; + + erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); + erts_send_exit_signal(p, p->common.id, p, &locks, + am_kill, NIL, NULL, 0); + erts_smp_proc_unlock(p, locks & ERTS_PROC_LOCKS_ALL_MINOR); + + /* erts_send_exit_signal looks for ERTS_PSFLG_GC, so + we have to remove it after the signal is sent */ + erts_smp_atomic32_read_band_nob(&p->state, ~ERTS_PSFLG_GC); + + /* We have to make sure that we have space for need on the heap */ + return delay_garbage_collection(p, live_hf_end, need, fcalls); + } + erts_smp_atomic32_read_band_nob(&p->state, ~ERTS_PSFLG_GC); + if (IS_TRACED_FL(p, F_TRACE_GC)) { + trace_gc(p, gc_trace_end_tag, reclaimed_now, THE_NON_VALUE); + } + if (erts_system_monitor_long_gc != 0) { ErtsMonotonicTime end_time; Uint gc_time; @@ -716,7 +748,7 @@ erts_garbage_collect_nobump(Process* p, int need, Eterm* objv, int nobj, int fca int reds_left = ERTS_REDS_LEFT(p, fcalls); if (reds > reds_left) reds = reds_left; - ASSERT(CONTEXT_REDS - (reds_left - reds) >= ERTS_PROC_GET_SCHDATA(p)->virtual_reds); + ASSERT(CONTEXT_REDS - (reds_left - reds) >= erts_proc_sched_data(p)->virtual_reds); return reds; } @@ -726,7 +758,7 @@ erts_garbage_collect(Process* p, int need, Eterm* objv, int nobj) int reds = garbage_collect(p, ERTS_INVALID_HFRAG_PTR, need, objv, nobj, p->fcalls); BUMP_REDS(p, reds); ASSERT(CONTEXT_REDS - ERTS_BIF_REDS_LEFT(p) - >= ERTS_PROC_GET_SCHDATA(p)->virtual_reds); + >= erts_proc_sched_data(p)->virtual_reds); } /* @@ -761,6 +793,7 @@ erts_garbage_collect_hibernate(Process* p) heap_size = p->heap_sz + (p->old_htop - p->old_heap) + p->mbuf_sz; + heap = (Eterm*) ERTS_HEAP_ALLOC(ERTS_ALC_T_TMP_HEAP, sizeof(Eterm)*heap_size); htop = heap; @@ -1031,6 +1064,34 @@ minor_collection(Process* p, ErlHeapFragment *live_hf_end, Uint size_before = young_gen_usage(p); /* + * Check if we have gone past the max heap size limit + */ + + if (MAX_HEAP_SIZE_GET(p)) { + Uint heap_size = size_before, + /* Note that we also count the un-allocated area + in between the stack and heap */ + stack_size = HEAP_END(p) - HEAP_TOP(p), + extra_heap_size, + extra_old_heap_size = 0; + + /* Add potential old heap size */ + if (OLD_HEAP(p) == NULL && mature_size != 0) { + extra_old_heap_size = erts_next_heap_size(size_before, 1); + heap_size += extra_old_heap_size; + } else if (OLD_HEAP(p)) + heap_size += OLD_HEND(p) - OLD_HEAP(p); + + /* Add potential new young heap size */ + extra_heap_size = next_heap_size(p, stack_size + size_before, 0); + heap_size += extra_heap_size; + + if (heap_size > MAX_HEAP_SIZE_GET(p)) + if (reached_max_heap_size(p, heap_size, extra_heap_size, extra_old_heap_size)) + return -2; + } + + /* * Allocate an old heap if we don't have one and if we'll need one. */ @@ -1139,6 +1200,16 @@ minor_collection(Process* p, ErlHeapFragment *live_hf_end, ASSERT(HEAP_SIZE(p) == next_heap_size(p, HEAP_SIZE(p), 0)); ASSERT(MBUF(p) == NULL); + /* The heap usage during GC should be larger than what we end up + after a GC, even if we grow it. If this assertion is not true + we have to check size in grow_new_heap and potentially kill the + process from there */ + ASSERT(!MAX_HEAP_SIZE_GET(p) || + !(MAX_HEAP_SIZE_FLAGS_GET(p) & MAX_HEAP_SIZE_KILL) || + MAX_HEAP_SIZE_GET(p) > (young_gen_usage(p) + + (OLD_HEND(p) - OLD_HEAP(p)) + + (HEAP_END(p) - HEAP_TOP(p)))); + return gc_cost(size_after, adjust_size); } @@ -1457,6 +1528,25 @@ major_collection(Process* p, ErlHeapFragment *live_hf_end, if (new_sz == HEAP_SIZE(p) && FLAGS(p) & F_HEAP_GROW) { new_sz = next_heap_size(p, HEAP_SIZE(p), 1); } + + + if (MAX_HEAP_SIZE_GET(p)) { + Uint heap_size = size_before; + + /* Add unused space in old heap */ + heap_size += OLD_HEND(p) - OLD_HTOP(p); + + /* Add stack + unused space in young heap */ + heap_size += HEAP_END(p) - HEAP_TOP(p); + + /* Add size of new young heap */ + heap_size += new_sz; + + if (MAX_HEAP_SIZE_GET(p) < heap_size) + if (reached_max_heap_size(p, heap_size, new_sz, 0)) + return -2; + } + FLAGS(p) &= ~(F_HEAP_GROW|F_NEED_FULLSWEEP); n_htop = n_heap = (Eterm *) ERTS_HEAP_ALLOC(ERTS_ALC_T_HEAP, sizeof(Eterm)*new_sz); @@ -2955,7 +3045,7 @@ reply_gc_info(void *vgcirp) Eterm erts_gc_info_request(Process *c_p) { - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(c_p); + ErtsSchedulerData *esdp = erts_proc_sched_data(c_p); Eterm ref; ErtsGCInfoReq *gcirp; Eterm *hp; @@ -2986,7 +3076,9 @@ erts_gc_info_request(Process *c_p) } Eterm -erts_process_gc_info(Process *p, Uint *sizep, Eterm **hpp) +erts_process_gc_info(Process *p, Uint *sizep, Eterm **hpp, + Uint extra_heap_block, + Uint extra_old_heap_block_size) { ERTS_DECL_AM(bin_vheap_size); ERTS_DECL_AM(bin_vheap_block_size); @@ -3009,8 +3101,9 @@ erts_process_gc_info(Process *p, Uint *sizep, Eterm **hpp) AM_bin_old_vheap_block_size }; UWord values[] = { - OLD_HEAP(p) ? OLD_HEND(p) - OLD_HEAP(p) : 0, - HEAP_SIZE(p), + OLD_HEAP(p) ? OLD_HEND(p) - OLD_HEAP(p) + extra_old_heap_block_size + : extra_old_heap_block_size, + HEAP_SIZE(p) + extra_heap_block, MBUF_SIZE(p), HIGH_WATER(p) - HEAP_START(p), STACK_START(p) - p->stop, @@ -3056,6 +3149,130 @@ erts_process_gc_info(Process *p, Uint *sizep, Eterm **hpp) return res; } +static int +reached_max_heap_size(Process *p, Uint total_heap_size, + Uint extra_heap_size, Uint extra_old_heap_size) +{ + Uint max_heap_flags = MAX_HEAP_SIZE_FLAGS_GET(p); + if (IS_TRACED_FL(p, F_TRACE_GC) || + max_heap_flags & MAX_HEAP_SIZE_LOG) { + Eterm msg; + Uint size = 0; + Eterm *o_hp , *hp; + erts_process_gc_info(p, &size, NULL, extra_heap_size, + extra_old_heap_size); + o_hp = hp = erts_alloc(ERTS_ALC_T_TMP, size * sizeof(Eterm)); + msg = erts_process_gc_info(p, NULL, &hp, extra_heap_size, + extra_old_heap_size); + + if (max_heap_flags & MAX_HEAP_SIZE_LOG) { + int alive = erts_is_alive; + erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); + Eterm *o_hp, *hp, args = NIL; + + /* Build the format message */ + erts_dsprintf(dsbufp, " Process: ~p "); + if (alive) + erts_dsprintf(dsbufp, "on node ~p"); + erts_dsprintf(dsbufp, "~n Context: maximum heap size reached~n"); + erts_dsprintf(dsbufp, " Max Heap Size: ~p~n"); + erts_dsprintf(dsbufp, " Total Heap Size: ~p~n"); + erts_dsprintf(dsbufp, " Kill: ~p~n"); + erts_dsprintf(dsbufp, " Error Logger: ~p~n"); + erts_dsprintf(dsbufp, " GC Info: ~p~n"); + + /* Build the args in reverse order */ + o_hp = hp = erts_alloc(ERTS_ALC_T_TMP, 2*(alive ? 7 : 6) * sizeof(Eterm)); + args = CONS(hp, msg, args); hp += 2; + args = CONS(hp, am_true, args); hp += 2; + args = CONS(hp, (max_heap_flags & MAX_HEAP_SIZE_KILL ? am_true : am_false), args); hp += 2; + args = CONS(hp, make_small(total_heap_size), args); hp += 2; + args = CONS(hp, make_small(MAX_HEAP_SIZE_GET(p)), args); hp += 2; + if (alive) { + args = CONS(hp, erts_this_node->sysname, args); hp += 2; + } + args = CONS(hp, p->common.id, args); hp += 2; + + erts_send_error_term_to_logger(p->group_leader, dsbufp, args); + erts_free(ERTS_ALC_T_TMP, o_hp); + } + + if (IS_TRACED_FL(p, F_TRACE_GC)) + trace_gc(p, am_gc_max_heap_size, 0, msg); + + erts_free(ERTS_ALC_T_TMP, o_hp); + } + /* returns true if we should kill the process */ + return max_heap_flags & MAX_HEAP_SIZE_KILL; +} + +Eterm +erts_max_heap_size_map(Sint max_heap_size, Uint max_heap_flags, + Eterm **hpp, Uint *sz) +{ + if (!hpp) { + *sz += (2*3 + 1 + MAP_HEADER_FLATMAP_SZ); + return THE_NON_VALUE; + } else { + Eterm *hp = *hpp; + Eterm keys = TUPLE3(hp, am_error_logger, am_kill, am_size); + flatmap_t *mp; + hp += 4; + mp = (flatmap_t*) hp; + mp->thing_word = MAP_HEADER_FLATMAP; + mp->size = 3; + mp->keys = keys; + hp += MAP_HEADER_FLATMAP_SZ; + *hp++ = max_heap_flags & MAX_HEAP_SIZE_LOG ? am_true : am_false; + *hp++ = max_heap_flags & MAX_HEAP_SIZE_KILL ? am_true : am_false; + *hp++ = make_small(max_heap_size); + *hpp = hp; + return make_flatmap(mp); + } +} + +int +erts_max_heap_size(Eterm arg, Uint *max_heap_size, Uint *max_heap_flags) +{ + Sint sz; + *max_heap_flags = H_MAX_FLAGS; + if (is_small(arg)) { + sz = signed_val(arg); + *max_heap_flags = H_MAX_FLAGS; + } else if (is_map(arg)) { + const Eterm *size = erts_maps_get(am_size, arg); + const Eterm *kill = erts_maps_get(am_kill, arg); + const Eterm *log = erts_maps_get(am_error_logger, arg); + if (size && is_small(*size)) { + sz = signed_val(*size); + } else { + /* size is mandatory */ + return 0; + } + if (kill) { + if (*kill == am_true) + *max_heap_flags |= MAX_HEAP_SIZE_KILL; + else if (*kill == am_false) + *max_heap_flags &= ~MAX_HEAP_SIZE_KILL; + else + return 0; + } + if (log) { + if (*log == am_true) + *max_heap_flags |= MAX_HEAP_SIZE_LOG; + else if (*log == am_false) + *max_heap_flags &= ~MAX_HEAP_SIZE_LOG; + else + return 0; + } + } else + return 0; + if (sz < 0) + return 0; + *max_heap_size = sz; + return 1; +} + #if defined(DEBUG) || defined(ERTS_OFFHEAP_DEBUG) static int diff --git a/erts/emulator/beam/erl_gc.h b/erts/emulator/beam/erl_gc.h index 8a6ff99990..54ea9ca3c0 100644 --- a/erts/emulator/beam/erl_gc.h +++ b/erts/emulator/beam/erl_gc.h @@ -135,7 +135,7 @@ typedef struct { #define ERTS_PROCESS_GC_INFO_MAX_TERMS (11) /* number of elements in process_gc_info*/ #define ERTS_PROCESS_GC_INFO_MAX_SIZE \ (ERTS_PROCESS_GC_INFO_MAX_TERMS * (2/*cons*/ + 3/*2-tuple*/ + BIG_UINT_HEAP_SIZE)) -Eterm erts_process_gc_info(struct process*, Uint *, Eterm **); +Eterm erts_process_gc_info(struct process*, Uint *, Eterm **, Uint, Uint); void erts_gc_info(ErtsGCInfo *gcip); void erts_init_gc(void); @@ -155,5 +155,7 @@ void erts_offset_off_heap(struct erl_off_heap*, Sint, Eterm*, Eterm*); void erts_offset_heap_ptr(Eterm*, Uint, Sint, Eterm*, Eterm*); void erts_offset_heap(Eterm*, Uint, Sint, Eterm*, Eterm*); void erts_free_heap_frags(struct process* p); +Eterm erts_max_heap_size_map(Sint, Uint, Eterm **, Uint *); +int erts_max_heap_size(Eterm, Uint *, Uint *); #endif /* __ERL_GC_H__ */ diff --git a/erts/emulator/beam/erl_hl_timer.c b/erts/emulator/beam/erl_hl_timer.c index c418762578..ebeff51aac 100644 --- a/erts/emulator/beam/erl_hl_timer.c +++ b/erts/emulator/beam/erl_hl_timer.c @@ -1766,7 +1766,7 @@ setup_bif_timer(Process *c_p, ErtsMonotonicTime timeout_pos, if (is_not_internal_pid(rcvr) && is_not_atom(rcvr)) goto badarg; - esdp = ERTS_PROC_GET_SCHDATA(c_p); + esdp = erts_proc_sched_data(c_p); hp = HAlloc(c_p, REF_THING_SIZE); ref = erts_sched_make_ref_in_buffer(esdp, hp); @@ -1871,7 +1871,7 @@ access_sched_local_btm(Process *c_p, Eterm pid, if (!c_p) esdp = erts_get_scheduler_data(); else { - esdp = ERTS_PROC_GET_SCHDATA(c_p); + esdp = erts_proc_sched_data(c_p); ERTS_HLT_ASSERT(esdp == erts_get_scheduler_data()); } @@ -2138,7 +2138,7 @@ access_bif_timer(Process *c_p, Eterm tref, int cancel, int async, int info) goto no_timer; } - esdp = ERTS_PROC_GET_SCHDATA(c_p); + esdp = erts_proc_sched_data(c_p); trefn = internal_ref_numbers(tref); sid = erts_get_ref_numbers_thr_id(trefn); @@ -2363,7 +2363,7 @@ typedef struct { int erts_cancel_bif_timers(Process *p, ErtsBifTimers *btm, void **vyspp) { - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(p); + ErtsSchedulerData *esdp = erts_proc_sched_data(p); ErtsBifTimerYieldState ys = {btm, {ERTS_RBT_YIELD_STAT_INITER}}; ErtsBifTimerYieldState *ysp; int res; @@ -2409,7 +2409,7 @@ detach_bif_timer(ErtsHLTimer *tmr, void *vesdp) int erts_detach_accessor_bif_timers(Process *p, ErtsBifTimers *btm, void **vyspp) { - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(p); + ErtsSchedulerData *esdp = erts_proc_sched_data(p); ErtsBifTimerYieldState ys = {btm, {ERTS_RBT_YIELD_STAT_INITER}}; ErtsBifTimerYieldState *ysp; int res; @@ -2516,7 +2516,7 @@ BIF_RETTYPE send_after_3(BIF_ALIST_3) ErtsMonotonicTime timeout_pos; int short_time, tres; - tres = parse_timeout_pos(ERTS_PROC_GET_SCHDATA(BIF_P), BIF_ARG_1, NULL, + tres = parse_timeout_pos(erts_proc_sched_data(BIF_P), BIF_ARG_1, NULL, 0, &timeout_pos, &short_time); if (tres != 0) BIF_ERROR(BIF_P, BADARG); @@ -2534,7 +2534,7 @@ BIF_RETTYPE send_after_4(BIF_ALIST_4) if (!parse_bif_timer_options(BIF_ARG_4, NULL, NULL, &abs, &accessor)) BIF_ERROR(BIF_P, BADARG); - tres = parse_timeout_pos(ERTS_PROC_GET_SCHDATA(BIF_P), BIF_ARG_1, NULL, + tres = parse_timeout_pos(erts_proc_sched_data(BIF_P), BIF_ARG_1, NULL, abs, &timeout_pos, &short_time); if (tres != 0) BIF_ERROR(BIF_P, BADARG); @@ -2548,7 +2548,7 @@ BIF_RETTYPE start_timer_3(BIF_ALIST_3) ErtsMonotonicTime timeout_pos; int short_time, tres; - tres = parse_timeout_pos(ERTS_PROC_GET_SCHDATA(BIF_P), BIF_ARG_1, NULL, + tres = parse_timeout_pos(erts_proc_sched_data(BIF_P), BIF_ARG_1, NULL, 0, &timeout_pos, &short_time); if (tres != 0) BIF_ERROR(BIF_P, BADARG); @@ -2566,7 +2566,7 @@ BIF_RETTYPE start_timer_4(BIF_ALIST_4) if (!parse_bif_timer_options(BIF_ARG_4, NULL, NULL, &abs, &accessor)) BIF_ERROR(BIF_P, BADARG); - tres = parse_timeout_pos(ERTS_PROC_GET_SCHDATA(BIF_P), BIF_ARG_1, NULL, + tres = parse_timeout_pos(erts_proc_sched_data(BIF_P), BIF_ARG_1, NULL, abs, &timeout_pos, &short_time); if (tres != 0) BIF_ERROR(BIF_P, BADARG); @@ -2720,7 +2720,7 @@ set_proc_timer_common(Process *c_p, ErtsSchedulerData *esdp, Sint64 tmo, int erts_set_proc_timer_term(Process *c_p, Eterm etmo) { - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(c_p); + ErtsSchedulerData *esdp = erts_proc_sched_data(c_p); ErtsMonotonicTime tmo, timeout_pos; int short_time, tres; @@ -2742,7 +2742,7 @@ erts_set_proc_timer_term(Process *c_p, Eterm etmo) void erts_set_proc_timer_uword(Process *c_p, UWord tmo) { - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(c_p); + ErtsSchedulerData *esdp = erts_proc_sched_data(c_p); ERTS_HLT_ASSERT(erts_smp_atomic_read_nob(&c_p->common.timer) == ERTS_PTMR_NONE); @@ -2776,7 +2776,7 @@ erts_cancel_proc_timer(Process *c_p) erts_smp_atomic_set_nob(&c_p->common.timer, ERTS_PTMR_NONE); return; } - continue_cancel_ptimer(ERTS_PROC_GET_SCHDATA(c_p), + continue_cancel_ptimer(erts_proc_sched_data(c_p), (ErtsTimer *) tval); } diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index fcd2739ac3..0649fb68de 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -164,6 +164,8 @@ int erts_use_sender_punish; Uint display_items; /* no of items to display in traces etc */ int H_MIN_SIZE; /* The minimum heap grain */ int BIN_VH_MIN_SIZE; /* The minimum binary virtual*/ +int H_MAX_SIZE; /* The maximum heap size */ +int H_MAX_FLAGS; /* The maximum heap flags */ Uint32 erts_debug_flags; /* Debug flags. */ int erts_backtrace_depth; /* How many functions to show in a backtrace @@ -576,6 +578,10 @@ void erts_usage(void) H_DEFAULT_SIZE); erts_fprintf(stderr, "-hmbs size set minimum binary virtual heap size in words (default %d)\n", VH_DEFAULT_SIZE); + erts_fprintf(stderr, "-hmax size set maximum heap size in words (default %d)\n", + H_DEFAULT_MAX_SIZE); + erts_fprintf(stderr, "-hmaxk bool enable or disable kill at max heap size (default true)\n"); + erts_fprintf(stderr, "-hmaxel bool enable or disable error_logger report at max heap size (default true)\n"); erts_fprintf(stderr, "-hpds size initial process dictionary size (default %d)\n", erts_pd_initial_size); erts_fprintf(stderr, "-hmqd val set default message queue data flag for processes,\n"); @@ -759,6 +765,8 @@ early_init(int *argc, char **argv) /* erts_async_thread_suggested_stack_size = ERTS_ASYNC_THREAD_MIN_STACK_SIZE; H_MIN_SIZE = H_DEFAULT_SIZE; BIN_VH_MIN_SIZE = VH_DEFAULT_SIZE; + H_MAX_SIZE = H_DEFAULT_MAX_SIZE; + H_MAX_FLAGS = MAX_HEAP_SIZE_KILL|MAX_HEAP_SIZE_LOG; erts_initialized = 0; @@ -1484,10 +1492,13 @@ erl_start(int argc, char **argv) char *sub_param = argv[i]+2; /* set default heap size * - * h|ms - min_heap_size - * h|mbs - min_bin_vheap_size - * h|pds - erts_pd_initial_size - * h|mqd - message_queue_data + * h|ms - min_heap_size + * h|mbs - min_bin_vheap_size + * h|pds - erts_pd_initial_size + * h|mqd - message_queue_data + * h|max - max_heap_size + * h|maxk - max_heap_kill + * h|maxel - max_heap_error_logger * */ if (has_prefix("mbs", sub_param)) { @@ -1530,6 +1541,41 @@ erl_start(int argc, char **argv) "Invalid message_queue_data flag: %s\n", arg); erts_usage(); } + } else if (has_prefix("maxk", sub_param)) { + arg = get_arg(sub_param+4, argv[i+1], &i); + if (strcmp(arg,"true") == 0) { + H_MAX_FLAGS |= MAX_HEAP_SIZE_KILL; + } else if (strcmp(arg,"false") == 0) { + H_MAX_FLAGS &= ~MAX_HEAP_SIZE_KILL; + } else { + erts_fprintf(stderr, "bad max heap kill %s\n", arg); + erts_usage(); + } + VERBOSE(DEBUG_SYSTEM, ("using max heap kill %d\n", H_MAX_FLAGS)); + } else if (has_prefix("maxel", sub_param)) { + arg = get_arg(sub_param+5, argv[i+1], &i); + if (strcmp(arg,"true") == 0) { + H_MAX_FLAGS |= MAX_HEAP_SIZE_LOG; + } else if (strcmp(arg,"false") == 0) { + H_MAX_FLAGS &= ~MAX_HEAP_SIZE_LOG; + } else { + erts_fprintf(stderr, "bad max heap error logger %s\n", arg); + erts_usage(); + } + VERBOSE(DEBUG_SYSTEM, ("using max heap log %d\n", H_MAX_FLAGS)); + } else if (has_prefix("max", sub_param)) { + arg = get_arg(sub_param+3, argv[i+1], &i); + if ((H_MAX_SIZE = atoi(arg)) < 0) { + erts_fprintf(stderr, "bad max heap size %s\n", arg); + erts_usage(); + } + if (H_MAX_SIZE < H_MIN_SIZE && H_MAX_SIZE) { + erts_fprintf(stderr, "max heap size (%s) is not allowed to be " + "smaller than min heap size (%d)\n", + arg, H_MIN_SIZE); + erts_usage(); + } + VERBOSE(DEBUG_SYSTEM, ("using max heap size %d\n", H_MAX_SIZE)); } else { /* backward compatibility */ arg = get_arg(argv[i]+2, argv[i+1], &i); diff --git a/erts/emulator/beam/erl_message.c b/erts/emulator/beam/erl_message.c index 9bb6e40a11..579f6e427d 100644 --- a/erts/emulator/beam/erl_message.c +++ b/erts/emulator/beam/erl_message.c @@ -492,7 +492,6 @@ queue_messages(Process* receiver, erts_proc_notify_new_message(receiver, receiver_locks); #else erts_proc_notify_new_message(receiver, 0); - ERTS_HOLE_CHECK(receiver); #endif return res; } @@ -601,7 +600,9 @@ erts_try_alloc_message_on_heap(Process *pp, ASSERT(!(*psp & ERTS_PSFLG_OFF_HEAP_MSGQ)); - if ( + if ((*psp) & ERTS_PSFLGS_VOLATILE_HEAP) + goto in_message_fragment; + else if ( #if defined(ERTS_SMP) *plp & ERTS_PROC_LOCK_MAIN #else @@ -611,7 +612,7 @@ erts_try_alloc_message_on_heap(Process *pp, #ifdef ERTS_SMP try_on_heap: #endif - if ((*psp & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT)) + if (((*psp) & ERTS_PSFLGS_VOLATILE_HEAP) || (pp->flags & F_DISABLE_GC) || HEAP_LIMIT(pp) - HEAP_TOP(pp) <= sz) { /* diff --git a/erts/emulator/beam/erl_message.h b/erts/emulator/beam/erl_message.h index 851ac37fda..4493df1c1a 100644 --- a/erts/emulator/beam/erl_message.h +++ b/erts/emulator/beam/erl_message.h @@ -112,8 +112,8 @@ typedef struct erl_heap_fragment ErlHeapFragment; struct erl_heap_fragment { ErlHeapFragment* next; /* Next heap fragment */ ErlOffHeap off_heap; /* Offset heap data. */ - unsigned alloc_size; /* Size in (half)words of mem */ - unsigned used_size; /* With terms to be moved to heap by GC */ + Uint alloc_size; /* Size in (half)words of mem */ + Uint used_size; /* With terms to be moved to heap by GC */ Eterm mem[1]; /* Data */ }; diff --git a/erts/emulator/beam/erl_msacc.c b/erts/emulator/beam/erl_msacc.c index 0e625f213b..544bc8b983 100644 --- a/erts/emulator/beam/erl_msacc.c +++ b/erts/emulator/beam/erl_msacc.c @@ -338,7 +338,7 @@ erts_msacc_request(Process *c_p, int action, Eterm *threads) { #ifdef ERTS_ENABLE_MSACC ErtsMsAcc *msacc = ERTS_MSACC_TSD_GET(); - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(c_p); + ErtsSchedulerData *esdp = erts_proc_sched_data(c_p); Eterm ref; ErtsMSAccReq *msaccrp; Eterm *hp; diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index a695a028ba..606b73c7b5 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -20,6 +20,23 @@ /* Erlang Native InterFace */ +/* + * Environment contains a pointer to currently executing process. + * In the dirty case this pointer do however not point to the + * actual process structure of the executing process, but instead + * a "shadow process structure". This in order to be able to handle + * heap allocation without the need to acquire the main lock on + * the process. + * + * The dirty process is allowed to allocate on the heap without + * the main lock, i.e., incrementing htop, but is not allowed to + * modify mbuf, offheap, etc without the main lock. The dirty + * process moves mbuf list and offheap list of the shadow process + * structure into the real structure when the dirty nif call + * completes. + */ + + #ifdef HAVE_CONFIG_H # include "config.h" #endif @@ -79,9 +96,46 @@ void dtrace_nifenv_str(ErlNifEnv *, char *); #endif #define MIN_HEAP_FRAG_SZ 200 -static Eterm* alloc_heap_heavy(ErlNifEnv* env, unsigned need, Eterm* hp); +static Eterm* alloc_heap_heavy(ErlNifEnv* env, size_t need, Eterm* hp); + +static ERTS_INLINE int +is_scheduler(void) +{ + ErtsSchedulerData *esdp = erts_get_scheduler_data(); + if (!esdp) + return 0; + if (ERTS_SCHEDULER_IS_DIRTY(esdp)) + return -1; + return 1; +} + +static ERTS_INLINE void +execution_state(ErlNifEnv *env, Process **c_pp, int *schedp) +{ + if (schedp) + *schedp = is_scheduler(); + if (c_pp) { + if (!env || env->proc->common.id == ERTS_INVALID_PID) + *c_pp = NULL; + else { + Process *c_p = env->proc; + + if (!(c_p->static_flags & ERTS_STC_FLG_SHADOW_PROC)) + ASSERT(is_scheduler() > 0); + else { + c_p = env->proc->next; + ASSERT(is_scheduler() < 0); + ASSERT(c_p && env->proc->common.id == c_p->common.id); + } + + *c_pp = c_p; -static ERTS_INLINE Eterm* alloc_heap(ErlNifEnv* env, unsigned need) + ASSERT(!(c_p->static_flags & ERTS_STC_FLG_SHADOW_PROC)); + } + } +} + +static ERTS_INLINE Eterm* alloc_heap(ErlNifEnv* env, size_t need) { Eterm* hp = env->hp; env->hp += need; @@ -91,7 +145,7 @@ static ERTS_INLINE Eterm* alloc_heap(ErlNifEnv* env, unsigned need) return alloc_heap_heavy(env, need, hp); } -static Eterm* alloc_heap_heavy(ErlNifEnv* env, unsigned need, Eterm* hp) +static Eterm* alloc_heap_heavy(ErlNifEnv* env, size_t need, Eterm* hp) { env->hp = hp; if (env->heap_frag == NULL) { @@ -112,7 +166,7 @@ static Eterm* alloc_heap_heavy(ErlNifEnv* env, unsigned need, Eterm* hp) } #if SIZEOF_LONG != ERTS_SIZEOF_ETERM -static ERTS_INLINE void ensure_heap(ErlNifEnv* env, unsigned may_need) +static ERTS_INLINE void ensure_heap(ErlNifEnv* env, size_t may_need) { if (env->hp + may_need > env->hp_end) { alloc_heap_heavy(env, may_need, env->hp); @@ -124,6 +178,9 @@ static ERTS_INLINE void ensure_heap(ErlNifEnv* env, unsigned may_need) void erts_pre_nif(ErlNifEnv* env, Process* p, struct erl_module_nif* mod_nif, Process* tracee) { +#ifdef ERTS_DIRTY_SCHEDULERS + ErtsSchedulerData *esdp; +#endif env->mod_nif = mod_nif; env->proc = p; env->hp = HEAP_TOP(p); @@ -133,6 +190,61 @@ void erts_pre_nif(ErlNifEnv* env, Process* p, struct erl_module_nif* mod_nif, env->tmp_obj_list = NULL; env->exception_thrown = 0; env->tracee = tracee; + + ASSERT(p->common.id != ERTS_INVALID_PID); + +#ifdef ERTS_DIRTY_SCHEDULERS + esdp = erts_get_scheduler_data(); + ASSERT(esdp); + + if (!ERTS_SCHEDULER_IS_DIRTY(esdp)) { +#ifdef DEBUG + erts_aint32_t state = erts_smp_atomic32_read_nob(&p->state); + + ASSERT(p->scheduler_data == esdp); + ASSERT((state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS)) + && !(state & (ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS))); +#endif + + } + else { + Process *sproc; +#ifdef DEBUG + erts_aint32_t state = erts_smp_atomic32_read_nob(&p->state); + + ASSERT(!p->scheduler_data); + ASSERT((state & ERTS_PSFLG_DIRTY_RUNNING) + && !(state & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS))); +#endif + + sproc = esdp->dirty_shadow_process; + ASSERT(sproc); + ASSERT(sproc->static_flags & ERTS_STC_FLG_SHADOW_PROC); + ASSERT(erts_smp_atomic32_read_nob(&sproc->state) + == (ERTS_PSFLG_ACTIVE + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_PROXY)); + + sproc->next = p; + sproc->common.id = p->common.id; + sproc->htop = p->htop; + sproc->stop = p->stop; + sproc->hend = p->hend; + sproc->heap = p->heap; + sproc->abandoned_heap = p->abandoned_heap; + sproc->heap_sz = p->heap_sz; + sproc->high_water = p->high_water; + sproc->old_hend = p->old_hend; + sproc->old_htop = p->old_htop; + sproc->old_heap = p->old_heap; + sproc->mbuf = NULL; + sproc->mbuf_sz = 0; + ERTS_INIT_OFF_HEAP(&sproc->off_heap); + env->proc = sproc; + } +#endif } /* Temporary object header, auto-deallocated when NIF returns @@ -157,18 +269,75 @@ static ERTS_INLINE void free_tmp_objs(ErlNifEnv* env) void erts_post_nif(ErlNifEnv* env) { erts_unblock_fpe(env->fpe_was_unmasked); - if (env->heap_frag == NULL) { - ASSERT(env->hp_end == HEAP_LIMIT(env->proc)); - ASSERT(env->hp >= HEAP_TOP(env->proc)); - ASSERT(env->hp <= HEAP_LIMIT(env->proc)); - HEAP_TOP(env->proc) = env->hp; + +#ifdef ERTS_DIRTY_SCHEDULERS + if (!(env->proc->static_flags & ERTS_STC_FLG_SHADOW_PROC)) +#endif + { + ASSERT(is_scheduler() > 0); + if (env->heap_frag == NULL) { + ASSERT(env->hp_end == HEAP_LIMIT(env->proc)); + ASSERT(env->hp >= HEAP_TOP(env->proc)); + ASSERT(env->hp <= HEAP_LIMIT(env->proc)); + HEAP_TOP(env->proc) = env->hp; + } + else { + ASSERT(env->hp_end != HEAP_LIMIT(env->proc)); + ASSERT(env->hp_end - env->hp <= env->heap_frag->alloc_size); + env->heap_frag->used_size = env->hp - env->heap_frag->mem; + ASSERT(env->heap_frag->used_size <= env->heap_frag->alloc_size); + } + env->exiting = ERTS_PROC_IS_EXITING(env->proc); } - else { - ASSERT(env->hp_end != HEAP_LIMIT(env->proc)); - ASSERT(env->hp_end - env->hp <= env->heap_frag->alloc_size); - env->heap_frag->used_size = env->hp - env->heap_frag->mem; - ASSERT(env->heap_frag->used_size <= env->heap_frag->alloc_size); +#ifdef ERTS_DIRTY_SCHEDULERS + else { /* Dirty nif call using shadow process struct */ + Process *c_p = env->proc->next; + + ASSERT(is_scheduler() < 0); + ASSERT(env->proc->common.id == c_p->common.id); + + if (!env->heap_frag) { + ASSERT(env->hp_end == HEAP_LIMIT(c_p)); + ASSERT(env->hp >= HEAP_TOP(c_p)); + ASSERT(env->hp <= HEAP_LIMIT(c_p)); + HEAP_TOP(c_p) = env->hp; + } + else { + ASSERT(env->hp_end != HEAP_LIMIT(c_p)); + ASSERT(env->hp_end - env->hp <= env->heap_frag->alloc_size); + + HEAP_TOP(c_p) = HEAP_TOP(env->proc); + env->heap_frag->used_size = env->hp - env->heap_frag->mem; + + ASSERT(env->heap_frag->used_size <= env->heap_frag->alloc_size); + + if (c_p->mbuf) { + ErlHeapFragment *bp; + for (bp = env->proc->mbuf; bp->next; bp = bp->next) + ; + bp->next = c_p->mbuf; + } + + c_p->mbuf = env->proc->mbuf; + c_p->mbuf_sz += env->proc->mbuf_sz; + + } + + if (!c_p->off_heap.first) + c_p->off_heap.first = env->proc->off_heap.first; + else if (env->proc->off_heap.first) { + struct erl_off_heap_header *ohhp; + for (ohhp = env->proc->off_heap.first; ohhp->next; ohhp = ohhp->next) + ; + ohhp->next = c_p->off_heap.first; + c_p->off_heap.first = env->proc->off_heap.first; + } + c_p->off_heap.overhead += env->proc->off_heap.overhead; + + env->exiting = ERTS_PROC_IS_EXITING(c_p); + BUMP_ALL_REDS(c_p); } +#endif free_tmp_objs(env); } @@ -400,9 +569,8 @@ error: #endif -int -enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, - ErlNifEnv* msg_env, ERL_NIF_TERM msg) +int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, + ErlNifEnv* msg_env, ERL_NIF_TERM msg) { struct enif_msg_environment_t* menv = (struct enif_msg_environment_t*)msg_env; ErtsProcLocks rp_locks = 0; @@ -413,34 +581,32 @@ enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, Process* c_p; ErtsMessage *mp; Eterm receiver = to_pid->pid; - int flush_me = 0; - ErtsSchedulerData *esdp = erts_get_scheduler_data(); - int scheduler = esdp ? esdp->no : 0; + int scheduler; - if (env != NULL) { - c_p = env->proc; - if (receiver == c_p->common.id) { + execution_state(env, &c_p, &scheduler); + +#ifndef ERTS_SMP + if (!scheduler) { + erts_exit(ERTS_ABORT_EXIT, + "enif_send: called from non-scheduler thread on non-SMP VM"); + return 0; + } +#endif + + if (scheduler > 0) { /* Normal scheduler */ + rp = erts_proc_lookup(receiver); + if (c_p == rp) rp_locks = ERTS_PROC_LOCK_MAIN; - flush_me = 1; - } } else { -#ifdef ERTS_SMP - c_p = NULL; -#else - erts_exit(ERTS_ABORT_EXIT,"enif_send: env==NULL on non-SMP VM"); -#endif + if (c_p && ERTS_PROC_IS_EXITING(c_p)) + return 0; + rp = erts_pid2proc_opt(c_p, 0, receiver, rp_locks, + ERTS_P2P_FLG_INC_REFC); } - - rp = (scheduler - ? erts_proc_lookup(receiver) - : erts_pid2proc_opt(c_p, ERTS_PROC_LOCK_MAIN, - receiver, rp_locks, ERTS_P2P_FLG_INC_REFC)); - - if (rp == NULL) { - ASSERT(env == NULL || receiver != c_p->common.id); + if (rp == NULL) return 0; - } + if (menv) { flush_env(msg_env); mp = erts_alloc_message(0, NULL); @@ -465,10 +631,6 @@ enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, ERL_MESSAGE_TERM(mp) = msg; - if (flush_me) { - flush_env(env); /* Needed for ERTS_HOLE_CHECK */ - } - if (!env || !env->tracee) { if (c_p && IS_TRACED_FL(c_p, F_TRACE_SEND)) @@ -505,6 +667,9 @@ enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, erts_smp_proc_trylock(rp, ERTS_PROC_LOCK_MSGQ) == EBUSY) { if (!msgq) { +#ifdef ERTS_SMP + ErtsThrPrgrDelayHandle dhndl; +#endif msgq = erts_alloc(ERTS_ALC_T_TRACE_MSG_QUEUE, sizeof(ErlTraceMessageQueue)); @@ -519,8 +684,15 @@ enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, erts_smp_proc_unlock(t_p, ERTS_PROC_LOCK_TRACE); +#ifdef ERTS_SMP + if (!scheduler) + dhndl = erts_thr_progress_unmanaged_delay(); +#endif erts_schedule_flush_trace_messages(t_p->common.id); - +#ifdef ERTS_SMP + if (!scheduler) + erts_thr_progress_unmanaged_continue(dhndl); +#endif } else { msgq->len++; *msgq->last = mp; @@ -546,11 +718,9 @@ done: if (rp_locks & ~lc_locks) erts_smp_proc_unlock(rp, rp_locks & ~lc_locks); #endif - if (!scheduler) + if (scheduler <= 0) erts_proc_dec_refc(rp); - if (flush_me) { - cache_env(env); - } + return 1; } @@ -558,26 +728,52 @@ int enif_port_command(ErlNifEnv *env, const ErlNifPort* to_port, ErlNifEnv *msg_env, ERL_NIF_TERM msg) { - - ErtsSchedulerData *esdp = erts_get_scheduler_data(); - int scheduler = esdp ? esdp->no : 0; + int iflags = (erts_port_synchronous_ops + ? ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP + : ERTS_PORT_SFLGS_INVALID_LOOKUP); + int scheduler; + Process *c_p; Port *prt; + int res; - if (scheduler == 0 || !env) - return 0; + if (!env) + erts_exit(ERTS_ABORT_EXIT, "enif_port_command: env == NULL"); - prt = erts_port_lookup(to_port->port_id, - (erts_port_synchronous_ops - ? ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP - : ERTS_PORT_SFLGS_INVALID_LOOKUP)); + execution_state(env, &c_p, &scheduler); + + if (!c_p) + c_p = env->proc; + + if (scheduler > 0) + prt = erts_port_lookup(to_port->port_id, iflags); +#ifdef ERTS_DIRTY_SCHEDULERS + else if (scheduler < 0) { + if (ERTS_PROC_IS_EXITING(c_p)) + return 0; + prt = erts_thr_port_lookup(to_port->port_id, iflags); + } +#endif + else { + erts_exit(ERTS_ABORT_EXIT, "enif_port_command: " + "called from non-scheduler thread"); + } if (!prt) - return 0; + res = 0; + else { - if (IS_TRACED_FL(prt, F_TRACE_RECEIVE)) - trace_port_receive(prt, env->proc->common.id, am_command, msg); + if (IS_TRACED_FL(prt, F_TRACE_RECEIVE)) + trace_port_receive(prt, c_p->common.id, am_command, msg); - return erts_port_output_async(prt, env->proc->common.id, msg); + res = erts_port_output_async(prt, c_p->common.id, msg); + } + +#ifdef ERTS_DIRTY_SCHEDULERS + if (scheduler < 0) + erts_port_dec_refc(prt); +#endif + + return res; } ERL_NIF_TERM enif_make_copy(ErlNifEnv* dst_env, ERL_NIF_TERM src_term) @@ -1011,7 +1207,7 @@ Eterm enif_make_sub_binary(ErlNifEnv* env, ERL_NIF_TERM bin_term, Eterm orig; Uint offset, bit_offset, bit_size; #ifdef DEBUG - unsigned src_size; + size_t src_size; ASSERT(is_binary(bin_term)); src_size = binary_size(bin_term); @@ -1039,15 +1235,21 @@ Eterm enif_make_badarg(ErlNifEnv* env) Eterm enif_raise_exception(ErlNifEnv* env, ERL_NIF_TERM reason) { + Process *c_p; + + execution_state(env, &c_p, NULL); + env->exception_thrown = 1; - env->proc->fvalue = reason; - BIF_ERROR(env->proc, EXC_ERROR); + c_p->fvalue = reason; + BIF_ERROR(c_p, EXC_ERROR); } int enif_has_pending_exception(ErlNifEnv* env, ERL_NIF_TERM* reason) { if (env->exception_thrown && reason != NULL) { - *reason = env->proc->fvalue; + Process *c_p; + execution_state(env, &c_p, NULL); + *reason = c_p->fvalue; } return env->exception_thrown; } @@ -1441,56 +1643,73 @@ int enif_make_reverse_list(ErlNifEnv* env, ERL_NIF_TERM term, ERL_NIF_TERM *list return 1; } +int enif_is_current_process_alive(ErlNifEnv* env) +{ + Process *c_p; + int scheduler; + + execution_state(env, &c_p, &scheduler); + + if (!c_p) + erts_exit(ERTS_ABORT_EXIT, + "enif_is_current_process_alive: " + "Invalid environment"); + + if (!scheduler) + erts_exit(ERTS_ABORT_EXIT, "enif_is_current_process_alive: " + "called from non-scheduler thread"); + + return !ERTS_PROC_IS_EXITING(c_p); +} + int enif_is_process_alive(ErlNifEnv* env, ErlNifPid *proc) { - ErtsProcLocks rp_locks = 0; /* We don't need any locks, - just to check if it is alive */ - Eterm target = proc->pid; - Process* rp; - Process* c_p; - int scheduler = erts_get_scheduler_id() != 0; + int scheduler; - if (env != NULL) { - c_p = env->proc; - if (target == c_p->common.id) { - /* We are alive! */ - return 1; - } - } + execution_state(env, NULL, &scheduler); + + if (scheduler > 0) + return !!erts_proc_lookup(proc->pid); else { #ifdef ERTS_SMP - c_p = NULL; + Process* rp = erts_pid2proc_opt(NULL, 0, proc->pid, 0, + ERTS_P2P_FLG_INC_REFC); + if (rp) + erts_proc_dec_refc(rp); + return !!rp; #else - erts_exit(ERTS_ABORT_EXIT,"enif_is_process_alive: " - "env==NULL on non-SMP VM"); -#endif - } - - rp = (scheduler - ? erts_proc_lookup(target) - : erts_pid2proc_opt(c_p, ERTS_PROC_LOCK_MAIN, - target, rp_locks, ERTS_P2P_FLG_INC_REFC)); - if (rp == NULL) { - ASSERT(env == NULL || target != c_p->common.id); + erts_exit(ERTS_ABORT_EXIT, "enif_is_process_alive: " + "called from non-scheduler thread " + "in non-smp emulator"); return 0; - } else { - if (!scheduler) - erts_proc_dec_refc(rp); - return 1; +#endif } } int enif_is_port_alive(ErlNifEnv *env, ErlNifPort *port) { - /* only allowed if called from scheduler */ - if (erts_get_scheduler_id() == 0) - erts_exit(ERTS_ABORT_EXIT,"enif_is_port_alive: called from non-scheduler"); + int scheduler; + Uint32 iflags = (erts_port_synchronous_ops + ? ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP + : ERTS_PORT_SFLGS_INVALID_LOOKUP); + + execution_state(env, NULL, &scheduler); - return erts_port_lookup( - port->port_id, - (erts_port_synchronous_ops - ? ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP - : ERTS_PORT_SFLGS_INVALID_LOOKUP)) != NULL; + if (scheduler > 0) + return !!erts_port_lookup(port->port_id, iflags); + else { +#ifdef ERTS_SMP + Port *prt = erts_thr_port_lookup(port->port_id, iflags); + if (prt) + erts_port_dec_refc(prt); + return !!prt; +#else + erts_exit(ERTS_ABORT_EXIT, "enif_is_port_alive: " + "called from non-scheduler thread " + "in non-smp emulator"); + return 0; +#endif + } } ERL_NIF_TERM @@ -1603,6 +1822,16 @@ int enif_fprintf(void* filep, const char* format, ...) return ret; } +int enif_snprintf(char *buffer, size_t size, const char* format, ...) +{ + int ret; + va_list arglist; + va_start(arglist, format); + ret = erts_vsnprintf(buffer, size, format, arglist); + va_end(arglist); + return ret; +} + /*********************************************************** ** Memory managed (GC'ed) "resource" objects ** ***********************************************************/ @@ -1957,16 +2186,19 @@ void* enif_dlsym(void* handle, const char* symbol, int enif_consume_timeslice(ErlNifEnv* env, int percent) { + Process *proc; Sint reds; + execution_state(env, &proc, NULL); + ASSERT(is_proc_bound(env) && percent >= 1 && percent <= 100); if (percent < 1) percent = 1; else if (percent > 100) percent = 100; reds = ((CONTEXT_REDS+99) / 100) * percent; ASSERT(reds > 0 && reds <= CONTEXT_REDS); - BUMP_REDS(env->proc, reds); - return ERTS_BIF_REDS_LEFT(env->proc) == 0; + BUMP_REDS(proc, reds); + return ERTS_BIF_REDS_LEFT(proc) == 0; } /* @@ -2061,10 +2293,19 @@ static ERL_NIF_TERM init_nif_sched_data(ErlNifEnv* env, NativeFunPtr direct_fp, NativeFunPtr indirect_fp, int need_save, int argc, const ERL_NIF_TERM argv[]) { - Process* proc = env->proc; - Eterm* reg = ERTS_PROC_GET_SCHDATA(proc)->x_reg_array; + Process* proc; + Eterm* reg; NifExport* ep; - int i; + int i, scheduler; + + execution_state(env, &proc, &scheduler); + + ASSERT(scheduler); + + ERTS_SMP_LC_ASSERT(erts_proc_lc_my_proc_locks(proc) + & ERTS_PROC_LOCK_MAIN); + + reg = erts_proc_sched_data(proc)->x_reg_array; ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); if (!ep) @@ -2076,12 +2317,13 @@ init_nif_sched_data(ErlNifEnv* env, NativeFunPtr direct_fp, NativeFunPtr indirec } if (env->exception_thrown) { ep->exception_thrown = 1; - ep->rootset[0] = env->proc->fvalue; + ep->rootset[0] = proc->fvalue; } else { ep->exception_thrown = 0; ep->rootset[0] = NIL; } - ERTS_VBUMP_ALL_REDS(proc); + if (scheduler > 0) + ERTS_VBUMP_ALL_REDS(proc); for (i = 0; i < argc; i++) { if (need_save) ep->rootset[i+1] = reg[i]; @@ -2113,7 +2355,12 @@ static void restore_nif_mfa(Process* proc, NifExport* ep, int exception) { int i; - Eterm* reg = ERTS_PROC_GET_SCHDATA(proc)->x_reg_array; + Eterm* reg = erts_proc_sched_data(proc)->x_reg_array; + + ERTS_SMP_LC_ASSERT(!(proc->static_flags + & ERTS_STC_FLG_SHADOW_PROC)); + ERTS_SMP_LC_ASSERT(erts_proc_lc_my_proc_locks(proc) + & ERTS_PROC_LOCK_MAIN); proc->current[0] = ep->saved_mfa[0]; proc->current[1] = ep->saved_mfa[1]; @@ -2138,11 +2385,13 @@ restore_nif_mfa(Process* proc, NifExport* ep, int exception) static ERL_NIF_TERM dirty_nif_finalizer(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { - Process* proc = env->proc; + Process* proc; NifExport* ep; + execution_state(env, &proc, NULL); + ASSERT(argc == 1); - ASSERT(!ERTS_SCHEDULER_IS_DIRTY(env->proc->scheduler_data)); + ASSERT(!ERTS_SCHEDULER_IS_DIRTY(erts_proc_sched_data(proc))); ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); ASSERT(ep); ASSERT(!ep->exception_thrown); @@ -2157,10 +2406,12 @@ dirty_nif_finalizer(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) static ERL_NIF_TERM dirty_nif_exception(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { - Process* proc = env->proc; + Process* proc; NifExport* ep; - ASSERT(!ERTS_SCHEDULER_IS_DIRTY(env->proc->scheduler_data)); + execution_state(env, &proc, NULL); + + ASSERT(!ERTS_SCHEDULER_IS_DIRTY(erts_proc_sched_data(proc))); ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); ASSERT(ep); ASSERT(ep->exception_thrown); @@ -2177,23 +2428,32 @@ dirty_nif_exception(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) static ERL_NIF_TERM execute_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { - Process* proc = env->proc; - NativeFunPtr fp = (NativeFunPtr) proc->current[6]; + Process* proc; + NativeFunPtr fp; NifExport* ep; ERL_NIF_TERM result; - ASSERT(ERTS_SCHEDULER_IS_DIRTY(env->proc->scheduler_data)); + execution_state(env, &proc, NULL); + + fp = (NativeFunPtr) proc->current[6]; + + ASSERT(ERTS_SCHEDULER_IS_DIRTY(erts_proc_sched_data(proc))); /* * Set ep->fp to NULL before the native call so we know later whether it scheduled another NIF for execution */ ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); - ASSERT(ep); + ASSERT(ep && fp); ep->fp = NULL; erts_smp_atomic32_read_band_mb(&proc->state, ~(ERTS_PSFLG_DIRTY_CPU_PROC | ERTS_PSFLG_DIRTY_IO_PROC)); + + erts_smp_proc_unlock(proc, ERTS_PROC_LOCK_MAIN); + result = (*fp)(env, argc, argv); + erts_smp_proc_lock(proc, ERTS_PROC_LOCK_MAIN); + if (erts_refc_dectest(&env->mod_nif->rt_dtor_cnt, 0) == 0 && env->mod_nif->mod == NULL) close_lib(env->mod_nif); /* @@ -2230,29 +2490,49 @@ execute_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) static ERTS_INLINE ERL_NIF_TERM schedule_dirty_nif(ErlNifEnv* env, int flags, int argc, const ERL_NIF_TERM argv[]) { - erts_aint32_t state, n, a; - Process* proc = env->proc; - NativeFunPtr fp = (NativeFunPtr) proc->current[6]; + ERL_NIF_TERM result; + erts_aint32_t act, dirty_flag; + Process* proc; + NativeFunPtr fp; NifExport* ep; - int need_save; + int need_save, scheduler; + + execution_state(env, &proc, &scheduler); + if (scheduler <= 0) { + ASSERT(scheduler < 0); + erts_smp_proc_lock(proc, ERTS_PROC_LOCK_MAIN); + } + + fp = (NativeFunPtr) proc->current[6]; + + ASSERT(fp); ASSERT(flags==ERL_NIF_DIRTY_JOB_IO_BOUND || flags==ERL_NIF_DIRTY_JOB_CPU_BOUND); - a = erts_smp_atomic32_read_acqb(&proc->state); - while (1) { - n = state = a; + if (flags == ERL_NIF_DIRTY_JOB_CPU_BOUND) + dirty_flag = ERTS_PSFLG_DIRTY_CPU_PROC; + else + dirty_flag = ERTS_PSFLG_DIRTY_IO_PROC; + + act = erts_smp_atomic32_read_bor_nob(&proc->state, dirty_flag); + if (!(act & (ERTS_PSFLG_DIRTY_CPU_PROC|ERTS_PSFLG_DIRTY_IO_PROC))) + erts_refc_inc(&env->mod_nif->rt_dtor_cnt, 1); + else if ((act & (ERTS_PSFLG_DIRTY_CPU_PROC + | ERTS_PSFLG_DIRTY_IO_PROC)) & ~dirty_flag) { + /* clear other flag... */ if (flags == ERL_NIF_DIRTY_JOB_CPU_BOUND) - n |= ERTS_PSFLG_DIRTY_CPU_PROC; + dirty_flag = ERTS_PSFLG_DIRTY_IO_PROC; else - n |= ERTS_PSFLG_DIRTY_IO_PROC; - a = erts_smp_atomic32_cmpxchg_mb(&proc->state, n, state); - if (a == state) - break; + dirty_flag = ERTS_PSFLG_DIRTY_CPU_PROC; + erts_smp_atomic32_read_band_nob(&proc->state, ~dirty_flag); } - erts_refc_inc(&env->mod_nif->rt_dtor_cnt, 1); + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); need_save = (ep == NULL || is_non_value(ep->saved_mfa[0])); - return init_nif_sched_data(env, execute_dirty_nif, fp, need_save, argc, argv); + result = init_nif_sched_data(env, execute_dirty_nif, fp, need_save, argc, argv); + if (scheduler <= 0) + erts_smp_proc_unlock(proc, ERTS_PROC_LOCK_MAIN); + return result; } static ERL_NIF_TERM @@ -2277,11 +2557,14 @@ schedule_dirty_cpu_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) static ERL_NIF_TERM execute_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { - Process* proc = env->proc; - NativeFunPtr fp = (NativeFunPtr) proc->current[6]; + Process* proc; + NativeFunPtr fp; NifExport* ep; ERL_NIF_TERM result; + execution_state(env, &proc, NULL); + fp = (NativeFunPtr) proc->current[6]; + ASSERT(!env->exception_thrown); ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); ASSERT(ep); @@ -2304,10 +2587,10 @@ enif_schedule_nif(ErlNifEnv* env, const char* fun_name, int flags, ERL_NIF_TERM (*fp)(ErlNifEnv*, int, const ERL_NIF_TERM[]), int argc, const ERL_NIF_TERM argv[]) { - Process* proc = env->proc; + Process* proc; NifExport* ep; ERL_NIF_TERM fun_name_atom, result; - int need_save; + int need_save, scheduler; if (argc > MAX_ARG) return enif_make_badarg(env); @@ -2315,6 +2598,13 @@ enif_schedule_nif(ErlNifEnv* env, const char* fun_name, int flags, if (enif_is_exception(env, fun_name_atom)) return fun_name_atom; + execution_state(env, &proc, &scheduler); + if (scheduler <= 0) { + if (scheduler == 0) + enif_make_badarg(env); + erts_smp_proc_lock(proc, ERTS_PROC_LOCK_MAIN); + } + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); need_save = (ep == NULL || is_non_value(ep->saved_mfa[0])); @@ -2326,12 +2616,15 @@ enif_schedule_nif(ErlNifEnv* env, const char* fun_name, int flags, sched_fun = schedule_dirty_io_nif; else if (chkflgs == ERL_NIF_DIRTY_JOB_CPU_BOUND) sched_fun = schedule_dirty_cpu_nif; - else - return enif_make_badarg(env); + else { + result = enif_make_badarg(env); + goto done; + } result = init_nif_sched_data(env, sched_fun, fp, need_save, argc, argv); #else - return enif_make_badarg(env); + result = enif_make_badarg(env); #endif + goto done; } else result = init_nif_sched_data(env, execute_nif, fp, need_save, argc, argv); @@ -2339,18 +2632,28 @@ enif_schedule_nif(ErlNifEnv* env, const char* fun_name, int flags, ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); ASSERT(ep); ep->exp.code[1] = (BeamInstr) fun_name_atom; + +done: + if (scheduler < 0) + erts_smp_proc_unlock(proc, ERTS_PROC_LOCK_MAIN); + return result; } -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT - int enif_is_on_dirty_scheduler(ErlNifEnv* env) { - return ERTS_SCHEDULER_IS_DIRTY(env->proc->scheduler_data); -} + int scheduler; + Process *c_p; -#endif /* ERL_NIF_DIRTY_SCHEDULER_SUPPORT */ + execution_state(env, &c_p, &scheduler); + + if (!c_p || !scheduler) + erts_exit(ERTS_ABORT_EXIT, "enif_is_on_dirty_scheduler: " + "Invalid env"); + + return scheduler < 0; +} /* Maps */ @@ -3051,16 +3354,16 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) (BeamInstr) BeamOp(op_i_generic_breakpoint)); g->orig_instr = (BeamInstr) BeamOp(op_call_nif); } +#ifdef ERTS_DIRTY_SCHEDULERS if ((entry->major > 2 || (entry->major == 2 && entry->minor >= 7)) && (entry->options & ERL_NIF_DIRTY_NIF_OPTION) && f->flags) { -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT code_ptr[5+3] = (BeamInstr) f->fptr; code_ptr[5+1] = (f->flags == ERL_NIF_DIRTY_JOB_IO_BOUND) ? (BeamInstr) schedule_dirty_io_nif : (BeamInstr) schedule_dirty_cpu_nif; -#endif } else +#endif code_ptr[5+1] = (BeamInstr) f->fptr; code_ptr[5+2] = (BeamInstr) lib; f = next_func(entry, &incr, f); diff --git a/erts/emulator/beam/erl_nif.h b/erts/emulator/beam/erl_nif.h index 3964f7f679..da7a754757 100644 --- a/erts/emulator/beam/erl_nif.h +++ b/erts/emulator/beam/erl_nif.h @@ -28,7 +28,6 @@ # include "config.h" #endif -#include "erl_native_features_config.h" #include "erl_drv_nif.h" /* Version history: @@ -50,9 +49,10 @@ ** 2.8: 18.0 add enif_has_pending_exception ** 2.9: 18.2 enif_getenv ** 2.10: Time API +** 2.11: 19.0 enif_snprintf */ #define ERL_NIF_MAJOR_VERSION 2 -#define ERL_NIF_MINOR_VERSION 10 +#define ERL_NIF_MINOR_VERSION 11 /* * The emulator will refuse to load a nif-lib with a major version @@ -167,13 +167,11 @@ typedef int ErlNifTSDKey; typedef ErlDrvThreadOpts ErlNifThreadOpts; -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT typedef enum { - ERL_NIF_DIRTY_JOB_CPU_BOUND = ERL_DRV_DIRTY_JOB_CPU_BOUND, - ERL_NIF_DIRTY_JOB_IO_BOUND = ERL_DRV_DIRTY_JOB_IO_BOUND + ERL_NIF_DIRTY_JOB_CPU_BOUND = ERL_DIRTY_JOB_CPU_BOUND, + ERL_NIF_DIRTY_JOB_IO_BOUND = ERL_DIRTY_JOB_IO_BOUND }ErlNifDirtyTaskFlags; -#endif typedef struct /* All fields all internal and may change */ { @@ -257,11 +255,7 @@ extern TWinDynNifCallbacks WinDynNifCallbacks; # define ERL_NIF_INIT_DECL(MODNAME) ERL_NIF_INIT_EXPORT ErlNifEntry* nif_init(ERL_NIF_INIT_ARGS) #endif -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT -# define ERL_NIF_ENTRY_OPTIONS ERL_NIF_DIRTY_NIF_OPTION -#else -# define ERL_NIF_ENTRY_OPTIONS 0 -#endif +#define ERL_NIF_ENTRY_OPTIONS ERL_NIF_DIRTY_NIF_OPTION #ifdef __cplusplus } diff --git a/erts/emulator/beam/erl_nif_api_funcs.h b/erts/emulator/beam/erl_nif_api_funcs.h index a5acd86551..b211ab4b16 100644 --- a/erts/emulator/beam/erl_nif_api_funcs.h +++ b/erts/emulator/beam/erl_nif_api_funcs.h @@ -166,26 +166,19 @@ ERL_NIF_API_FUNC_DECL(ErlNifTime, enif_convert_time_unit, (ErlNifTime, ErlNifTim ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM, enif_now_time, (ErlNifEnv *env)); ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM, enif_cpu_time, (ErlNifEnv *env)); ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM, enif_make_unique_integer, (ErlNifEnv *env, ErlNifUniqueInteger properties)); +ERL_NIF_API_FUNC_DECL(int, enif_is_current_process_alive, (ErlNifEnv *env)); ERL_NIF_API_FUNC_DECL(int, enif_is_process_alive, (ErlNifEnv *env, ErlNifPid *pid)); ERL_NIF_API_FUNC_DECL(int, enif_is_port_alive, (ErlNifEnv *env, ErlNifPort *port_id)); ERL_NIF_API_FUNC_DECL(int, enif_get_local_port, (ErlNifEnv* env, ERL_NIF_TERM, ErlNifPort* port_id)); ERL_NIF_API_FUNC_DECL(int, enif_term_to_binary, (ErlNifEnv *env, ERL_NIF_TERM term, ErlNifBinary *bin)); ERL_NIF_API_FUNC_DECL(size_t, enif_binary_to_term, (ErlNifEnv *env, const unsigned char* data, size_t sz, ERL_NIF_TERM *term, unsigned int opts)); ERL_NIF_API_FUNC_DECL(int, enif_port_command, (ErlNifEnv *env, const ErlNifPort* to_port, ErlNifEnv *msg_env, ERL_NIF_TERM msg)); +ERL_NIF_API_FUNC_DECL(int,enif_is_on_dirty_scheduler,(ErlNifEnv*)); +ERL_NIF_API_FUNC_DECL(int,enif_snprintf,(char * buffer, size_t size, const char *format, ...)); /* ** ADD NEW ENTRIES HERE (before this comment) !!! */ - - -/* - * Conditional EXPERIMENTAL stuff always last. - * Must be moved up and made unconditional to support binary backward - * compatibility on Windows. - */ -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT -ERL_NIF_API_FUNC_DECL(int,enif_is_on_dirty_scheduler,(ErlNifEnv*)); -#endif #endif /* ERL_NIF_API_FUNC_DECL */ /* @@ -330,12 +323,15 @@ ERL_NIF_API_FUNC_DECL(int,enif_is_on_dirty_scheduler,(ErlNifEnv*)); # define enif_now_time ERL_NIF_API_FUNC_MACRO(enif_now_time) # define enif_cpu_time ERL_NIF_API_FUNC_MACRO(enif_cpu_time) # define enif_make_unique_integer ERL_NIF_API_FUNC_MACRO(enif_make_unique_integer) +# define enif_is_current_process_alive ERL_NIF_API_FUNC_MACRO(enif_is_current_process_alive) # define enif_is_process_alive ERL_NIF_API_FUNC_MACRO(enif_is_process_alive) # define enif_is_port_alive ERL_NIF_API_FUNC_MACRO(enif_is_port_alive) # define enif_get_local_port ERL_NIF_API_FUNC_MACRO(enif_get_local_port) # define enif_term_to_binary ERL_NIF_API_FUNC_MACRO(enif_term_to_binary) # define enif_binary_to_term ERL_NIF_API_FUNC_MACRO(enif_binary_to_term) # define enif_port_command ERL_NIF_API_FUNC_MACRO(enif_port_command) +# define enif_is_on_dirty_scheduler ERL_NIF_API_FUNC_MACRO(enif_is_on_dirty_scheduler) +# define enif_snprintf ERL_NIF_API_FUNC_MACRO(enif_snprintf) /* ** ADD NEW ENTRIES HERE (before this comment) @@ -346,9 +342,6 @@ ERL_NIF_API_FUNC_DECL(int,enif_is_on_dirty_scheduler,(ErlNifEnv*)); * Must be moved up and made unconditional to support binary backward * compatibility on Windows. */ -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT -# define enif_is_on_dirty_scheduler ERL_NIF_API_FUNC_MACRO(enif_is_on_dirty_scheduler) -#endif #endif /* ERL_NIF_API_FUNC_MACRO */ diff --git a/erts/emulator/beam/erl_port.h b/erts/emulator/beam/erl_port.h index c2588e718d..f0075ca2b9 100644 --- a/erts/emulator/beam/erl_port.h +++ b/erts/emulator/beam/erl_port.h @@ -487,6 +487,7 @@ ERTS_GLB_INLINE Port*erts_id2port(Eterm id); ERTS_GLB_INLINE Port *erts_id2port_sflgs(Eterm, Process *, ErtsProcLocks, Uint32); ERTS_GLB_INLINE void erts_port_release(Port *); #ifdef ERTS_SMP +ERTS_GLB_INLINE Port *erts_thr_port_lookup(Eterm id, Uint32 invalid_sflgs); ERTS_GLB_INLINE Port *erts_thr_id2port_sflgs(Eterm id, Uint32 invalid_sflgs); ERTS_GLB_INLINE void erts_thr_port_release(Port *prt); #endif @@ -626,6 +627,44 @@ erts_port_release(Port *prt) } #ifdef ERTS_SMP +/* + * erts_thr_id2port_sflgs() and erts_port_dec_refc(prt) can + * be used by unmanaged threads in the SMP case. + */ +ERTS_GLB_INLINE Port * +erts_thr_port_lookup(Eterm id, Uint32 invalid_sflgs) +{ + Port *prt; + ErtsThrPrgrDelayHandle dhndl; + + if (is_not_internal_port(id)) + return NULL; + + dhndl = erts_thr_progress_unmanaged_delay(); + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + + if (!prt || prt->common.id != id) { + erts_thr_progress_unmanaged_continue(dhndl); + return NULL; + } + else { + erts_aint32_t state; + erts_port_inc_refc(prt); + + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); + + state = erts_atomic32_read_acqb(&prt->state); + if (state & invalid_sflgs) { + erts_port_dec_refc(prt); + return NULL; + } + + return prt; + } +} /* * erts_thr_id2port_sflgs() and erts_thr_port_release() can diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 36b0af0c1a..a853ec585b 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -409,6 +409,10 @@ ErtsAlignedSchedulerData *erts_aligned_scheduler_data; #ifdef ERTS_DIRTY_SCHEDULERS ErtsAlignedSchedulerData *erts_aligned_dirty_cpu_scheduler_data; ErtsAlignedSchedulerData *erts_aligned_dirty_io_scheduler_data; +typedef union { + Process dsp; + char align[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(Process))]; +} ErtsAlignedDirtyShadowProcess; #endif typedef union { @@ -589,6 +593,7 @@ dbg_chk_aux_work_val(erts_aint32_t value) valid |= ERTS_SSI_AUX_WORK_CNCLD_TMRS; valid |= ERTS_SSI_AUX_WORK_CNCLD_TMRS_THR_PRGR; valid |= ERTS_SSI_AUX_WORK_THR_PRGR_LATER_OP; + valid |= ERTS_SSI_AUX_WORK_PENDING_EXITERS; #endif #if HAVE_ERTS_MSEG valid |= ERTS_SSI_AUX_WORK_MSEG_CACHE_CHECK; @@ -611,7 +616,7 @@ dbg_chk_aux_work_val(erts_aint32_t value) #endif #ifdef ERTS_SMP -static void handle_pending_exiters(ErtsProcList *); +static void do_handle_pending_exiters(ErtsProcList *); static void wake_scheduler(ErtsRunQueue *rq); #endif @@ -679,6 +684,8 @@ erts_pre_init_process(void) = "MISC_THR_PRGR"; erts_aux_work_flag_descr[ERTS_SSI_AUX_WORK_MISC_IX] = "MISC"; + erts_aux_work_flag_descr[ERTS_SSI_AUX_WORK_PENDING_EXITERS_IX] + = "PENDING_EXITERS"; erts_aux_work_flag_descr[ERTS_SSI_AUX_WORK_SET_TMO_IX] = "SET_TMO"; erts_aux_work_flag_descr[ERTS_SSI_AUX_WORK_MSEG_CACHE_CHECK_IX] @@ -1156,7 +1163,7 @@ reply_sched_wall_time(void *vswtrp) Eterm erts_sched_wall_time_request(Process *c_p, int set, int enable) { - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(c_p); + ErtsSchedulerData *esdp = erts_proc_sched_data(c_p); Eterm ref; ErtsSchedWallTimeReq *swtrp; Eterm *hp; @@ -1234,7 +1241,7 @@ reply_system_check(void *vscrp) Eterm erts_system_check_request(Process *c_p) { - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(c_p); + ErtsSchedulerData *esdp = erts_proc_sched_data(c_p); Eterm ref; ErtsSystemCheckReq *scrp; Eterm *hp; @@ -2336,6 +2343,30 @@ handle_mseg_cache_check(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiti #endif +#ifdef ERTS_SMP + +static ERTS_INLINE erts_aint32_t +handle_pending_exiters(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) +{ + ErtsProcList *pnd_xtrs; + ErtsRunQueue *rq; + + rq = awdp->esdp->run_queue; + unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_PENDING_EXITERS); + + erts_smp_runq_lock(rq); + pnd_xtrs = rq->procs.pending_exiters; + rq->procs.pending_exiters = NULL; + erts_smp_runq_unlock(rq); + + if (erts_proclist_fetch(&pnd_xtrs, NULL)) + do_handle_pending_exiters(pnd_xtrs); + + return aux_work & ~ERTS_SSI_AUX_WORK_PENDING_EXITERS; +} + +#endif + static ERTS_INLINE erts_aint32_t handle_setup_aux_work_timer(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) { @@ -2427,6 +2458,10 @@ handle_aux_work(ErtsAuxWorkData *awdp, erts_aint32_t orig_aux_work, int waiting) HANDLE_AUX_WORK(ERTS_SSI_AUX_WORK_MISC, handle_misc_aux_work); +#ifdef ERTS_SMP + HANDLE_AUX_WORK(ERTS_SSI_AUX_WORK_PENDING_EXITERS, + handle_pending_exiters); +#endif HANDLE_AUX_WORK(ERTS_SSI_AUX_WORK_SET_TMO, handle_setup_aux_work_timer); @@ -3979,6 +4014,33 @@ schedule_bound_processes(ErtsRunQueue *rq, } } +#ifdef ERTS_DIRTY_SCHEDULERS + +static ERTS_INLINE void +clear_proc_dirty_queue_bit(Process *p, ErtsRunQueue *rq, int prio_bit) +{ +#ifdef DEBUG + erts_aint32_t old; +#endif + erts_aint32_t qb = prio_bit; + if (rq == ERTS_DIRTY_CPU_RUNQ) + qb <<= ERTS_PDSFLGS_IN_CPU_PRQ_MASK_OFFSET; + else { + ASSERT(rq == ERTS_DIRTY_IO_RUNQ); + qb <<= ERTS_PDSFLGS_IN_IO_PRQ_MASK_OFFSET; + } +#ifdef DEBUG + old = (int) +#else + (void) +#endif + erts_smp_atomic32_read_band_mb(&p->dirty_state, ~qb); + ASSERT(old & qb); +} + +#endif /* ERTS_DIRTY_SCHEDULERS */ + + static void evacuate_run_queue(ErtsRunQueue *rq, ErtsStuckBoundProcesses *sbpp) @@ -4141,29 +4203,8 @@ evacuate_run_queue(ErtsRunQueue *rq, } #ifdef ERTS_DIRTY_SCHEDULERS - - if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix)) { - erts_aint32_t dqbit = qbit; -#ifdef DEBUG - erts_aint32_t old_dqbit; -#endif - - if (rq == ERTS_DIRTY_CPU_RUNQ) - dqbit <<= ERTS_PDSFLGS_IN_CPU_PRQ_MASK_OFFSET; - else { - ASSERT(rq == ERTS_DIRTY_IO_RUNQ); - dqbit <<= ERTS_PDSFLGS_IN_IO_PRQ_MASK_OFFSET; - } - -#ifdef DEBUG - old_dqbit = (int) -#else - (void) -#endif - erts_smp_atomic32_read_band_mb(&real_proc->dirty_state, - ~dqbit); - ASSERT(old_dqbit & dqbit); - } + if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix)) + clear_proc_dirty_queue_bit(real_proc, rq, qbit); #endif if (ERTS_PSFLG_BOUND & real_state) { @@ -5653,7 +5694,8 @@ static void init_scheduler_data(ErtsSchedulerData* esdp, int num, ErtsSchedulerSleepInfo* ssi, ErtsRunQueue* runq, - char** daww_ptr, size_t daww_sz) + char** daww_ptr, size_t daww_sz, + Process *shadow_proc) { esdp->timer_wheel = NULL; #ifdef ERTS_SMP @@ -5677,6 +5719,15 @@ init_scheduler_data(ErtsSchedulerData* esdp, int num, esdp->no = (Uint) num; ERTS_DIRTY_SCHEDULER_NO(esdp) = 0; } + esdp->dirty_shadow_process = shadow_proc; + if (shadow_proc) { + erts_init_empty_process(shadow_proc); + erts_smp_atomic32_init_nob(&shadow_proc->state, + (ERTS_PSFLG_ACTIVE + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_PROXY)); + shadow_proc->static_flags = ERTS_STC_FLG_SHADOW_PROC; + } #else esdp->no = (Uint) num; #endif @@ -5928,31 +5979,41 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online for (ix = 0; ix < n; ix++) { ErtsSchedulerData *esdp = ERTS_SCHEDULER_IX(ix); init_scheduler_data(esdp, ix+1, ERTS_SCHED_SLEEP_INFO_IX(ix), - ERTS_RUNQ_IX(ix), &daww_ptr, daww_sz); + ERTS_RUNQ_IX(ix), &daww_ptr, daww_sz, + NULL); } #ifdef ERTS_DIRTY_SCHEDULERS -#ifdef ERTS_SMP - erts_aligned_dirty_cpu_scheduler_data = - erts_alloc_permanent_cache_aligned( - ERTS_ALC_T_SCHDLR_DATA, - no_dirty_cpu_schedulers*sizeof(ErtsAlignedSchedulerData)); - for (ix = 0; ix < no_dirty_cpu_schedulers; ix++) { - ErtsSchedulerData *esdp = ERTS_DIRTY_CPU_SCHEDULER_IX(ix); - init_scheduler_data(esdp, ix+1, ERTS_DIRTY_CPU_SCHED_SLEEP_INFO_IX(ix), - ERTS_DIRTY_CPU_RUNQ, NULL, 0); - } - erts_aligned_dirty_io_scheduler_data = - erts_alloc_permanent_cache_aligned( - ERTS_ALC_T_SCHDLR_DATA, - no_dirty_io_schedulers*sizeof(ErtsAlignedSchedulerData)); - for (ix = 0; ix < no_dirty_io_schedulers; ix++) { - ErtsSchedulerData *esdp = ERTS_DIRTY_IO_SCHEDULER_IX(ix); - init_scheduler_data(esdp, ix+1, ERTS_DIRTY_IO_SCHED_SLEEP_INFO_IX(ix), - ERTS_DIRTY_IO_RUNQ, NULL, 0); + { + int dirty_scheds = no_dirty_cpu_schedulers + no_dirty_io_schedulers; + int adspix = 0; + ErtsAlignedDirtyShadowProcess *adsp = + erts_alloc_permanent_cache_aligned( + ERTS_ALC_T_SCHDLR_DATA, + dirty_scheds * sizeof(ErtsAlignedDirtyShadowProcess)); + + erts_aligned_dirty_cpu_scheduler_data = + erts_alloc_permanent_cache_aligned( + ERTS_ALC_T_SCHDLR_DATA, + dirty_scheds * sizeof(ErtsAlignedSchedulerData)); + + erts_aligned_dirty_io_scheduler_data = + &erts_aligned_dirty_cpu_scheduler_data[no_dirty_cpu_schedulers]; + + for (ix = 0; ix < no_dirty_cpu_schedulers; ix++) { + ErtsSchedulerData *esdp = ERTS_DIRTY_CPU_SCHEDULER_IX(ix); + init_scheduler_data(esdp, ix+1, ERTS_DIRTY_CPU_SCHED_SLEEP_INFO_IX(ix), + ERTS_DIRTY_CPU_RUNQ, NULL, 0, + &adsp[adspix++].dsp); + } + for (ix = 0; ix < no_dirty_io_schedulers; ix++) { + ErtsSchedulerData *esdp = ERTS_DIRTY_IO_SCHEDULER_IX(ix); + init_scheduler_data(esdp, ix+1, ERTS_DIRTY_IO_SCHED_SLEEP_INFO_IX(ix), + ERTS_DIRTY_IO_RUNQ, NULL, 0, + &adsp[adspix++].dsp); + } } #endif -#endif init_misc_aux_work(); init_swtreq_alloc(); @@ -6167,7 +6228,7 @@ check_dirty_enqueue_in_prio_queue(Process *c_p, erts_aint32_t dact, max_qbit; /* Termination should be done on an ordinary scheduler */ - if (actual & ERTS_PSFLG_EXITING) { + if ((*newp) & ERTS_PSFLG_EXITING) { *newp &= ~ERTS_PSFLGS_DIRTY_WORK; return ERTS_ENQUEUE_NORMAL_QUEUE; } @@ -6176,7 +6237,7 @@ check_dirty_enqueue_in_prio_queue(Process *c_p, * If we have system tasks, we enqueue on ordinary run-queue * and take care of those system tasks first. */ - if (actual & ERTS_PSFLG_ACTIVE_SYS) + if ((*newp) & ERTS_PSFLG_ACTIVE_SYS) return ERTS_ENQUEUE_NORMAL_QUEUE; dact = erts_smp_atomic32_read_mb(&c_p->dirty_state); @@ -6356,23 +6417,29 @@ select_enqueue_run_queue(int enqueue, int enq_prio, Process *p, erts_aint32_t st * schedule_out_process() return with c_rq locked. */ static ERTS_INLINE int -schedule_out_process(ErtsRunQueue *c_rq, erts_aint32_t state, Process *p, Process *proxy) +schedule_out_process(ErtsRunQueue *c_rq, erts_aint32_t state, Process *p, + Process *proxy, int is_normal_sched) { - erts_aint32_t a, e, n, enq_prio = -1; + erts_aint32_t a, e, n, enq_prio = -1, running_flgs; int enqueue; /* < 0 -> use proxy */ ErtsRunQueue* runq; + if (is_normal_sched) + running_flgs = ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS; + else + running_flgs = ERTS_PSFLG_DIRTY_RUNNING|ERTS_PSFLG_DIRTY_RUNNING_SYS; + a = state; while (1) { n = e = a; - ASSERT(a & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS)); + ASSERT(a & running_flgs); enqueue = ERTS_ENQUEUE_NOT; - n &= ~(ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS); - if (a & (ERTS_PSFLG_ACTIVE_SYS|ERTS_PSFLG_DIRTY_ACTIVE_SYS) + n &= ~running_flgs; + if ((a & (ERTS_PSFLG_ACTIVE_SYS|ERTS_PSFLG_DIRTY_ACTIVE_SYS)) || (a & (ERTS_PSFLG_ACTIVE|ERTS_PSFLG_SUSPENDED)) == ERTS_PSFLG_ACTIVE) { enqueue = check_enqueue_in_prio_queue(p, &enq_prio, &n, a); } @@ -6485,8 +6552,9 @@ change_proc_schedule_state(Process *p, ErtsProcLocks locks) { /* - * NOTE: ERTS_PSFLG_RUNNING, ERTS_PSFLG_RUNNING_SYS and - * ERTS_PSFLG_ACTIVE_SYS are not allowed to be + * NOTE: ERTS_PSFLG_RUNNING, ERTS_PSFLG_RUNNING_SYS, + * ERTS_PSFLG_DIRTY_RUNNING, ERTS_PSFLG_DIRTY_RUNNING_SYS + * and ERTS_PSFLG_ACTIVE_SYS are not allowed to be * altered by this function! */ erts_aint32_t a = *statep, n; @@ -6500,9 +6568,13 @@ change_proc_schedule_state(Process *p, ASSERT(!(a & ERTS_PSFLG_PROXY)); ASSERT((clear_state_flags & (ERTS_PSFLG_RUNNING | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS | ERTS_PSFLG_ACTIVE_SYS)) == 0); ASSERT((set_state_flags & (ERTS_PSFLG_RUNNING | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS | ERTS_PSFLG_ACTIVE_SYS)) == 0); if (lock_status) @@ -6526,8 +6598,16 @@ change_proc_schedule_state(Process *p, if ((n & (ERTS_PSFLG_SUSPENDED | ERTS_PSFLG_RUNNING | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS | ERTS_PSFLG_IN_RUNQ - | ERTS_PSFLG_ACTIVE)) == ERTS_PSFLG_ACTIVE) { + | ERTS_PSFLG_ACTIVE)) == ERTS_PSFLG_ACTIVE +#ifdef ERTS_DIRTY_SCHEDULERS + || (n & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_EXITING)) == ERTS_PSFLG_EXITING +#endif + ) { /* * Active and seemingly need to be enqueued, but * process may be in a run queue via proxy, need @@ -6551,7 +6631,9 @@ change_proc_schedule_state(Process *p, | ERTS_PSFLG_ACTIVE)) == ERTS_PSFLG_ACTIVE) && (!(a & (ERTS_PSFLG_ACTIVE_SYS | ERTS_PSFLG_RUNNING - | ERTS_PSFLG_RUNNING_SYS) + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS) && (!(a & ERTS_PSFLG_ACTIVE) || (a & ERTS_PSFLG_SUSPENDED))))) { /* We activated a prevously inactive process */ @@ -6693,7 +6775,10 @@ schedule_process_sys_task(Process *p, erts_aint32_t prio, ErtsProcSysTask *st) enqueue = ERTS_ENQUEUE_NOT; n |= ERTS_PSFLG_ACTIVE_SYS; - if (!(a & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS))) + if (!(a & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS))) enqueue = check_enqueue_in_prio_queue(p, &enq_prio, &n, a); a = erts_smp_atomic32_cmpxchg_mb(&p->state, n, e); if (a == e) @@ -6706,7 +6791,9 @@ schedule_process_sys_task(Process *p, erts_aint32_t prio, ErtsProcSysTask *st) if (!(a & (ERTS_PSFLG_ACTIVE_SYS | ERTS_PSFLG_RUNNING - | ERTS_PSFLG_RUNNING_SYS)) + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS)) && (!(a & ERTS_PSFLG_ACTIVE) || (a & ERTS_PSFLG_SUSPENDED))) { /* We activated a prevously inactive process */ profile_runnable_proc(p, am_active); @@ -6746,11 +6833,16 @@ suspend_process(Process *c_p, Process *p) if (c_p == p) { state = erts_smp_atomic32_read_bor_relb(&p->state, ERTS_PSFLG_SUSPENDED); - ASSERT(state & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS)); + ASSERT(state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS)); suspended = (state & ERTS_PSFLG_SUSPENDED) ? -1: 1; } else { - while (!(state & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_EXITING))) { + while (!(state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_EXITING))) { erts_aint32_t n, e; n = e = state; @@ -6776,8 +6868,11 @@ suspend_process(Process *c_p, Process *p) if ((state & (ERTS_PSFLG_ACTIVE | ERTS_PSFLG_ACTIVE_SYS + | ERTS_PSFLG_DIRTY_ACTIVE_SYS | ERTS_PSFLG_RUNNING | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS | ERTS_PSFLG_SUSPENDED)) == ERTS_PSFLG_ACTIVE) { /* We made process inactive */ profile_runnable_proc(p, am_inactive); @@ -7759,8 +7854,10 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int normal plp = proclist_create(p); erts_proclist_store_last(&msbp->blckrs, plp); p->flags |= have_blckd_flg; - ASSERT(schdlr_sspnd.active == ERTS_SCHDLR_SSPND_MAKE_NSCHEDS_VAL(1, 0, 0)); - ASSERT(p->scheduler_data->no == 1); + ASSERT(normal + ? 1 == schdlr_sspnd_get_nscheds(&schdlr_sspnd.active, ERTS_SCHED_NORMAL) + : schdlr_sspnd.active == ERTS_SCHDLR_SSPND_MAKE_NSCHEDS_VAL(1, 0, 0)); + ASSERT(erts_proc_sched_data(p)->no == 1); if (schdlr_sspnd.msb.ongoing) res = ERTS_SCHDLR_SSPND_DONE_MSCHED_BLOCKED; else @@ -7780,7 +7877,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int normal if (schdlr_sspnd.active == ERTS_SCHDLR_SSPND_MAKE_NSCHEDS_VAL(1, 0, 0) || (normal && schdlr_sspnd_get_nscheds(&schdlr_sspnd.active, ERTS_SCHED_NORMAL) == 1)) { - ASSERT(p->scheduler_data->no == 1); + ASSERT(erts_proc_sched_data(p)->no == 1); plp = proclist_create(p); erts_proclist_store_last(&msbp->blckrs, plp); if (schdlr_sspnd.msb.ongoing) @@ -7830,7 +7927,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int normal else res = ERTS_SCHDLR_SSPND_YIELD_DONE_NMSCHED_BLOCKED; } - ASSERT(p->scheduler_data); + ASSERT(erts_proc_sched_data(p)); } } else if (!msbp->ongoing) { @@ -8420,9 +8517,23 @@ pid2proc_not_running(Process *c_p, ErtsProcLocks c_p_locks, if (!suspend_process(c_p, rp)) { /* Other process running */ - ASSERT(ERTS_PSFLG_RUNNING + ASSERT((ERTS_PSFLG_RUNNING | ERTS_PSFLG_DIRTY_RUNNING) & erts_smp_atomic32_read_nob(&rp->state)); +#ifdef ERTS_DIRTY_SCHEDULERS + if (!suspend + && (erts_smp_atomic32_read_nob(&rp->state) + & ERTS_PSFLG_DIRTY_RUNNING)) { + ErtsProcLocks need_locks = pid_locks & ~ERTS_PROC_LOCK_STATUS; + if (need_locks && erts_smp_proc_trylock(rp, need_locks) == EBUSY) { + erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_STATUS); + rp = erts_pid2proc(c_p, c_p_locks|ERTS_PROC_LOCK_STATUS, + pid, pid_locks|ERTS_PROC_LOCK_STATUS); + } + goto done; + } +#endif + running: /* @@ -8447,7 +8558,7 @@ pid2proc_not_running(Process *c_p, ErtsProcLocks c_p_locks, else { ErtsProcLocks need_locks = pid_locks & ~ERTS_PROC_LOCK_STATUS; if (need_locks && erts_smp_proc_trylock(rp, need_locks) == EBUSY) { - if (ERTS_PSFLG_RUNNING_SYS + if ((ERTS_PSFLG_RUNNING_SYS|ERTS_PSFLG_DIRTY_RUNNING_SYS) & erts_smp_atomic32_read_nob(&rp->state)) { /* Executing system task... */ resume_process(rp, ERTS_PROC_LOCK_STATUS); @@ -8474,7 +8585,7 @@ pid2proc_not_running(Process *c_p, ErtsProcLocks c_p_locks, * from being selected for normal execution regardless * of locks held or not held on it... */ - ASSERT(!(ERTS_PSFLG_RUNNING + ASSERT(!((ERTS_PSFLG_RUNNING|ERTS_PSFLG_DIRTY_RUNNING_SYS) & erts_smp_atomic32_read_nob(&rp->state))); if (!suspend) @@ -9015,28 +9126,43 @@ erts_run_queues_len(Uint *qlen, int atomic_queues_read, int incl_active_sched) } Eterm -erts_process_status(Process *c_p, ErtsProcLocks c_p_locks, - Process *rp, Eterm rpid) +erts_process_state2status(erts_aint32_t state) +{ + if (state & ERTS_PSFLG_FREE) + return am_free; + + if (state & ERTS_PSFLG_EXITING) + return am_exiting; + + if (state & ERTS_PSFLG_GC) + return am_garbage_collecting; + + if (state & ERTS_PSFLG_SUSPENDED) + return am_suspended; + + if (state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS)) + return am_running; + + if (state & (ERTS_PSFLG_ACTIVE + | ERTS_PSFLG_ACTIVE_SYS + | ERTS_PSFLG_DIRTY_ACTIVE_SYS)) + return am_runnable; + + return am_waiting; +} + +Eterm +erts_process_status(Process *rp, Eterm rpid) { Eterm res = am_undefined; Process *p = rp ? rp : erts_proc_lookup_raw(rpid); if (p) { erts_aint32_t state = erts_smp_atomic32_read_acqb(&p->state); - if (state & ERTS_PSFLG_FREE) - res = am_free; - else if (state & ERTS_PSFLG_EXITING) - res = am_exiting; - else if (state & ERTS_PSFLG_GC) - res = am_garbage_collecting; - else if (state & ERTS_PSFLG_SUSPENDED) - res = am_suspended; - else if (state & ERTS_PSFLG_RUNNING) - res = am_running; - else if (state & ERTS_PSFLG_ACTIVE) - res = am_runnable; - else - res = am_waiting; + res = erts_process_state2status(state); } #ifdef ERTS_SMP else { @@ -9251,7 +9377,76 @@ scheduler_gc_proc(Process *c_p, int reds_left) return reds; } +static ERTS_INLINE void +clean_dirty_start(Process *p) +{ +#if defined(ERTS_DIRTY_SCHEDULERS) && !defined(ARCH_64) + void *ptr = ERTS_PROC_SET_DIRTY_CPU_START(p, NULL); + if (ptr) + erts_free(ERTS_ALC_T_DIRTY_START, ptr); +#endif +} +static ERTS_INLINE void +save_dirty_start(ErtsSchedulerData *esdp, Process *c_p) +{ +#ifdef ERTS_DIRTY_SCHEDULERS + if (ERTS_RUNQ_IS_DIRTY_CPU_RUNQ(esdp->run_queue)) { + ErtsMonotonicTime time = erts_get_monotonic_time(esdp); +#ifdef ARCH_64 + ERTS_PROC_SET_DIRTY_CPU_START(c_p, (void *) time); +#else + ErtsMonotonicTime *stimep; + + stimep = (ErtsMonotonicTime *) ERTS_PROC_GET_DIRTY_CPU_START(c_p); + if (!stimep) { + stimep = erts_alloc(ERTS_ALC_T_DIRTY_START, + sizeof(ErtsMonotonicTime)); + ERTS_PROC_SET_DIRTY_CPU_START(c_p, (void *) stimep); + } + *stimep = time; +#endif + } +#endif +} + +static ERTS_INLINE int +get_dirty_reds(ErtsSchedulerData *esdp, Process *c_p) +{ + +#ifndef ERTS_DIRTY_SCHEDULERS + return -1; +#else + ErtsMonotonicTime stime, time; + + if (!ERTS_RUNQ_IS_DIRTY_CPU_RUNQ(esdp->run_queue)) + return 1; + +#ifdef ARCH_64 + stime = (ErtsMonotonicTime) ERTS_PROC_GET_DIRTY_CPU_START(c_p); +#else + { + ErtsMonotonicTime *stimep; + stimep = (ErtsMonotonicTime *) ERTS_PROC_GET_DIRTY_CPU_START(c_p); + ASSERT(stimep); + stime = *stimep; + } +#endif + + time = erts_get_monotonic_time(esdp); + + ASSERT(stime && stime < time); + + time -= stime; + time = ERTS_MONOTONIC_TO_USEC(time); + time *= 2; + + if (time > INT_MAX) + return INT_MAX; + return (int) time; +#endif + +} /* * schedule() is called from BEAM (process_main()) or HiPE @@ -9283,6 +9478,7 @@ Process *schedule(Process *p, int calls) int reds; Uint32 flags; erts_aint32_t state = 0; /* Supress warning... */ + int is_normal_sched; ERTS_MSACC_DECLARE_CACHE(); @@ -9312,25 +9508,44 @@ Process *schedule(Process *p, int calls) */ if (!p) { /* NULL in the very first schedule() call */ esdp = erts_get_scheduler_data(); + is_normal_sched = !ERTS_SCHEDULER_IS_DIRTY(esdp); rq = erts_get_runq_current(esdp); ASSERT(esdp); fcalls = (int) erts_smp_atomic32_read_acqb(&function_calls); actual_reds = reds = 0; erts_smp_runq_lock(rq); } else { - sched_out_proc: - #ifdef ERTS_SMP - ERTS_SMP_CHK_HAVE_ONLY_MAIN_PROC_LOCK(p); +#ifdef ERTS_DIRTY_SCHEDULERS + esdp = p->scheduler_data; + is_normal_sched = esdp != NULL; + if (is_normal_sched) + ASSERT(!ERTS_SCHEDULER_IS_DIRTY(esdp)); + else { + esdp = erts_get_scheduler_data(); + ASSERT(ERTS_SCHEDULER_IS_DIRTY(esdp)); + } +#else esdp = p->scheduler_data; + is_normal_sched = 1; +#endif ASSERT(esdp->current_process == p || esdp->free_process == p); #else esdp = erts_scheduler_data; ASSERT(esdp->current_process == p); + is_normal_sched = 1; #endif - reds = actual_reds = calls - esdp->virtual_reds; + sched_out_proc: + + ERTS_SMP_CHK_HAVE_ONLY_MAIN_PROC_LOCK(p); + + if (is_normal_sched) + reds = actual_reds = calls - esdp->virtual_reds; + else + reds = actual_reds = get_dirty_reds(esdp, p); + ASSERT(actual_reds >= 0); if (reds < ERTS_PROC_MIN_CONTEXT_SWITCH_REDS_COST) reds = ERTS_PROC_MIN_CONTEXT_SWITCH_REDS_COST; @@ -9377,7 +9592,7 @@ Process *schedule(Process *p, int calls) state = erts_smp_atomic32_read_nob(&p->state); #ifdef ERTS_SMP - if (state & ERTS_PSFLG_PENDING_EXIT) + if (is_normal_sched && (state & ERTS_PSFLG_PENDING_EXIT)) erts_handle_pending_exit(p, (ERTS_PROC_LOCK_MAIN | ERTS_PROC_LOCK_STATUS)); if (p->pending_suspenders) @@ -9387,7 +9602,8 @@ Process *schedule(Process *p, int calls) esdp->reductions += reds; - schedule_out_process(rq, state, p, proxy_p); /* Returns with rq locked! */ + /* schedule_out_process() returns with rq locked! */ + schedule_out_process(rq, state, p, proxy_p, is_normal_sched); proxy_p = NULL; ERTS_PROC_REDUCTIONS_EXECUTED(esdp, rq, @@ -9405,13 +9621,20 @@ Process *schedule(Process *p, int calls) ERTS_MSACC_SET_STATE_CACHED_M(ERTS_MSACC_STATE_OTHER); if (state & ERTS_PSFLG_FREE) { + if (!is_normal_sched) { + ASSERT(p->flags & F_DELAYED_DEL_PROC); + erts_proc_dec_refc(p); + } + else { #ifdef ERTS_SMP - ASSERT(esdp->free_process == p); - esdp->free_process = NULL; + ASSERT(esdp->free_process == p); + esdp->free_process = NULL; #else - erts_proc_dec_refc(p); + erts_proc_dec_refc(p); #endif + } } + #ifdef ERTS_SMP ASSERT(!esdp->free_process); #endif @@ -9419,7 +9642,7 @@ Process *schedule(Process *p, int calls) ERTS_SMP_CHK_NO_PROC_LOCKS; - if (!ERTS_SCHEDULER_IS_DIRTY(esdp)) { + if (is_normal_sched) { if (esdp->check_time_reds >= ERTS_CHECK_TIME_REDS) (void) erts_get_monotonic_time(esdp); @@ -9433,23 +9656,15 @@ Process *schedule(Process *p, int calls) } - ERTS_SMP_LC_ASSERT(ERTS_SCHEDULER_IS_DIRTY(esdp) - || !erts_thr_progress_is_blocking()); + ERTS_SMP_LC_ASSERT(!is_normal_sched || !erts_thr_progress_is_blocking()); check_activities_to_run: { + erts_aint32_t psflg_running, psflg_running_sys; #ifdef ERTS_SMP ErtsMigrationPaths *mps; ErtsMigrationPath *mp; - if (!ERTS_SCHEDULER_IS_DIRTY(esdp)) { - ErtsProcList *pnd_xtrs = rq->procs.pending_exiters; - if (erts_proclist_fetch(&pnd_xtrs, NULL)) { - rq->procs.pending_exiters = NULL; - erts_smp_runq_unlock(rq); - handle_pending_exiters(pnd_xtrs); - erts_smp_runq_lock(rq); - } - + if (is_normal_sched) { if (rq->check_balance_reds <= 0) check_balance(rq); @@ -9466,32 +9681,35 @@ Process *schedule(Process *p, int calls) continue_check_activities_to_run: flags = ERTS_RUNQ_FLGS_GET_NOB(rq); continue_check_activities_to_run_known_flags: - ASSERT(ERTS_SCHEDULER_IS_DIRTY(esdp) - || flags & ERTS_RUNQ_FLG_NONEMPTY); + ASSERT(!is_normal_sched || (flags & ERTS_RUNQ_FLG_NONEMPTY)); - if (flags & (ERTS_RUNQ_FLG_CHK_CPU_BIND|ERTS_RUNQ_FLG_SUSPENDED)) { - if (flags & ERTS_RUNQ_FLG_SUSPENDED) { - (void) ERTS_RUNQ_FLGS_UNSET_NOB(rq, ERTS_RUNQ_FLG_EXEC); + if (!is_normal_sched) { + if (erts_smp_atomic32_read_acqb(&esdp->ssi->flags) + & ERTS_SSI_FLG_SUSPENDED) { suspend_scheduler(esdp); - flags = ERTS_RUNQ_FLGS_SET_NOB(rq, ERTS_RUNQ_FLG_EXEC); - flags |= ERTS_RUNQ_FLG_EXEC; - } - if (flags & ERTS_RUNQ_FLG_CHK_CPU_BIND) { - flags = ERTS_RUNQ_FLGS_UNSET(rq, ERTS_RUNQ_FLG_CHK_CPU_BIND); - flags &= ~ ERTS_RUNQ_FLG_CHK_CPU_BIND; - erts_sched_check_cpu_bind(esdp); } } -#ifdef ERTS_DIRTY_SCHEDULERS - else if (ERTS_SCHEDULER_IS_DIRTY(esdp) - && (erts_smp_atomic32_read_acqb(&esdp->ssi->flags) - & ERTS_SSI_FLG_SUSPENDED)) - suspend_scheduler(esdp); -#endif - - if (!ERTS_SCHEDULER_IS_DIRTY(esdp)) { + else { erts_aint32_t aux_work; - int leader_update = erts_thr_progress_update(esdp); + int leader_update; + + ASSERT(is_normal_sched); + + if (flags & (ERTS_RUNQ_FLG_CHK_CPU_BIND|ERTS_RUNQ_FLG_SUSPENDED)) { + if (flags & ERTS_RUNQ_FLG_SUSPENDED) { + (void) ERTS_RUNQ_FLGS_UNSET_NOB(rq, ERTS_RUNQ_FLG_EXEC); + suspend_scheduler(esdp); + flags = ERTS_RUNQ_FLGS_SET_NOB(rq, ERTS_RUNQ_FLG_EXEC); + flags |= ERTS_RUNQ_FLG_EXEC; + } + if (flags & ERTS_RUNQ_FLG_CHK_CPU_BIND) { + flags = ERTS_RUNQ_FLGS_UNSET(rq, ERTS_RUNQ_FLG_CHK_CPU_BIND); + flags &= ~ERTS_RUNQ_FLG_CHK_CPU_BIND; + erts_sched_check_cpu_bind(esdp); + } + } + + leader_update = erts_thr_progress_update(esdp); aux_work = erts_atomic32_read_acqb(&esdp->ssi->aux_work); if (aux_work | leader_update) { erts_smp_runq_unlock(rq); @@ -9517,19 +9735,13 @@ Process *schedule(Process *p, int calls) flags = ERTS_RUNQ_FLGS_GET_NOB(rq); -#ifdef ERTS_DIRTY_SCHEDULERS - if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix) && rq->halt_in_progress) { - /* - * TODO: if halt in progress, need to put the dirty scheduler - * to sleep somewhere around here to prevent it from picking up - * new work - */ + if (!is_normal_sched && rq->halt_in_progress) { + /* Wait for emulator to terminate... */ + while (1) + erts_milli_sleep(1000*1000); } - else -#endif - - if ((!(flags & ERTS_RUNQ_FLGS_QMASK) && !rq->misc.start) - || (rq->halt_in_progress && ERTS_EMPTY_RUNQ_PORTS(rq))) { + else if ((!(flags & ERTS_RUNQ_FLGS_QMASK) && !rq->misc.start) + || (rq->halt_in_progress && ERTS_EMPTY_RUNQ_PORTS(rq))) { /* Prepare for scheduler wait */ #ifdef ERTS_SMP ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(rq)); @@ -9543,7 +9755,7 @@ Process *schedule(Process *p, int calls) if (flags & ERTS_RUNQ_FLG_INACTIVE) empty_runq(rq); else { - if (!ERTS_RUNQ_IX_IS_DIRTY(rq->ix) && try_steal_task(rq)) + if (is_normal_sched && try_steal_task(rq)) goto continue_check_activities_to_run; empty_runq(rq); @@ -9572,9 +9784,9 @@ Process *schedule(Process *p, int calls) goto check_activities_to_run; } - else if (!ERTS_SCHEDULER_IS_DIRTY(esdp) && - (fcalls > input_reductions && - prepare_for_sys_schedule(!0))) { + else if (is_normal_sched + && (fcalls > input_reductions + && prepare_for_sys_schedule(!0))) { ErtsMonotonicTime current_time; /* * Schedule system-level activities. @@ -9688,11 +9900,17 @@ Process *schedule(Process *p, int calls) ASSERT(p); /* Wrong qmask in rq->flags? */ - if (ERTS_SCHEDULER_IS_DIRTY(esdp)) - psflg_band_mask = ~((erts_aint32_t) 0); - else + if (is_normal_sched) { + psflg_running = ERTS_PSFLG_RUNNING; + psflg_running_sys = ERTS_PSFLG_RUNNING_SYS; psflg_band_mask = ~(((erts_aint32_t) 1) << (ERTS_PSFLGS_GET_PRQ_PRIO(state) + ERTS_PSFLGS_IN_PRQ_MASK_OFFSET)); + } + else { + psflg_running = ERTS_PSFLG_DIRTY_RUNNING; + psflg_running_sys = ERTS_PSFLG_DIRTY_RUNNING_SYS; + psflg_band_mask = ~((erts_aint32_t) 0); + } if (!(state & ERTS_PSFLG_PROXY)) psflg_band_mask &= ~ERTS_PSFLG_IN_RUNQ; @@ -9707,34 +9925,53 @@ Process *schedule(Process *p, int calls) state = erts_smp_atomic32_read_nob(&p->state); } +#ifdef ERTS_DIRTY_SCHEDULERS + if (!is_normal_sched) + clear_proc_dirty_queue_bit(p, rq, qbit); +#endif + while (1) { - erts_aint32_t exp, new, tmp; - tmp = new = exp = state; + erts_aint32_t exp, new; + int run_process; + new = exp = state; new &= psflg_band_mask; - if (!(state & (ERTS_PSFLG_RUNNING - | ERTS_PSFLG_RUNNING_SYS))) { - tmp = state & (ERTS_PSFLG_SUSPENDED - | ERTS_PSFLG_PENDING_EXIT - | ERTS_PSFLG_ACTIVE_SYS - | ERTS_PSFLG_DIRTY_ACTIVE_SYS); - if (tmp != ERTS_PSFLG_SUSPENDED) { - if (state & (ERTS_PSFLG_ACTIVE_SYS - | ERTS_PSFLG_DIRTY_ACTIVE_SYS)) - new |= ERTS_PSFLG_RUNNING_SYS; - else - new |= ERTS_PSFLG_RUNNING; - } + /* + * Run process if not already running (or free) + * or exiting and not running on a normal + * scheduler, and not suspended (and not in a + * state where suspend should be ignored). + */ + run_process = (((!(state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS + | ERTS_PSFLG_FREE))) +#ifdef ERTS_DIRTY_SCHEDULERS + | (((state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_FREE + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_EXITING)) + == ERTS_PSFLG_EXITING) + & (!!is_normal_sched)) +#endif + ) + & ((state & (ERTS_PSFLG_SUSPENDED + | ERTS_PSFLG_EXITING + | ERTS_PSFLG_FREE + | ERTS_PSFLG_PENDING_EXIT + | ERTS_PSFLG_ACTIVE_SYS + | ERTS_PSFLG_DIRTY_ACTIVE_SYS)) + != ERTS_PSFLG_SUSPENDED)); + if (run_process) { + if (state & (ERTS_PSFLG_ACTIVE_SYS + | ERTS_PSFLG_DIRTY_ACTIVE_SYS)) + new |= psflg_running_sys; + else + new |= psflg_running; } state = erts_smp_atomic32_cmpxchg_relb(&p->state, new, exp); if (state == exp) { - if ((state & (ERTS_PSFLG_RUNNING - | ERTS_PSFLG_RUNNING_SYS - | ERTS_PSFLG_FREE)) - || ((state & (ERTS_PSFLG_SUSPENDED - | ERTS_PSFLG_PENDING_EXIT - | ERTS_PSFLG_ACTIVE_SYS - | ERTS_PSFLG_DIRTY_ACTIVE_SYS)) - == ERTS_PSFLG_SUSPENDED)) { + if (!run_process) { if (proxy_p) { free_proxy_proc(proxy_p); proxy_p = NULL; @@ -9761,34 +9998,13 @@ Process *schedule(Process *p, int calls) erts_smp_runq_unlock(rq); -#ifdef ERTS_DIRTY_SCHEDULERS - if (ERTS_SCHEDULER_IS_DIRTY(esdp)) { -#ifdef DEBUG - int old_dqbit; -#endif - int dqbit = qbit; - - if (rq == ERTS_DIRTY_CPU_RUNQ) - dqbit <<= ERTS_PDSFLGS_IN_CPU_PRQ_MASK_OFFSET; - else { - ASSERT(rq == ERTS_DIRTY_IO_RUNQ); - dqbit <<= ERTS_PDSFLGS_IN_IO_PRQ_MASK_OFFSET; - } - -#ifdef DEBUG - old_dqbit = (int) -#else - (void) -#endif - erts_smp_atomic32_read_band_mb(&p->dirty_state, ~dqbit); - ASSERT(old_dqbit & dqbit); - } -#endif /* ERTS_DIRTY_SCHEDULERS */ - #endif /* ERTS_SMP */ } + if (!is_normal_sched) + save_dirty_start(esdp, p); + #ifdef ERTS_SMP if (flags & ERTS_RUNQ_FLG_PROTECTED) @@ -9805,9 +10021,7 @@ Process *schedule(Process *p, int calls) UWord old = ERTS_PROC_SCHED_ID(p, (UWord) esdp->no); int migrated = old && old != esdp->no; -#ifdef ERTS_DIRTY_SCHEDULERS - ASSERT(!ERTS_SCHEDULER_IS_DIRTY(esdp)); -#endif + ASSERT(is_normal_sched); prio = (int) ERTS_PSFLGS_GET_USR_PRIO(state); @@ -9821,20 +10035,21 @@ Process *schedule(Process *p, int calls) erts_smp_spin_unlock(&erts_sched_stat.lock); } - ASSERT(!p->scheduler_data); - p->scheduler_data = esdp; - state = erts_smp_atomic32_read_nob(&p->state); -#ifdef ERTS_DIRTY_SCHEDULERS - if (!ERTS_SCHEDULER_IS_DIRTY(esdp)) { - if (!!(state & ERTS_PSFLGS_DIRTY_WORK) - & !(state & ERTS_PSFLG_ACTIVE_SYS)) { + ASSERT(!p->scheduler_data); +#ifndef ERTS_DIRTY_SCHEDULERS + p->scheduler_data = esdp; +#else /* ERTS_DIRTY_SCHEDULERS */ + if (is_normal_sched) { + if ((!!(state & ERTS_PSFLGS_DIRTY_WORK)) + & (!(state & ERTS_PSFLG_ACTIVE_SYS))) { /* Migrate to dirty scheduler... */ sunlock_sched_out_proc: erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); goto sched_out_proc; } + p->scheduler_data = esdp; } else { if (state & (ERTS_PSFLG_ACTIVE_SYS @@ -9870,7 +10085,10 @@ Process *schedule(Process *p, int calls) erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); - if (IS_TRACED(p)) { + /* Clear tracer if it has been removed */ + if (IS_TRACED(p) && erts_is_tracer_proc_enabled( + p, ERTS_PROC_LOCK_MAIN, &p->common)) { + if (state & ERTS_PSFLG_EXITING) { if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_EXIT)) trace_sched(p, ERTS_PROC_LOCK_MAIN, am_in_exiting); @@ -9885,13 +10103,8 @@ Process *schedule(Process *p, int calls) } } - -#ifdef ERTS_SMP - /* Clears tracer if it has been removed */ - (void)ERTS_TRACER_PROC_IS_ENABLED(p); -#endif - - if (state & ERTS_PSFLG_RUNNING_SYS) { + if (state & (ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING_SYS)) { /* * GC is normally never delayed when a process * is scheduled out, but might be when executing @@ -9905,7 +10118,7 @@ Process *schedule(Process *p, int calls) reds -= cost; if (reds <= 0 #ifdef ERTS_DIRTY_SCHEDULERS - || ERTS_SCHEDULER_IS_DIRTY(esdp) + || !is_normal_sched || (state & ERTS_PSFLGS_DIRTY_WORK) #endif ) { @@ -9913,8 +10126,8 @@ Process *schedule(Process *p, int calls) } } - ASSERT(state & ERTS_PSFLG_RUNNING_SYS); - ASSERT(!(state & ERTS_PSFLG_RUNNING)); + ASSERT(state & psflg_running_sys); + ASSERT(!(state & psflg_running)); while (1) { erts_aint32_t n, e; @@ -9926,8 +10139,8 @@ Process *schedule(Process *p, int calls) } n = e = state; - n &= ~ERTS_PSFLG_RUNNING_SYS; - n |= ERTS_PSFLG_RUNNING; + n &= ~psflg_running_sys; + n |= psflg_running; state = erts_smp_atomic32_cmpxchg_mb(&p->state, n, e); if (state == e) { @@ -9935,8 +10148,8 @@ Process *schedule(Process *p, int calls) break; } - ASSERT(state & ERTS_PSFLG_RUNNING_SYS); - ASSERT(!(state & ERTS_PSFLG_RUNNING)); + ASSERT(state & psflg_running_sys); + ASSERT(!(state & psflg_running)); } } @@ -10525,7 +10738,10 @@ save_gc_task(Process *c_p, ErtsProcSysTask *st, int prio) } state = erts_smp_atomic32_read_nob(&c_p->state); - ASSERT((ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS) & state); + ASSERT((ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS) & state); while (!(state & ERTS_PSFLG_DELAYED_SYS) || prio < ERTS_PSFLGS_GET_ACT_PRIO(state)) { @@ -10850,6 +11066,8 @@ erts_get_exact_total_reductions(Process *c_p, Uint *redsp, Uint *diffp) erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); } +static void delete_process(Process* p); + void erts_free_proc(Process *p) { @@ -10858,6 +11076,8 @@ erts_free_proc(Process *p) #endif ASSERT(erts_smp_atomic32_read_nob(&p->state) & ERTS_PSFLG_FREE); ASSERT(0 == erts_proc_read_refc(p)); + if (p->flags & F_DELAYED_DEL_PROC) + delete_process(p); erts_free(ERTS_ALC_T_PROC, (void *) p); } @@ -11021,9 +11241,13 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). p->min_heap_size = so->min_heap_size; p->min_vheap_size = so->min_vheap_size; p->max_gen_gcs = so->max_gen_gcs; + MAX_HEAP_SIZE_SET(p, so->max_heap_size); + MAX_HEAP_SIZE_FLAGS_SET(p, so->max_heap_flags); } else { p->min_heap_size = H_MIN_SIZE; p->min_vheap_size = BIN_VH_MIN_SIZE; + MAX_HEAP_SIZE_SET(p, H_MAX_SIZE); + MAX_HEAP_SIZE_FLAGS_SET(p, H_MAX_FLAGS); p->max_gen_gcs = (Uint16) erts_smp_atomic32_read_nob(&erts_max_gen_gcs); } p->schedule_count = 0; @@ -11486,18 +11710,36 @@ erts_cleanup_empty_process(Process* p) #endif } -/* - * p must be the currently executing process. - */ static void delete_process(Process* p) { Eterm *heap; ErtsPSD *psd; + struct saved_calls *scb; + process_breakpoint_time_t *pbt; + void *nif_export; + VERBOSE(DEBUG_PROCESSES, ("Removing process: %T\n",p->common.id)); VERBOSE(DEBUG_SHCOPY, ("[pid=%T] delete process: %p %p %p %p\n", p->common.id, HEAP_START(p), HEAP_END(p), OLD_HEAP(p), OLD_HEND(p))); + scb = ERTS_PROC_SET_SAVED_CALLS_BUF(p, NULL); + + if (scb) { + p->fcalls += CONTEXT_REDS; /* Reduction counting depends on this... */ + erts_free(ERTS_ALC_T_CALLS_BUF, (void *) scb); + } + + pbt = ERTS_PROC_SET_CALL_TIME(p, NULL); + if (pbt) + erts_free(ERTS_ALC_T_BPD, (void *) pbt); + + nif_export = ERTS_PROC_SET_NIF_TRAP_EXPORT(p, NULL); + if (nif_export) + erts_destroy_nif_export(nif_export); + + clean_dirty_start(p); + /* Cleanup psd */ psd = (ErtsPSD *) erts_smp_atomic_read_nob(&p->psd); @@ -11600,7 +11842,8 @@ set_proc_exiting(Process *p, p->i = (BeamInstr *) beam_exit; #ifndef ERTS_SMP - if (state & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS)) { + if (state & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS) + && !(state & ERTS_PSFLG_GC)) { /* * I non smp case: * @@ -11629,7 +11872,10 @@ set_proc_self_exiting(Process *c_p) ERTS_SMP_LC_ASSERT(erts_proc_lc_my_proc_locks(c_p) == ERTS_PROC_LOCKS_ALL); state = erts_smp_atomic32_read_nob(&c_p->state); - ASSERT(state & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS)); + ASSERT(state & (ERTS_PSFLG_RUNNING + |ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS)); #ifdef DEBUG enqueue = @@ -11679,51 +11925,73 @@ erts_handle_pending_exit(Process *c_p, ErtsProcLocks locks) erts_smp_proc_unlock(c_p, xlocks); } +static void save_pending_exiter(Process *p, ErtsProcList *plp); + static void -handle_pending_exiters(ErtsProcList *pnd_xtrs) +do_handle_pending_exiters(ErtsProcList *pnd_xtrs) { /* 'list' is expected to have been fetched (i.e. not a ring anymore) */ ErtsProcList *plp = pnd_xtrs; while (plp) { - ErtsProcList *free_plp; - Process *p = erts_pid2proc(NULL, 0, plp->pid, ERTS_PROC_LOCKS_ALL); + ErtsProcList *next_plp = plp->next; + Process *p = erts_proc_lookup(plp->pid); if (p) { - if (erts_proclist_same(plp, p)) { - erts_aint32_t state = erts_smp_atomic32_read_acqb(&p->state); - if (!(state & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS))) { - ASSERT(state & ERTS_PSFLG_PENDING_EXIT); - erts_handle_pending_exit(p, ERTS_PROC_LOCKS_ALL); + erts_aint32_t state; + /* + * If the process is running on a normal scheduler, the + * pending exit will soon be detected and handled by the + * scheduler running the process (at schedule in/out). + */ + if (erts_smp_proc_trylock(p, ERTS_PROC_LOCKS_ALL) != EBUSY) { + if (erts_proclist_same(plp, p)) { + state = erts_smp_atomic32_read_acqb(&p->state); + if (!(state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_EXITING))) { + ASSERT(state & ERTS_PSFLG_PENDING_EXIT); + erts_handle_pending_exit(p, ERTS_PROC_LOCKS_ALL); + } + } + erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL); + } + else { + erts_smp_proc_lock(p, ERTS_PROC_LOCK_STATUS); + if (erts_proclist_same(plp, p)) { + state = erts_smp_atomic32_read_acqb(&p->state); + if (!(state & (ERTS_PSFLG_RUNNING + | ERTS_PSFLG_RUNNING_SYS + | ERTS_PSFLG_EXITING))) { + /* + * Save process and try to acquire all + * locks at a later time... + */ + save_pending_exiter(p, plp); + plp = NULL; + } } + erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); } - erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL); } - free_plp = plp; - plp = plp->next; - proclist_destroy(free_plp); + if (plp) + proclist_destroy(plp); + plp = next_plp; } } static void -save_pending_exiter(Process *p) +save_pending_exiter(Process *p, ErtsProcList *plp) { - ErtsProcList *plp; + ErtsSchedulerSleepInfo *ssi; ErtsRunQueue *rq; - ErtsSchedulerData *esdp = erts_get_scheduler_data(); ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_STATUS & erts_proc_lc_my_proc_locks(p)); - if (!esdp) - rq = RUNQ_READ_RQ(&p->run_queue); - else - rq = esdp->run_queue; - -#ifdef ERTS_DIRTY_SCHEDULERS - if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix)) - rq = ERTS_RUNQ_IX(0); /* Handle on ordinary scheduler */ -#endif + rq = RUNQ_READ_RQ(&p->run_queue); + ASSERT(rq && !ERTS_RUNQ_IX_IS_DIRTY(rq->ix)); - plp = proclist_create(p); + if (!plp) + plp = proclist_create(p); erts_smp_runq_lock(rq); @@ -11731,9 +11999,11 @@ save_pending_exiter(Process *p) non_empty_runq(rq); + ssi = rq->scheduler->ssi; + erts_smp_runq_unlock(rq); - wake_scheduler(rq); + set_aux_work_flags_wakeup_nob(ssi, ERTS_SSI_AUX_WORK_PENDING_EXITERS); } #endif @@ -11933,7 +12203,7 @@ send_exit_signal(Process *c_p, /* current process if and only if (need_locks && erts_smp_proc_trylock(rp, need_locks) == EBUSY) { /* ... but we havn't got all locks on it ... */ - save_pending_exiter(rp); + save_pending_exiter(rp, NULL); /* * The pending exit will be discovered when next * process is scheduled in @@ -12401,10 +12671,8 @@ erts_continue_exit_process(Process *p) ErtsProcLocks curr_locks = ERTS_PROC_LOCK_MAIN; Eterm reason = p->fvalue; DistEntry *dep; - struct saved_calls *scb; - process_breakpoint_time_t *pbt; erts_aint32_t state; - void *nif_export; + int delay_del_proc = 0; #ifdef DEBUG int yield_allowed = 1; @@ -12547,7 +12815,7 @@ erts_continue_exit_process(Process *p) { /* Do *not* use erts_get_runq_proc() */ ErtsRunQueue *rq; - rq = erts_get_runq_current(ERTS_GET_SCHEDULER_DATA_FROM_PROC(p)); + rq = erts_get_runq_current(erts_proc_sched_data(p)); erts_smp_runq_lock(rq); @@ -12593,16 +12861,24 @@ erts_continue_exit_process(Process *p) break; } +#ifdef ERTS_DIRTY_SCHEDULERS + if (a & (ERTS_PSFLG_DIRTY_RUNNING + | ERTS_PSFLG_DIRTY_RUNNING_SYS)) { + p->flags |= F_DELAYED_DEL_PROC; + delay_del_proc = 1; + /* + * The dirty scheduler will also decrease + * refc when done... + */ + erts_proc_inc_refc(p); + } +#endif + if (refc_inced && !(n & ERTS_PSFLG_IN_RUNQ)) erts_proc_dec_refc(p); } dep = (p->flags & F_DISTRIBUTION) ? erts_this_dist_entry : NULL; - scb = ERTS_PROC_SET_SAVED_CALLS_BUF(p, NULL); - if (scb) - p->fcalls += CONTEXT_REDS; /* Reduction counting depends on this... */ - pbt = ERTS_PROC_SET_CALL_TIME(p, NULL); - nif_export = ERTS_PROC_SET_NIF_TRAP_EXPORT(p, NULL); erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL); #ifdef BM_COUNTERS @@ -12642,22 +12918,14 @@ erts_continue_exit_process(Process *p) have none here */ } - if (scb) - erts_free(ERTS_ALC_T_CALLS_BUF, (void *) scb); - - if (pbt) - erts_free(ERTS_ALC_T_BPD, (void *) pbt); - - if (nif_export) - erts_destroy_nif_export(nif_export); - #ifdef ERTS_SMP erts_flush_trace_messages(p, 0); #endif ERTS_TRACER_CLEAR(&ERTS_TRACER(p)); - delete_process(p); + if (!delay_del_proc) + delete_process(p); #ifdef ERTS_SMP erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); @@ -12687,6 +12955,7 @@ erts_continue_exit_process(Process *p) ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN == erts_proc_lc_my_proc_locks(p)); + BUMP_ALL_REDS(p); } /* @@ -13006,11 +13275,13 @@ void erts_halt(int code) int erts_dbg_check_halloc_lock(Process *p) { + ErtsSchedulerData *esdp; if (ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)) return 1; if (p->common.id == ERTS_INVALID_PID) return 1; - if (p->scheduler_data && p == p->scheduler_data->match_pseudo_process) + esdp = erts_proc_sched_data(p); + if (esdp && p == esdp->match_pseudo_process) return 1; if (erts_thr_progress_is_blocking()) return 1; diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index 59da9c1779..b44ac442aa 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -304,6 +304,7 @@ typedef enum { ERTS_SSI_AUX_WORK_ASYNC_READY_CLEAN_IX, ERTS_SSI_AUX_WORK_MISC_THR_PRGR_IX, ERTS_SSI_AUX_WORK_MISC_IX, + ERTS_SSI_AUX_WORK_PENDING_EXITERS_IX, ERTS_SSI_AUX_WORK_SET_TMO_IX, ERTS_SSI_AUX_WORK_MSEG_CACHE_CHECK_IX, ERTS_SSI_AUX_WORK_REAP_PORTS_IX, @@ -336,6 +337,8 @@ typedef enum { (((erts_aint32_t) 1) << ERTS_SSI_AUX_WORK_MISC_THR_PRGR_IX) #define ERTS_SSI_AUX_WORK_MISC \ (((erts_aint32_t) 1) << ERTS_SSI_AUX_WORK_MISC_IX) +#define ERTS_SSI_AUX_WORK_PENDING_EXITERS \ + (((erts_aint32_t) 1) << ERTS_SSI_AUX_WORK_PENDING_EXITERS_IX) #define ERTS_SSI_AUX_WORK_SET_TMO \ (((erts_aint32_t) 1) << ERTS_SSI_AUX_WORK_SET_TMO_IX) #define ERTS_SSI_AUX_WORK_MSEG_CACHE_CHECK \ @@ -645,6 +648,7 @@ struct ErtsSchedulerData_ { Uint no; /* Scheduler number for normal schedulers */ #ifdef ERTS_DIRTY_SCHEDULERS ErtsDirtySchedId dirty_no; /* Scheduler number for dirty schedulers */ + Process *dirty_shadow_process; #endif Port *current_port; ErtsRunQueue *run_queue; @@ -805,14 +809,26 @@ erts_smp_reset_max_len(ErtsRunQueue *rq, ErtsRunQueueInfo *rqi) #define ERTS_PSD_CALL_TIME_BP 3 #define ERTS_PSD_DELAYED_GC_TASK_QS 4 #define ERTS_PSD_NIF_TRAP_EXPORT 5 -#ifdef HIPE #define ERTS_PSD_SUSPENDED_SAVED_CALLS_BUF 6 -#endif - -#ifdef HIPE -#define ERTS_PSD_SIZE 7 -#else -#define ERTS_PSD_SIZE 6 +#define ERTS_PSD_DIRTY_CPU_START 7 + +#define ERTS_PSD_SIZE 8 + +#if !defined(HIPE) && !defined(ERTS_DIRTY_SCHEDULERS) +# undef ERTS_PSD_SUSPENDED_SAVED_CALLS_BUF +# undef ERTS_PSD_DIRTY_CPU_START +# undef ERTS_PSD_SIZE +# define ERTS_PSD_SIZE 6 +#elif !defined(HIPE) +# undef ERTS_PSD_SUSPENDED_SAVED_CALLS_BUF +# undef ERTS_PSD_DIRTY_CPU_START +# undef ERTS_PSD_SIZE +# define ERTS_PSD_DIRTY_CPU_START 6 +# define ERTS_PSD_SIZE 7 +#elif !defined(ERTS_DIRTY_SCHEDULERS) +# undef ERTS_PSD_DIRTY_CPU_START +# undef ERTS_PSD_SIZE +# define ERTS_PSD_SIZE 7 #endif typedef struct { @@ -918,6 +934,15 @@ struct ErtsPendingSuspend_ { # define BIN_OLD_VHEAP_SZ(p) (p)->bin_old_vheap_sz # define BIN_OLD_VHEAP(p) (p)->bin_old_vheap +# define MAX_HEAP_SIZE_GET(p) ((p)->max_heap_size >> 2) +# define MAX_HEAP_SIZE_SET(p, sz) ((p)->max_heap_size = ((sz) << 2) | \ + MAX_HEAP_SIZE_FLAGS_GET(p)) +# define MAX_HEAP_SIZE_FLAGS_GET(p) ((p)->max_heap_size & 0x3) +# define MAX_HEAP_SIZE_FLAGS_SET(p, flags) ((p)->max_heap_size = flags | \ + ((p)->max_heap_size & ~0x3)) +# define MAX_HEAP_SIZE_KILL 1 +# define MAX_HEAP_SIZE_LOG 2 + struct process { ErtsPTabElementCommon common; /* *Need* to be first in struct */ @@ -1035,6 +1060,7 @@ struct process { Eterm *old_hend; /* Heap pointers for generational GC. */ Eterm *old_htop; Eterm *old_heap; + Uint max_heap_size; /* Maximum size of heap (in words). */ Uint16 gen_gcs; /* Number of (minor) generational GCs. */ Uint16 max_gen_gcs; /* Max minor gen GCs before fullsweep. */ ErlOffHeap off_heap; /* Off-heap data updated by copy_struct(). */ @@ -1183,7 +1209,10 @@ void erts_check_for_holes(Process* p); #define ERTS_PSFLG_DIRTY_CPU_PROC ERTS_PSFLG_BIT(20) #define ERTS_PSFLG_DIRTY_IO_PROC ERTS_PSFLG_BIT(21) #define ERTS_PSFLG_DIRTY_ACTIVE_SYS ERTS_PSFLG_BIT(22) -#define ERTS_PSFLG_MAX (ERTS_PSFLGS_ZERO_BIT_OFFSET + 22) +#define ERTS_PSFLG_DIRTY_RUNNING ERTS_PSFLG_BIT(23) +#define ERTS_PSFLG_DIRTY_RUNNING_SYS ERTS_PSFLG_BIT(24) + +#define ERTS_PSFLG_MAX (ERTS_PSFLGS_ZERO_BIT_OFFSET + 24) #define ERTS_PSFLGS_DIRTY_WORK (ERTS_PSFLG_DIRTY_CPU_PROC \ | ERTS_PSFLG_DIRTY_IO_PROC \ @@ -1194,6 +1223,11 @@ void erts_check_for_holes(Process* p); | ERTS_PSFLG_IN_PRQ_NORMAL \ | ERTS_PSFLG_IN_PRQ_LOW) +#define ERTS_PSFLGS_VOLATILE_HEAP (ERTS_PSFLG_EXITING \ + | ERTS_PSFLG_PENDING_EXIT \ + | ERTS_PSFLG_DIRTY_RUNNING \ + | ERTS_PSFLG_DIRTY_RUNNING_SYS) + #define ERTS_PSFLGS_GET_ACT_PRIO(PSFLGS) \ (((PSFLGS) >> ERTS_PSFLGS_ACT_PRIO_OFFSET) & ERTS_PSFLGS_PRIO_MASK) #define ERTS_PSFLGS_GET_USR_PRIO(PSFLGS) \ @@ -1235,6 +1269,7 @@ void erts_check_for_holes(Process* p); * Static flags that do not change after process creation. */ #define ERTS_STC_FLG_SYSTEM_PROC (((Uint32) 1) << 0) +#define ERTS_STC_FLG_SHADOW_PROC (((Uint32) 1) << 1) /* The sequential tracing token is a tuple of size 5: * @@ -1285,6 +1320,8 @@ typedef struct { Uint min_vheap_size; /* Minimum virtual heap size */ int priority; /* Priority for process. */ Uint16 max_gen_gcs; /* Maximum number of gen GCs before fullsweep. */ + Uint max_heap_size; /* Maximum heap size in words */ + Uint max_heap_flags; /* Maximum heap flags (kill | log) */ int scheduler; } ErlSpawnOpts; @@ -1302,7 +1339,7 @@ ERTS_GLB_INLINE void erts_heap_frag_shrink(Process* p, Eterm* hp) { ErlHeapFragment* hf = MBUF(p); - ASSERT(hf!=NULL && (hp - hf->mem < (unsigned long)hf->alloc_size)); + ASSERT(hf!=NULL && (hp - hf->mem < hf->alloc_size)); hf->used_size = hp - hf->mem; } @@ -1363,6 +1400,7 @@ extern int erts_system_profile_ts_type; #define F_SCHDLR_ONLN_WAITQ (1 << 17) /* Process enqueued waiting to change schedulers online */ #define F_HAVE_BLCKD_NMSCHED (1 << 18) /* Process has blocked normal multi-scheduling */ #define F_HIPE_MODE (1 << 19) +#define F_DELAYED_DEL_PROC (1 << 20) /* Delay delete process (dirty proc exit case) */ /* * F_DISABLE_GC and F_DELAY_GC are similar. Both will prevent @@ -1783,7 +1821,8 @@ erts_aint32_t erts_set_aux_work_timeout(int, erts_aint32_t, int); void erts_sched_notify_check_cpu_bind(void); Uint erts_active_schedulers(void); void erts_init_process(int, int, int); -Eterm erts_process_status(Process *, ErtsProcLocks, Process *, Eterm); +Eterm erts_process_state2status(erts_aint32_t); +Eterm erts_process_status(Process *, Eterm); Uint erts_run_queues_len(Uint *, int, int); void erts_add_to_runq(Process *); Eterm erts_bound_schedulers_term(Process *c_p); @@ -1860,19 +1899,11 @@ int erts_debug_wait_completed(Process *c_p, int flags); Uint erts_process_memory(Process *c_p, int incl_msg_inq); -#ifdef ERTS_SMP -# define ERTS_GET_SCHEDULER_DATA_FROM_PROC(PROC) ((PROC)->scheduler_data) -# define ERTS_PROC_GET_SCHDATA(PROC) ((PROC)->scheduler_data) -#else -# define ERTS_GET_SCHEDULER_DATA_FROM_PROC(PROC) (erts_scheduler_data) -# define ERTS_PROC_GET_SCHDATA(PROC) (erts_scheduler_data) -#endif - #ifdef ERTS_DO_VERIFY_UNUSED_TEMP_ALLOC # define ERTS_VERIFY_UNUSED_TEMP_ALLOC(P) \ do { \ ErtsSchedulerData *esdp__ = ((P) \ - ? ERTS_PROC_GET_SCHDATA((Process *) (P)) \ + ? erts_proc_sched_data((Process *) (P)) \ : erts_get_scheduler_data()); \ if (esdp__ && !ERTS_SCHEDULER_IS_DIRTY(esdp__)) \ esdp__->verify_unused_temp_alloc( \ @@ -1965,12 +1996,15 @@ erts_psd_set(Process *p, int ix, void *data) ErtsPSD *psd; #if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) ErtsProcLocks locks = erts_proc_lc_my_proc_locks(p); - if (ERTS_LC_PSD_ANY_LOCK == erts_psd_required_locks[ix].set_locks) - ERTS_SMP_LC_ASSERT(locks || erts_thr_progress_is_blocking()); - else { - locks &= erts_psd_required_locks[ix].set_locks; - ERTS_SMP_LC_ASSERT(erts_psd_required_locks[ix].set_locks == locks - || erts_thr_progress_is_blocking()); + erts_aint32_t state = state = erts_smp_atomic32_read_nob(&p->state); + if (!(state & ERTS_PSFLG_FREE)) { + if (ERTS_LC_PSD_ANY_LOCK == erts_psd_required_locks[ix].set_locks) + ERTS_SMP_LC_ASSERT(locks || erts_thr_progress_is_blocking()); + else { + locks &= erts_psd_required_locks[ix].set_locks; + ERTS_SMP_LC_ASSERT(erts_psd_required_locks[ix].set_locks == locks + || erts_thr_progress_is_blocking()); + } } #endif psd = (ErtsPSD *) erts_smp_atomic_read_nob(&p->psd); @@ -2027,6 +2061,13 @@ erts_psd_set(Process *p, int ix, void *data) ((struct saved_calls *) erts_psd_set((P), ERTS_PSD_SUSPENDED_SAVED_CALLS_BUF, (void *) (SCB))) #endif +#ifdef ERTS_DIRTY_SCHEDULERS +#define ERTS_PROC_GET_DIRTY_CPU_START(P) \ + ((void *) erts_psd_get((P), ERTS_PSD_DIRTY_CPU_START)) +#define ERTS_PROC_SET_DIRTY_CPU_START(P, DCS) \ + ((void *) erts_psd_set((P), ERTS_PSD_DIRTY_CPU_START, (void *) (DCS))) +#endif + ERTS_GLB_INLINE Eterm erts_proc_get_error_handler(Process *p); ERTS_GLB_INLINE Eterm erts_proc_set_error_handler(Process *p, Eterm handler); @@ -2169,6 +2210,7 @@ erts_check_emigration_need(ErtsRunQueue *c_rq, int prio) #endif +ERTS_GLB_INLINE ErtsSchedulerData *erts_proc_sched_data(Process *c_p); ERTS_GLB_INLINE int erts_is_scheduler_bound(ErtsSchedulerData *esdp); ERTS_GLB_INLINE Process *erts_get_current_process(void); ERTS_GLB_INLINE Eterm erts_get_current_pid(void); @@ -2202,6 +2244,31 @@ ERTS_GLB_INLINE void erts_shrink_message_heap(ErtsMessage **msgpp, Process *pp, #if ERTS_GLB_INLINE_INCL_FUNC_DEF ERTS_GLB_INLINE +ErtsSchedulerData *erts_proc_sched_data(Process *c_p) +{ + ErtsSchedulerData *esdp; + ASSERT(c_p); +#if !defined(ERTS_SMP) + esdp = erts_get_scheduler_data(); +#else + esdp = c_p->scheduler_data; +# if defined(ERTS_DIRTY_SCHEDULERS) + if (esdp) { + ASSERT(esdp == erts_get_scheduler_data()); + ASSERT(!ERTS_SCHEDULER_IS_DIRTY(esdp)); + } + else { + esdp = erts_get_scheduler_data(); + ASSERT(esdp); + ASSERT(ERTS_SCHEDULER_IS_DIRTY(esdp)); + } +# endif +#endif + ASSERT(esdp); + return esdp; +} + +ERTS_GLB_INLINE int erts_is_scheduler_bound(ErtsSchedulerData *esdp) { if (!esdp) @@ -2416,7 +2483,7 @@ ERTS_GLB_INLINE ErtsAtomCacheMap * erts_get_atom_cache_map(Process *c_p) { ErtsSchedulerData *esdp = (c_p - ? ERTS_PROC_GET_SCHDATA(c_p) + ? erts_proc_sched_data(c_p) : erts_get_scheduler_data()); ASSERT(esdp); return &esdp->atom_cache_map; diff --git a/erts/emulator/beam/erl_process_dump.c b/erts/emulator/beam/erl_process_dump.c index fa76773cac..eeaa9a569c 100644 --- a/erts/emulator/beam/erl_process_dump.c +++ b/erts/emulator/beam/erl_process_dump.c @@ -568,23 +568,21 @@ dump_externally(int to, void *to_arg, Eterm term) } } -void erts_dump_process_state(int to, void *to_arg, erts_aint32_t psflg) { - if (psflg & ERTS_PSFLG_FREE) - erts_print(to, to_arg, "Non Existing\n"); /* Should never happen */ - else if (psflg & ERTS_PSFLG_EXITING) - erts_print(to, to_arg, "Exiting\n"); - else if (psflg & ERTS_PSFLG_GC) { - erts_print(to, to_arg, "Garbing\n"); - } - else if (psflg & ERTS_PSFLG_SUSPENDED) - erts_print(to, to_arg, "Suspended\n"); - else if (psflg & ERTS_PSFLG_RUNNING) { - erts_print(to, to_arg, "Running\n"); +void erts_dump_process_state(int to, void *to_arg, erts_aint32_t psflg) +{ + char *s; + switch (erts_process_state2status(psflg)) { + case am_free: s = "Non Existing"; break; /* Should never happen */ + case am_exiting: s = "Exiting"; break; + case am_garbage_collecting: s = "Garbing"; break; + case am_suspended: s = "Suspended"; break; + case am_running: s = "Running"; break; + case am_runnable: s = "Scheduled"; break; + case am_waiting: s = "Waiting"; break; + default: s = "Undefined"; break; /* Should never happen */ } - else if (psflg & ERTS_PSFLG_ACTIVE) - erts_print(to, to_arg, "Scheduled\n"); - else - erts_print(to, to_arg, "Waiting\n"); + + erts_print(to, to_arg, "%s\n", s); } void @@ -668,6 +666,10 @@ erts_dump_extended_process_state(int to, void *to_arg, erts_aint32_t psflg) { erts_print(to, to_arg, "DIRTY_IO_PROC"); break; case ERTS_PSFLG_DIRTY_ACTIVE_SYS: erts_print(to, to_arg, "DIRTY_ACTIVE_SYS"); break; + case ERTS_PSFLG_DIRTY_RUNNING: + erts_print(to, to_arg, "DIRTY_RUNNING"); break; + case ERTS_PSFLG_DIRTY_RUNNING_SYS: + erts_print(to, to_arg, "DIRTY_RUNNING_SYS"); break; default: erts_print(to, to_arg, "UNKNOWN(%d)", chk); break; } diff --git a/erts/emulator/beam/erl_time_sup.c b/erts/emulator/beam/erl_time_sup.c index fadbd704bd..9e37106b88 100644 --- a/erts/emulator/beam/erl_time_sup.c +++ b/erts/emulator/beam/erl_time_sup.c @@ -2348,7 +2348,7 @@ erts_napi_convert_time_unit(ErtsMonotonicTime val, int from, int to) BIF_RETTYPE monotonic_time_0(BIF_ALIST_0) { ErtsMonotonicTime mtime = time_sup.r.o.get_time(); - update_last_mtime(ERTS_PROC_GET_SCHDATA(BIF_P), mtime); + update_last_mtime(erts_proc_sched_data(BIF_P), mtime); mtime += ERTS_MONOTONIC_OFFSET_NATIVE; BIF_RET(make_time_val(BIF_P, mtime)); } @@ -2356,7 +2356,7 @@ BIF_RETTYPE monotonic_time_0(BIF_ALIST_0) BIF_RETTYPE monotonic_time_1(BIF_ALIST_1) { ErtsMonotonicTime mtime = time_sup.r.o.get_time(); - update_last_mtime(ERTS_PROC_GET_SCHDATA(BIF_P), mtime); + update_last_mtime(erts_proc_sched_data(BIF_P), mtime); BIF_RET(time_unit_conversion(BIF_P, BIF_ARG_1, mtime, 1)); } @@ -2365,7 +2365,7 @@ BIF_RETTYPE system_time_0(BIF_ALIST_0) ErtsMonotonicTime mtime, offset; mtime = time_sup.r.o.get_time(); offset = get_time_offset(); - update_last_mtime(ERTS_PROC_GET_SCHDATA(BIF_P), mtime); + update_last_mtime(erts_proc_sched_data(BIF_P), mtime); BIF_RET(make_time_val(BIF_P, mtime + offset)); } @@ -2374,7 +2374,7 @@ BIF_RETTYPE system_time_1(BIF_ALIST_0) ErtsMonotonicTime mtime, offset; mtime = time_sup.r.o.get_time(); offset = get_time_offset(); - update_last_mtime(ERTS_PROC_GET_SCHDATA(BIF_P), mtime); + update_last_mtime(erts_proc_sched_data(BIF_P), mtime); BIF_RET(time_unit_conversion(BIF_P, BIF_ARG_1, mtime + offset, 0)); } @@ -2404,7 +2404,7 @@ BIF_RETTYPE timestamp_0(BIF_ALIST_0) mtime = time_sup.r.o.get_time(); offset = get_time_offset(); - update_last_mtime(ERTS_PROC_GET_SCHDATA(BIF_P), mtime); + update_last_mtime(erts_proc_sched_data(BIF_P), mtime); make_timestamp_value(&mega_sec, &sec, µ_sec, mtime, offset); diff --git a/erts/emulator/beam/erl_trace.c b/erts/emulator/beam/erl_trace.c index 1c0fc0a11f..ca001fc156 100644 --- a/erts/emulator/beam/erl_trace.c +++ b/erts/emulator/beam/erl_trace.c @@ -396,7 +396,7 @@ send_to_tracer_nif(Process *c_p, ErtsPTabElementCommon *t_p, Eterm tag, Eterm msg, Eterm extra, Eterm pam_result); static ERTS_INLINE Eterm -call_enabled_tracer(Process *c_p, const ErtsTracer tracer, +call_enabled_tracer(const ErtsTracer tracer, ErtsTracerNif **tnif_ref, enum ErtsTracerOpt topt, Eterm tag, Eterm t_p_id); @@ -459,8 +459,7 @@ erts_set_system_seq_tracer(Process *c_p, ErtsProcLocks c_p_locks, ErtsTracer new if (!ERTS_TRACER_IS_NIL(new)) { Eterm nif_result = call_enabled_tracer( - NULL, new, NULL, - TRACE_FUN_ENABLED, am_trace_status, am_undefined); + new, NULL, TRACE_FUN_ENABLED, am_trace_status, am_undefined); switch (nif_result) { case am_trace: break; default: @@ -492,7 +491,7 @@ erts_get_system_seq_tracer(void) erts_smp_rwmtx_runlock(&sys_trace_rwmtx); if (st != erts_tracer_nil && - call_enabled_tracer(NULL, st, NULL, TRACE_FUN_ENABLED, + call_enabled_tracer(st, NULL, TRACE_FUN_ENABLED, am_trace_status, am_undefined) == am_remove) { erts_set_system_seq_tracer(NULL, 0, erts_tracer_nil); st = erts_tracer_nil; @@ -513,7 +512,7 @@ get_default_tracing(Uint *flagsp, ErtsTracer *tracerp, *default_trace_flags &= ~TRACEE_FLAGS; } else { Eterm nif_res; - nif_res = call_enabled_tracer(NULL, *default_tracer, + nif_res = call_enabled_tracer(*default_tracer, NULL, TRACE_FUN_ENABLED, am_trace_status, am_undefined); switch (nif_res) { @@ -915,8 +914,8 @@ seq_trace_update_send(Process *p) ASSERT((is_tuple(SEQ_TRACE_TOKEN(p)) || is_nil(SEQ_TRACE_TOKEN(p)))); if (have_no_seqtrace(SEQ_TRACE_TOKEN(p)) || (seq_tracer != NIL && - call_enabled_tracer(NULL, seq_tracer, NULL, - TRACE_FUN_ENABLED, am_trace_status, + call_enabled_tracer(seq_tracer, NULL, + TRACE_FUN_ENABLED, am_seq_trace, p ? p->common.id : am_undefined) != am_trace) #ifdef USE_VM_PROBES || (SEQ_TRACE_TOKEN(p) == am_have_dt_utag) @@ -963,9 +962,9 @@ seq_trace_output_generic(Eterm token, Eterm msg, Uint type, ASSERT(is_tuple(token) || is_nil(token)); if (token == NIL || (process && ERTS_TRACE_FLAGS(process) & F_SENSITIVE) || ERTS_TRACER_IS_NIL(seq_tracer) || - call_enabled_tracer(NULL, seq_tracer, + call_enabled_tracer(seq_tracer, NULL, TRACE_FUN_ENABLED, - am_trace_status, + am_seq_trace, process ? process->common.id : am_undefined) != am_trace) { return; } @@ -1186,7 +1185,11 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, * use process flags */ tracee_flags = &ERTS_TRACE_FLAGS(p); + /* Is is not ideal at all to call this check twice, + it should be optimized so that only one call is made. */ if (!is_tracer_enabled(p, ERTS_PROC_LOCK_MAIN, &p->common, &tnif, + TRACE_FUN_ENABLED, am_trace_status) + || !is_tracer_enabled(p, ERTS_PROC_LOCK_MAIN, &p->common, &tnif, TRACE_FUN_E_CALL, am_call)) { return 0; } @@ -1202,13 +1205,21 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, } meta_flags = F_TRACE_CALLS | F_NOW_TS; tracee_flags = &meta_flags; - switch (call_enabled_tracer(p, *tracer, - &tnif, TRACE_FUN_T_CALL, - am_call, p->common.id)) { + switch (call_enabled_tracer(*tracer, + &tnif, TRACE_FUN_ENABLED, + am_trace_status, p->common.id)) { default: case am_remove: *tracer = erts_tracer_nil; case am_discard: return 0; - case am_trace: break; + case am_trace: + switch (call_enabled_tracer(*tracer, + &tnif, TRACE_FUN_T_CALL, + am_call, p->common.id)) { + default: + case am_discard: return 0; + case am_trace: break; + } + break; } } @@ -1346,9 +1357,9 @@ trace_proc(Process *c_p, ErtsProcLocks c_p_locks, Process *t_p, Eterm what, Eterm data) { ErtsTracerNif *tnif = NULL; - if (is_tracer_enabled(c_p, c_p_locks, &t_p->common, &tnif, + if (is_tracer_enabled(NULL, 0, &t_p->common, &tnif, TRACE_FUN_E_PROCS, what)) - send_to_tracer_nif(c_p, &t_p->common, t_p->common.id, tnif, TRACE_FUN_T_PROCS, + send_to_tracer_nif(NULL, &t_p->common, t_p->common.id, tnif, TRACE_FUN_T_PROCS, what, data, THE_NON_VALUE, am_true); } @@ -1365,16 +1376,15 @@ trace_proc_spawn(Process *p, Eterm what, Eterm pid, Eterm mod, Eterm func, Eterm args) { ErtsTracerNif *tnif = NULL; - if (is_tracer_enabled(p, ERTS_PROC_LOCKS_ALL & - ~(ERTS_PROC_LOCK_STATUS|ERTS_PROC_LOCK_TRACE), - &p->common, &tnif, TRACE_FUN_E_PROCS, what)) { + if (is_tracer_enabled(NULL, 0, + &p->common, &tnif, TRACE_FUN_E_PROCS, what)) { Eterm mfa; Eterm* hp; hp = HAlloc(p, 4); mfa = TUPLE3(hp, mod, func, args); hp += 4; - send_to_tracer_nif(p, &p->common, p->common.id, tnif, TRACE_FUN_T_PROCS, + send_to_tracer_nif(NULL, &p->common, p->common.id, tnif, TRACE_FUN_T_PROCS, what, pid, mfa, am_true); } } @@ -1409,25 +1419,28 @@ void save_calls(Process *p, Export *e) * are all small (atomic) integers. */ void -trace_gc(Process *p, Eterm what, Uint size) +trace_gc(Process *p, Eterm what, Uint size, Eterm msg) { ErtsTracerNif *tnif = NULL; Eterm* hp; - Eterm msg = NIL; Uint sz = 0; Eterm tup; - if (is_tracer_enabled(p, ERTS_PROC_LOCK_MAIN, &p->common, &tnif, TRACE_FUN_E_GC, what)) { + if (is_tracer_enabled(p, ERTS_PROC_LOCK_MAIN, &p->common, &tnif, + TRACE_FUN_E_GC, what)) { + + if (is_non_value(msg)) { - (void) erts_process_gc_info(p, &sz, NULL); - hp = HAlloc(p, sz + 3 + 2); + (void) erts_process_gc_info(p, &sz, NULL, 0, 0); + hp = HAlloc(p, sz + 3 + 2); - msg = erts_process_gc_info(p, NULL, &hp); - tup = TUPLE2(hp, am_wordsize, make_small(size)); hp += 3; - msg = CONS(hp, tup, msg); hp += 2; + msg = erts_process_gc_info(p, NULL, &hp, 0, 0); + tup = TUPLE2(hp, am_wordsize, make_small(size)); hp += 3; + msg = CONS(hp, tup, msg); hp += 2; + } send_to_tracer_nif(p, &p->common, p->common.id, tnif, TRACE_FUN_T_GC, - what, msg, am_undefined, am_true); + what, msg, THE_NON_VALUE, am_true); } } @@ -2896,23 +2909,28 @@ send_to_tracer_nif(Process *c_p, ErtsPTabElementCommon *t_p, } static ERTS_INLINE Eterm -call_enabled_tracer(Process *c_p, const ErtsTracer tracer, +call_enabled_tracer(const ErtsTracer tracer, ErtsTracerNif **tnif_ret, enum ErtsTracerOpt topt, Eterm tag, Eterm t_p_id) { ErtsTracerNif *tnif = lookup_tracer_nif(tracer); if (tnif) { - Eterm argv[] = {tag, ERTS_TRACER_STATE(tracer), t_p_id}; + Eterm argv[] = {tag, ERTS_TRACER_STATE(tracer), t_p_id}, + ret; topt = (tnif->tracers[topt].cb) ? topt : TRACE_FUN_ENABLED; ASSERT(topt < NIF_TRACER_TYPES); ASSERT(tnif->tracers[topt].cb != NULL); if (tnif_ret) *tnif_ret = tnif; - return erts_nif_call_function(c_p, NULL, tnif->nif_mod, - tnif->tracers[topt].cb, - tnif->tracers[topt].arity, - argv); + ret = erts_nif_call_function(NULL, NULL, tnif->nif_mod, + tnif->tracers[topt].cb, + tnif->tracers[topt].arity, + argv); + if (tag == am_trace_status && ret != am_remove) + return am_trace; + ASSERT(tag == am_trace_status || ret != am_remove); + return ret; } - return am_remove; + return tag == am_trace_status ? am_remove : am_discard; } static int @@ -2937,12 +2955,12 @@ is_tracer_enabled(Process* c_p, ErtsProcLocks c_p_locks, } #endif - nif_result = call_enabled_tracer(c_p, t_p->tracer, tnif_ret, topt, tag, t_p->id); + nif_result = call_enabled_tracer(t_p->tracer, tnif_ret, topt, tag, t_p->id); switch (nif_result) { case am_discard: return 0; case am_trace: return 1; case THE_NON_VALUE: - case am_remove: break; + case am_remove: ASSERT(tag == am_trace_status); break; default: /* only am_remove should be returned, but if something else is returned we fall-through @@ -2973,19 +2991,14 @@ is_tracer_enabled(Process* c_p, ErtsProcLocks c_p_locks, return 0; } -int erts_is_tracer_proc_enabled(Process* c_p, ErtsProcLocks c_p_locks, - ErtsPTabElementCommon *t_p, Eterm type) -{ - return is_tracer_enabled(c_p, c_p_locks, t_p, NULL, TRACE_FUN_ENABLED, am_trace_status); -} - -int erts_is_tracer_enabled(Process *c_p, const ErtsTracer tracer) +int erts_is_tracer_enabled(const ErtsTracer tracer, ErtsPTabElementCommon *t_p) { ErtsTracerNif *tnif = lookup_tracer_nif(tracer); if (tnif) { - Eterm nif_result = call_enabled_tracer(c_p, tracer, &tnif, - TRACE_FUN_ENABLED, am_trace_status, - c_p->common.id); + Eterm nif_result = call_enabled_tracer(tracer, &tnif, + TRACE_FUN_ENABLED, + am_trace_status, + t_p->id); switch (nif_result) { case am_discard: case am_trace: return 1; @@ -2996,6 +3009,20 @@ int erts_is_tracer_enabled(Process *c_p, const ErtsTracer tracer) return 0; } +int erts_is_tracer_proc_enabled(Process* c_p, ErtsProcLocks c_p_locks, + ErtsPTabElementCommon *t_p) +{ + return is_tracer_enabled(c_p, c_p_locks, t_p, NULL, TRACE_FUN_ENABLED, + am_trace_status); +} + +int erts_is_tracer_proc_enabled_send(Process* c_p, ErtsProcLocks c_p_locks, + ErtsPTabElementCommon *t_p) +{ + return is_tracer_enabled(c_p, c_p_locks, t_p, NULL, TRACE_FUN_T_SEND, am_send); +} + + void erts_tracer_replace(ErtsPTabElementCommon *t_p, const ErtsTracer tracer) { #if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) diff --git a/erts/emulator/beam/erl_trace.h b/erts/emulator/beam/erl_trace.h index de4beadb7e..0095d4386b 100644 --- a/erts/emulator/beam/erl_trace.h +++ b/erts/emulator/beam/erl_trace.h @@ -112,7 +112,7 @@ void trace_sched(Process*, ErtsProcLocks, Eterm); void trace_proc(Process*, ErtsProcLocks, Process*, Eterm, Eterm); void trace_proc_spawn(Process*, Eterm what, Eterm pid, Eterm mod, Eterm func, Eterm args); void save_calls(Process *p, Export *); -void trace_gc(Process *p, Eterm what, Uint size); +void trace_gc(Process *p, Eterm what, Uint size, Eterm msg); /* port tracing */ void trace_virtual_sched(Process*, ErtsProcLocks, Eterm); void trace_sched_ports(Port *pp, Eterm); @@ -193,8 +193,10 @@ int erts_finish_breakpointing(void); /* Nif tracer functions */ int erts_is_tracer_proc_enabled(Process *c_p, ErtsProcLocks c_p_locks, - ErtsPTabElementCommon *t_p, Eterm type); -int erts_is_tracer_enabled(Process *c_p, const ErtsTracer tracer); + ErtsPTabElementCommon *t_p); +int erts_is_tracer_proc_enabled_send(Process* c_p, ErtsProcLocks c_p_locks, + ErtsPTabElementCommon *t_p); +int erts_is_tracer_enabled(const ErtsTracer tracer, ErtsPTabElementCommon *t_p); Eterm erts_tracer_to_term(Process *p, ErtsTracer tracer); ErtsTracer erts_term_to_tracer(Eterm prefix, Eterm term); void erts_tracer_replace(ErtsPTabElementCommon *t_p, @@ -224,9 +226,4 @@ ERTS_DECLARE_DUMMY(erts_tracer_nil) = NIL; #define ERTS_TRACER_FROM_ETERM(termp) \ ((ErtsTracer*)(termp)) -#define ERTS_TRACER_PROC_IS_ENABLED(PROC) \ - (!ERTS_TRACER_IS_NIL(ERTS_TRACER(PROC)) \ - && erts_is_tracer_proc_enabled(PROC, ERTS_PROC_LOCK_MAIN, \ - &(PROC)->common, am_trace_status)) - #endif /* ERL_TRACE_H__ */ diff --git a/erts/emulator/beam/erl_unicode.c b/erts/emulator/beam/erl_unicode.c index 3c3536c021..bd5e1482fb 100644 --- a/erts/emulator/beam/erl_unicode.c +++ b/erts/emulator/beam/erl_unicode.c @@ -55,7 +55,7 @@ static BIF_RETTYPE finalize_list_to_list(Process *p, Uint num_processed_bytes, Uint num_bytes_to_process, Uint num_resulting_chars, - int state, int left, + int state, Sint left, Eterm tail); static BIF_RETTYPE characters_to_utf8_trap(BIF_ALIST_3); static BIF_RETTYPE characters_to_list_trap_1(BIF_ALIST_3); @@ -173,12 +173,13 @@ static ERTS_INLINE int allowed_iterations(Process *p) else return tmp; } -static ERTS_INLINE int cost_to_proc(Process *p, int cost) + +static ERTS_INLINE void cost_to_proc(Process *p, Sint cost) { - int x = (cost / LOOP_FACTOR); + Sint x = (cost / LOOP_FACTOR); BUMP_REDS(p,x); - return x; } + static ERTS_INLINE int simple_loops_to_common(int cost) { int factor = (LOOP_FACTOR_SIMPLE / LOOP_FACTOR); @@ -243,14 +244,15 @@ static int utf8_len(byte first) return -1; } -static int copy_utf8_bin(byte *target, byte *source, Uint size, - byte *leftover, int *num_leftovers, - byte **err_pos, Uint *characters) { - int copied = 0; +static Uint copy_utf8_bin(byte *target, byte *source, Uint size, + byte *leftover, int *num_leftovers, + byte **err_pos, Uint *characters) +{ + Uint copied = 0; if (leftover != NULL && *num_leftovers) { int need = utf8_len(leftover[0]); int from_source = need - (*num_leftovers); - int c; + Uint c; byte *tmp_err_pos = NULL; ASSERT(need > 0); ASSERT(from_source > 0); @@ -502,8 +504,8 @@ L_Again: /* Restart with sublist, old listend was pushed on stack */ } -static Eterm do_build_utf8(Process *p, Eterm ioterm, int *left, int latin1, - byte *target, int *pos, Uint *characters, int *err, +static Eterm do_build_utf8(Process *p, Eterm ioterm, Sint *left, int latin1, + byte *target, Uint *pos, Uint *characters, int *err, byte *leftover, int *num_leftovers) { int c; @@ -573,7 +575,7 @@ static Eterm do_build_utf8(Process *p, Eterm ioterm, int *left, int latin1, } if (!latin1) { - int num; + Uint num; byte *err_pos = NULL; num = copy_utf8_bin(target + (*pos), bytes, size, leftover, num_leftovers,&err_pos,characters); @@ -804,7 +806,7 @@ static int check_leftovers(byte *source, int size) -static BIF_RETTYPE build_utf8_return(Process *p,Eterm bin,int pos, +static BIF_RETTYPE build_utf8_return(Process *p,Eterm bin,Uint pos, Eterm rest_term,int err, byte *leftover,int num_leftovers,Eterm latin1) { @@ -859,8 +861,8 @@ static BIF_RETTYPE characters_to_utf8_trap(BIF_ALIST_3) #endif byte* bytes; Eterm rest_term; - int left, sleft; - int pos; + Sint left, sleft; + Uint pos; int err; byte leftover[4]; /* used for temp buffer too, otherwise 3 bytes would have been enough */ @@ -874,7 +876,7 @@ static BIF_RETTYPE characters_to_utf8_trap(BIF_ALIST_3) real_bin = binary_val(BIF_ARG_1); ASSERT(*real_bin == HEADER_PROC_BIN); #endif - pos = (int) binary_size(BIF_ARG_1); + pos = binary_size(BIF_ARG_1); bytes = binary_bytes(BIF_ARG_1); sleft = left = allowed_iterations(BIF_P); err = 0; @@ -934,9 +936,9 @@ BIF_RETTYPE unicode_characters_to_binary_2(BIF_ALIST_2) int latin1; Eterm bin; byte *bytes; - int pos; + Uint pos; int err; - int left, sleft; + Sint left, sleft; Eterm rest_term, subject; byte leftover[4]; /* used for temp buffer too, o therwise 3 bytes would have been enough */ @@ -999,7 +1001,7 @@ BIF_RETTYPE unicode_characters_to_binary_2(BIF_ALIST_2) byte *t = NULL; Uint sz = binary_size(bin); byte *by = erts_get_aligned_binary_bytes(bin,&t); - int i; + Uint i; erts_printf("<<"); for (i = 0;i < sz; ++i) { erts_printf((i == sz -1) ? "0x%X" : "0x%X, ", (unsigned) by[i]); @@ -1007,7 +1009,7 @@ BIF_RETTYPE unicode_characters_to_binary_2(BIF_ALIST_2) erts_printf(">>: "); erts_free_aligned_binary_bytes(t); } - erts_printf("%d - %d = %d\n",sleft,left,sleft - left); + erts_printf("%ld - %ld = %ld\n", sleft, left, sleft - left); } #endif cost_to_proc(BIF_P, sleft - left); @@ -1015,10 +1017,10 @@ BIF_RETTYPE unicode_characters_to_binary_2(BIF_ALIST_2) leftover,num_leftovers,BIF_ARG_2); } -static BIF_RETTYPE build_list_return(Process *p, byte *bytes, int pos, Uint characters, +static BIF_RETTYPE build_list_return(Process *p, byte *bytes, Uint pos, Uint characters, Eterm rest_term, int err, byte *leftover, int num_leftovers, - Eterm latin1, int left) + Eterm latin1, Sint left) { Eterm *hp; @@ -1070,11 +1072,11 @@ static BIF_RETTYPE characters_to_list_trap_1(BIF_ALIST_3) { RestartContext *rc; byte* bytes; - int pos; + Uint pos; Uint characters; int err; Eterm rest_term; - int left, sleft; + Sint left, sleft; int latin1 = 0; byte leftover[4]; /* used for temp buffer too, @@ -1107,9 +1109,9 @@ BIF_RETTYPE unicode_characters_to_list_2(BIF_ALIST_2) int latin1; Uint characters = 0; byte *bytes; - int pos; + Uint pos; int err; - int left, sleft; + Sint left, sleft; Eterm rest_term; byte leftover[4]; /* used for temp buffer too, o therwise 3 bytes would have been enough */ @@ -1541,7 +1543,7 @@ static BIF_RETTYPE finalize_list_to_list(Process *p, Uint num_processed_bytes, Uint num_bytes_to_process, Uint num_resulting_chars, - int state, int left, + int state, Sint left, Eterm tail) { Uint num_built; /* characters */ @@ -2016,7 +2018,7 @@ char *erts_convert_filename_to_encoding(Eterm name, char *statbuf, size_t statbu ++need; } if (used) - *used = (Sint) need; + *used = need; if (need+extra > statbuf_size) { name_buf = (char *) erts_alloc(alloc_type, need+extra); } else { diff --git a/erts/emulator/beam/erl_vm.h b/erts/emulator/beam/erl_vm.h index f3c54de214..f97716d030 100644 --- a/erts/emulator/beam/erl_vm.h +++ b/erts/emulator/beam/erl_vm.h @@ -50,6 +50,7 @@ #define H_DEFAULT_SIZE 233 /* default (heap + stack) min size */ #define VH_DEFAULT_SIZE 32768 /* default virtual (bin) heap min size (words) */ +#define H_DEFAULT_MAX_SIZE 0 /* default max heap size is off */ #define CP_SIZE 1 @@ -160,6 +161,8 @@ extern int num_instructions; /* Number of instruction in opc[]. */ extern int H_MIN_SIZE; /* minimum (heap + stack) */ extern int BIN_VH_MIN_SIZE; /* minimum virtual (bin) heap */ +extern int H_MAX_SIZE; /* maximum (heap + stack) */ +extern int H_MAX_FLAGS; /* maximum heap flags */ extern int erts_atom_table_size;/* Atom table size */ extern int erts_pd_initial_size;/* Initial Process dictionary table size */ diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h index 550db95ba8..b76b9cd874 100644 --- a/erts/emulator/beam/global.h +++ b/erts/emulator/beam/global.h @@ -57,6 +57,7 @@ struct enif_environment_t /* ErlNifEnv */ struct enif_tmp_obj_t* tmp_obj_list; int exception_thrown; /* boolean */ Process *tracee; + int exiting; /* boolean (dirty nifs might return in exiting state) */ }; extern void erts_pre_nif(struct enif_environment_t*, Process*, struct erl_module_nif*, Process* tracee); @@ -946,7 +947,7 @@ ERTS_GLB_INLINE Eterm erts_equeue_get(ErtsEQueue *q) { void erts_emasculate_writable_binary(ProcBin* pb); Eterm erts_new_heap_binary(Process *p, byte *buf, int len, byte** datap); -Eterm erts_new_mso_binary(Process*, byte*, int); +Eterm erts_new_mso_binary(Process*, byte*, Uint); Eterm new_binary(Process*, byte*, Uint); Eterm erts_realloc_binary(Eterm bin, size_t size); diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index b9f6ab04c6..0377f6cb5e 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -2512,7 +2512,7 @@ erts_port_output(Process *c_p, sigdp->flags &= ~ERTS_P2P_SIG_DATA_FLG_NOSUSPEND; else if (async_nosuspend) { ErtsSchedulerData *esdp = (c_p - ? ERTS_PROC_GET_SCHDATA(c_p) + ? erts_proc_sched_data(c_p) : erts_get_scheduler_data()); ASSERT(esdp); ns_pthp = &esdp->nosuspend_port_task_handle; @@ -5140,7 +5140,7 @@ erts_request_io_bytes(Process *c_p) Uint *hp; Eterm ref; Uint32 *refn; - ErtsSchedulerData *esdp = ERTS_PROC_GET_SCHDATA(c_p); + ErtsSchedulerData *esdp = erts_proc_sched_data(c_p); ErtsIOBytesReq *req = erts_alloc(ERTS_ALC_T_IOB_REQ, sizeof(ErtsIOBytesReq)); @@ -5941,7 +5941,7 @@ driver_deliver_term(Port *prt, Eterm to, ErlDrvTermData* data, int len) if (!rp) { if (!prt || !IS_TRACED_FL(prt, F_TRACE_SEND)) goto done; - if (!erts_is_tracer_proc_enabled(NULL, 0, &prt->common, am_send)) + if (!erts_is_tracer_proc_enabled_send(NULL, 0, &prt->common)) goto done; res = -2; @@ -7881,11 +7881,13 @@ driver_system_info(ErlDrvSysInfo *sip, size_t si_size) * (driver version 3.1, NIF version 2.7) */ if (si_size >= ERL_DRV_SYS_INFO_SIZE(dirty_scheduler_support)) { -#if defined(ERL_NIF_DIRTY_SCHEDULER_SUPPORT) && defined(USE_THREADS) - sip->dirty_scheduler_support = 1; + sip->dirty_scheduler_support = +#ifdef ERTS_DIRTY_SCHEDULERS + 1 #else - sip->dirty_scheduler_support = 0; + 0 #endif + ; } } diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index dda3ba565c..f303d4f167 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -34,6 +34,10 @@ (((__GNUC__ << 24) | (__GNUC_MINOR__ << 12) | __GNUC_PATCHLEVEL__) >= (((MAJ) << 24) | ((MIN) << 12) | (PL))) #endif +#if defined(ERTS_DIRTY_SCHEDULERS) && !defined(ERTS_SMP) +# error "Dirty schedulers not supported without smp support" +#endif + #ifdef ERTS_INLINE # ifndef ERTS_CAN_INLINE # define ERTS_CAN_INLINE 1 diff --git a/erts/emulator/hipe/hipe_mode_switch.c b/erts/emulator/hipe/hipe_mode_switch.c index 9ad44b25ac..884331e969 100644 --- a/erts/emulator/hipe/hipe_mode_switch.c +++ b/erts/emulator/hipe/hipe_mode_switch.c @@ -59,6 +59,7 @@ * TODO: check PCB consistency at native BIF calls */ int hipe_modeswitch_debug = 0; +extern BeamInstr beam_exit[]; #define HIPE_DEBUG 0 @@ -173,6 +174,33 @@ void hipe_mode_switch_init(void) make_catch(beam_catches_cons(hipe_beam_pc_throw, BEAM_CATCHES_NIL)); hipe_mfa_info_table_init(); + +#if (defined(__i386__) || defined(__x86_64__)) && defined(__linux__) + /* Verify that the offset of c-p->hipe does not change. + The ErLLVM hipe backend depends on it being in a specific + position. Kostis et al has promised to fix this in upstream + llvm by OTP 20, so it should be possible to remove these asserts + after that. */ + ERTS_CT_ASSERT(sizeof(ErtsPTabElementCommon) == + (sizeof(Eterm) + /* id */ + sizeof(((ErtsPTabElementCommon*)0)->refc) + + sizeof(ErtsTracer) + /* tracer */ + sizeof(Uint) + /* trace_flags */ + sizeof(erts_smp_atomic_t) + /* timer */ + sizeof(((ErtsPTabElementCommon*)0)->u))); + + ERTS_CT_ASSERT(offsetof(Process, hipe) == + (sizeof(ErtsPTabElementCommon) + /* common */ + sizeof(Eterm*) + /* htop */ + sizeof(Eterm*) + /* stop */ + sizeof(Eterm*) + /* heap */ + sizeof(Eterm*) + /* hend */ + sizeof(Uint) + /* heap_sz */ + sizeof(Uint) + /* min_heap_size */ + sizeof(Uint) + /* min_vheap_size */ + sizeof(volatile unsigned long))); /* fp_exception */ +#endif + } void hipe_set_call_trap(Uint *bfun, void *nfun, int is_closure) @@ -509,6 +537,10 @@ Process *hipe_mode_switch(Process *p, unsigned cmd, Eterm reg[]) if (scb) ERTS_PROC_SET_SAVED_CALLS_BUF(p, scb); + /* The process may have died while it was executing, + if so we return out from native code to the interpreter */ + if (erts_smp_atomic32_read_nob(&p->state) & ERTS_PSFLG_EXITING) + p->i = beam_exit; #ifdef DEBUG ASSERT(p->debug_reds_in == reds_in); #endif diff --git a/erts/emulator/nifs/common/erl_tracer_nif.c b/erts/emulator/nifs/common/erl_tracer_nif.c index 8a9a1bf16c..6dddc80607 100644 --- a/erts/emulator/nifs/common/erl_tracer_nif.c +++ b/erts/emulator/nifs/common/erl_tracer_nif.c @@ -70,6 +70,7 @@ ERL_NIF_INIT(erl_tracer, nif_funcs, load, NULL, upgrade, unload) ATOM_DECL(strict_monotonic); \ ATOM_DECL(timestamp); \ ATOM_DECL(trace); \ + ATOM_DECL(trace_status); \ ATOM_DECL(trace_ts); \ ATOM_DECL(true); \ ATOM_DECL(gc_minor_start); \ @@ -118,19 +119,22 @@ static ERL_NIF_TERM enabled(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { ErlNifPid to_pid; ErlNifPort to_port; + ERL_NIF_TERM ret = enif_is_identical(argv[0], atom_trace_status) ? + atom_remove : atom_discard; + ASSERT(argc == 3); if (enif_get_local_pid(env, argv[1], &to_pid)) { if (!enif_is_process_alive(env, &to_pid)) /* tracer is dead so we should remove this trace point */ - return atom_remove; + return ret; } else if (enif_get_local_port(env, argv[1], &to_port)) { if (!enif_is_port_alive(env, &to_port)) /* tracer is dead so we should remove this trace point */ - return atom_remove; + return ret; } else { /* The state was not a pid or a port */ - return atom_remove; + return ret; } /* Only generate trace for when tracer != tracee */ diff --git a/erts/emulator/test/Makefile b/erts/emulator/test/Makefile index de395dfb97..b580211eff 100644 --- a/erts/emulator/test/Makefile +++ b/erts/emulator/test/Makefile @@ -53,6 +53,7 @@ MODULES= \ crypto_SUITE \ ddll_SUITE \ decode_packet_SUITE \ + dirty_nif_SUITE \ distribution_SUITE \ driver_SUITE \ efile_SUITE \ diff --git a/erts/emulator/test/alloc_SUITE.erl b/erts/emulator/test/alloc_SUITE.erl index 1f7b499dcb..84cf4921d3 100644 --- a/erts/emulator/test/alloc_SUITE.erl +++ b/erts/emulator/test/alloc_SUITE.erl @@ -73,16 +73,32 @@ migration(Cfg) -> end. erts_mmap(Config) when is_list(Config) -> - case os:type() of - {unix, _} -> + case {os:type(), mmsc_flags()} of + {{unix,_}, false} -> [erts_mmap_do(Config, SCO, SCRPM, SCRFSD) || SCO <-[true,false], SCRFSD <-[1234,0], SCRPM <- [true,false]]; - {SkipOs,_} -> + {{unix,_}, Flags} -> + {skipped, Flags}; + {{SkipOs,_},_} -> {skipped, lists:flatten(["Not run on " | io_lib:format("~p",[SkipOs])])} end. +%% Check if there are ERL_FLAGS set that will mess up this test case +mmsc_flags() -> + case mmsc_flags("ERL_FLAGS") of + false -> mmsc_flags("ERL_ZFLAGS"); + Flags -> Flags + end. +mmsc_flags(Env) -> + case os:getenv(Env) of + false -> false; + V -> case string:str(V, "+MMsc") of + 0 -> false; + P -> Env ++ "=" ++ string:substr(V, P) + end + end. erts_mmap_do(Config, SCO, SCRPM, SCRFSD) -> %% We use the number of schedulers + 1 * approx main carriers size diff --git a/erts/emulator/test/dirty_nif_SUITE.erl b/erts/emulator/test/dirty_nif_SUITE.erl new file mode 100644 index 0000000000..c3afbc0803 --- /dev/null +++ b/erts/emulator/test/dirty_nif_SUITE.erl @@ -0,0 +1,327 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-module(dirty_nif_SUITE). + +%%-define(line_trace,true). +-define(CHECK(Exp,Got), check(Exp,Got,?LINE)). +%%-define(CHECK(Exp,Got), Exp = Got). + +-include_lib("common_test/include/ct.hrl"). + +-export([all/0, suite/0, + init_per_suite/1, end_per_suite/1, + init_per_testcase/2, end_per_testcase/2, + dirty_nif/1, dirty_nif_send/1, + dirty_nif_exception/1, call_dirty_nif_exception/1, + dirty_scheduler_exit/1, dirty_call_while_terminated/1, + dirty_heap_access/1]). + +-define(nif_stub,nif_stub_error(?LINE)). + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [dirty_nif, + dirty_nif_send, + dirty_nif_exception, + dirty_scheduler_exit, + dirty_call_while_terminated, + dirty_heap_access]. + +init_per_suite(Config) -> + try erlang:system_info(dirty_cpu_schedulers) of + N when is_integer(N), N > 0 -> + case lib_loaded() of + false -> + ok = erlang:load_nif( + filename:join(?config(data_dir, Config), + "dirty_nif_SUITE"), []); + true -> + ok + end, + Config + catch _:_ -> + {skipped, "No dirty scheduler support"} + end. + +end_per_suite(_Config) -> + ok. + +init_per_testcase(Case, Config) -> + [{testcase, Case} | Config]. + +end_per_testcase(_Case, _Config) -> + ok. + +dirty_nif(Config) when is_list(Config) -> + Val1 = 42, + Val2 = "Erlang", + Val3 = list_to_binary([Val2, 0]), + {Val1, Val2, Val3} = call_dirty_nif(Val1, Val2, Val3), + LargeArray = lists:duplicate(1000, ok), + LargeArray = call_dirty_nif_zero_args(), + ok. + +dirty_nif_send(Config) when is_list(Config) -> + Parent = self(), + Pid = spawn_link(fun() -> + Self = self(), + {ok, Self} = receive_any(), + Parent ! {ok, Self} + end), + {ok, Pid} = send_from_dirty_nif(Pid), + {ok, Pid} = receive_any(), + ok. + +dirty_nif_exception(Config) when is_list(Config) -> + try + %% this checks that the expected exception occurs when the + %% dirty NIF returns the result of enif_make_badarg + %% directly + call_dirty_nif_exception(1), + ct:fail(expected_badarg) + catch + error:badarg -> + [{?MODULE,call_dirty_nif_exception,[1],_}|_] = + erlang:get_stacktrace(), + ok + end, + try + %% this checks that the expected exception occurs when the + %% dirty NIF calls enif_make_badarg at some point but then + %% returns a value that isn't an exception + call_dirty_nif_exception(0), + ct:fail(expected_badarg) + catch + error:badarg -> + [{?MODULE,call_dirty_nif_exception,[0],_}|_] = + erlang:get_stacktrace(), + ok + end, + %% this checks that a dirty NIF can raise various terms as + %% exceptions + ok = nif_raise_exceptions(call_dirty_nif_exception). + +nif_raise_exceptions(NifFunc) -> + ExcTerms = [{error, test}, "a string", <<"a binary">>, + 42, [1,2,3,4,5], [{p,1},{p,2},{p,3}]], + lists:foldl(fun(Term, ok) -> + try + erlang:apply(?MODULE,NifFunc,[Term]), + ct:fail({expected,Term}) + catch + error:Term -> + [{?MODULE,NifFunc,[Term],_}|_] = erlang:get_stacktrace(), + ok + end + end, ok, ExcTerms). + +dirty_scheduler_exit(Config) when is_list(Config) -> + {ok, Node} = start_node(Config, "+SDio 1"), + Path = proplists:get_value(data_dir, Config), + NifLib = filename:join(Path, atom_to_list(?MODULE)), + [ok] = mcall(Node, + [fun() -> + ok = erlang:load_nif(NifLib, []), + Start = erlang:monotonic_time(milli_seconds), + ok = test_dirty_scheduler_exit(), + End = erlang:monotonic_time(milli_seconds), + io:format("Time=~p ms~n", [End-Start]), + ok + end]), + stop_node(Node), + ok. + +test_dirty_scheduler_exit() -> + process_flag(trap_exit,true), + test_dse(10,[]). +test_dse(0,Pids) -> + timer:sleep(100), + kill_dse(Pids,[]); +test_dse(N,Pids) -> + Pid = spawn_link(fun dirty_sleeper/0), + test_dse(N-1,[Pid|Pids]). + +kill_dse([],Killed) -> + wait_dse(Killed); +kill_dse([Pid|Pids],AlreadyKilled) -> + exit(Pid,kill), + kill_dse(Pids,[Pid|AlreadyKilled]). + +wait_dse([]) -> + ok; +wait_dse([Pid|Pids]) -> + receive + {'EXIT',Pid,Reason} -> + killed = Reason + end, + wait_dse(Pids). + +dirty_call_while_terminated(Config) when is_list(Config) -> + Me = self(), + Bin = list_to_binary(lists:duplicate(4711, $r)), + {value, {BinAddr, 4711, 1}} = lists:keysearch(4711, 2, + element(2, + process_info(self(), + binary))), + {Dirty, DM} = spawn_opt(fun () -> + dirty_call_while_terminated_nif(Me), + blipp:blupp(Bin) + end, + [monitor,link]), + receive {dirty_alive, Pid} -> ok end, + {value, {BinAddr, 4711, 2}} = lists:keysearch(4711, 2, + element(2, + process_info(self(), + binary))), + Reason = die_dirty_process, + OT = process_flag(trap_exit, true), + exit(Dirty, Reason), + receive + {'DOWN', DM, process, Dirty, R0} -> + R0 = Reason + end, + receive + {'EXIT', Dirty, R1} -> + R1 = Reason + end, + undefined = process_info(Dirty), + undefined = process_info(Dirty, status), + false = erlang:is_process_alive(Dirty), + false = lists:member(Dirty, processes()), + %% Binary still refered by Dirty process not yet cleaned up + %% since the dirty nif has not yet returned... + {value, {BinAddr, 4711, 2}} = lists:keysearch(4711, 2, + element(2, + process_info(self(), + binary))), + receive after 2000 -> ok end, + receive + Msg -> + ct:fail({unexpected_message, Msg}) + after + 0 -> + ok + end, + {value, {BinAddr, 4711, 1}} = lists:keysearch(4711, 2, + element(2, + process_info(self(), + binary))), + process_flag(trap_exit, OT), + ok. + +dirty_heap_access(Config) when is_list(Config) -> + {ok, Node} = start_node(Config), + Me = self(), + RGL = rpc:call(Node,erlang,whereis,[init]), + Ref = rpc:call(Node,erlang,make_ref,[]), + Dirty = spawn_link(fun () -> + Res = dirty_heap_access_nif(Ref), + garbage_collect(), + Me ! {self(), Res}, + receive after infinity -> ok end + end), + {N, R} = access_dirty_heap(Dirty, RGL, 0, 0), + receive + {Pid, Res} -> + 1000 = length(Res), + lists:foreach(fun (X) -> Ref = X end, Res) + end, + unlink(Dirty), + exit(Dirty, kill), + stop_node(Node), + {comment, integer_to_list(N) ++ " GL change loops; " + ++ integer_to_list(R) ++ " while running dirty"}. + +access_dirty_heap(Dirty, RGL, N, R) -> + case process_info(Dirty, status) of + {status, waiting} -> + {N, R}; + {status, Status} -> + {group_leader, GL} = process_info(Dirty, group_leader), + true = group_leader(RGL, Dirty), + {group_leader, RGL} = process_info(Dirty, group_leader), + true = group_leader(GL, Dirty), + {group_leader, GL} = process_info(Dirty, group_leader), + access_dirty_heap(Dirty, RGL, N+1, case Status of + running -> + R+1; + _ -> + R + end) + end. + +%% +%% Internal... +%% + +receive_any() -> + receive M -> M end. + +start_node(Config) -> + start_node(Config, ""). + +start_node(Config, Args) when is_list(Config) -> + Pa = filename:dirname(code:which(?MODULE)), + Name = list_to_atom(atom_to_list(?MODULE) + ++ "-" + ++ atom_to_list(proplists:get_value(testcase, Config)) + ++ "-" + ++ integer_to_list(erlang:system_time(seconds)) + ++ "-" + ++ integer_to_list(erlang:unique_integer([positive]))), + test_server:start_node(Name, slave, [{args, "-pa "++Pa++" "++Args}]). + +stop_node(Node) -> + test_server:stop_node(Node). + +mcall(Node, Funs) -> + Parent = self(), + Refs = lists:map(fun (Fun) -> + Ref = make_ref(), + spawn_link(Node, + fun () -> + Res = Fun(), + unlink(Parent), + Parent ! {Ref, Res} + end), + Ref + end, Funs), + lists:map(fun (Ref) -> + receive + {Ref, Res} -> + Res + end + end, Refs). + +%% The NIFs: +lib_loaded() -> false. +call_nif_schedule(_,_) -> ?nif_stub. +call_dirty_nif(_,_,_) -> ?nif_stub. +send_from_dirty_nif(_) -> ?nif_stub. +call_dirty_nif_exception(_) -> ?nif_stub. +call_dirty_nif_zero_args() -> ?nif_stub. +dirty_call_while_terminated_nif(_) -> ?nif_stub. +dirty_sleeper() -> ?nif_stub. +dirty_heap_access_nif(_) -> ?nif_stub. + +nif_stub_error(Line) -> + exit({nif_not_loaded,module,?MODULE,line,Line}). diff --git a/erts/emulator/test/dirty_nif_SUITE_data/Makefile.src b/erts/emulator/test/dirty_nif_SUITE_data/Makefile.src new file mode 100644 index 0000000000..e9301753b0 --- /dev/null +++ b/erts/emulator/test/dirty_nif_SUITE_data/Makefile.src @@ -0,0 +1,6 @@ + +NIF_LIBS = dirty_nif_SUITE@dll@ + +all: $(NIF_LIBS) + +@SHLIB_RULES@ diff --git a/erts/emulator/test/dirty_nif_SUITE_data/dirty_nif_SUITE.c b/erts/emulator/test/dirty_nif_SUITE_data/dirty_nif_SUITE.c new file mode 100644 index 0000000000..2013c88167 --- /dev/null +++ b/erts/emulator/test/dirty_nif_SUITE_data/dirty_nif_SUITE.c @@ -0,0 +1,223 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2009-2014. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + */ +#include "erl_nif.h" +#include <assert.h> +#ifndef __WIN32__ +#include <unistd.h> +#endif + +static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) +{ + return 0; +} + +static ERL_NIF_TERM lib_loaded(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + return enif_make_atom(env, "true"); +} + +static int have_dirty_schedulers(void) +{ + ErlNifSysInfo si; + enif_system_info(&si, sizeof(si)); + return si.dirty_scheduler_support; +} + +static ERL_NIF_TERM dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + int n; + char s[10]; + ErlNifBinary b; + if (have_dirty_schedulers()) { + assert(enif_is_on_dirty_scheduler(env)); + } + assert(argc == 3); + enif_get_int(env, argv[0], &n); + enif_get_string(env, argv[1], s, sizeof s, ERL_NIF_LATIN1); + enif_inspect_binary(env, argv[2], &b); + return enif_make_tuple3(env, + enif_make_int(env, n), + enif_make_string(env, s, ERL_NIF_LATIN1), + enif_make_binary(env, &b)); +} + +static ERL_NIF_TERM call_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + int n; + char s[10]; + ErlNifBinary b; + assert(!enif_is_on_dirty_scheduler(env)); + if (argc != 3) + return enif_make_badarg(env); + if (have_dirty_schedulers()) { + if (enif_get_int(env, argv[0], &n) && + enif_get_string(env, argv[1], s, sizeof s, ERL_NIF_LATIN1) && + enif_inspect_binary(env, argv[2], &b)) + return enif_schedule_nif(env, "call_dirty_nif", ERL_NIF_DIRTY_JOB_CPU_BOUND, dirty_nif, argc, argv); + else + return enif_make_badarg(env); + } else { + return dirty_nif(env, argc, argv); + } +} + +static ERL_NIF_TERM send_from_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + ERL_NIF_TERM result; + ErlNifPid pid; + ErlNifEnv* menv; + int res; + + if (!enif_get_local_pid(env, argv[0], &pid)) + return enif_make_badarg(env); + result = enif_make_tuple2(env, enif_make_atom(env, "ok"), enif_make_pid(env, &pid)); + menv = enif_alloc_env(); + res = enif_send(env, &pid, menv, result); + enif_free_env(menv); + if (!res) + return enif_make_badarg(env); + else + return result; +} + +static ERL_NIF_TERM call_dirty_nif_exception(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + switch (argc) { + case 1: { + int arg; + if (enif_get_int(env, argv[0], &arg) && arg < 2) { + ERL_NIF_TERM args[255]; + int i; + args[0] = argv[0]; + for (i = 1; i < 255; i++) + args[i] = enif_make_int(env, i); + return enif_schedule_nif(env, "call_dirty_nif_exception", ERL_NIF_DIRTY_JOB_CPU_BOUND, + call_dirty_nif_exception, 255, args); + } else { + return enif_raise_exception(env, argv[0]); + } + } + case 2: { + int return_badarg_directly; + enif_get_int(env, argv[0], &return_badarg_directly); + assert(return_badarg_directly == 1 || return_badarg_directly == 0); + if (return_badarg_directly) + return enif_make_badarg(env); + else { + /* ignore return value */ enif_make_badarg(env); + return enif_make_atom(env, "ok"); + } + } + default: + return enif_schedule_nif(env, "call_dirty_nif_exception", ERL_NIF_DIRTY_JOB_CPU_BOUND, + call_dirty_nif_exception, argc-1, argv); + } +} + +static ERL_NIF_TERM call_dirty_nif_zero_args(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + int i; + ERL_NIF_TERM result[1000]; + ERL_NIF_TERM ok = enif_make_atom(env, "ok"); + assert(argc == 0); + for (i = 0; i < sizeof(result)/sizeof(*result); i++) { + result[i] = ok; + } + return enif_make_list_from_array(env, result, i); +} + +static ERL_NIF_TERM +dirty_sleeper(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + assert(enif_is_on_dirty_scheduler(env)); +#ifdef __WIN32__ + Sleep(6000); +#else + sleep(6); +#endif + return enif_make_atom(env, "ok"); +} + +static ERL_NIF_TERM dirty_call_while_terminated_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + ErlNifPid self; + ERL_NIF_TERM result, self_term; + ErlNifPid to; + ErlNifEnv* menv; + int res; + + if (!enif_get_local_pid(env, argv[0], &to)) + return enif_make_badarg(env); + + if (!enif_self(env, &self)) + return enif_make_badarg(env); + + self_term = enif_make_pid(env, &self); + + result = enif_make_tuple2(env, enif_make_atom(env, "dirty_alive"), self_term); + menv = enif_alloc_env(); + res = enif_send(env, &to, menv, result); + enif_free_env(menv); + if (!res) + return enif_make_badarg(env); + + /* Wait until we have been killed */ + while (enif_is_process_alive(env, &self)) + ; + + result = enif_make_tuple2(env, enif_make_atom(env, "dirty_dead"), self_term); + menv = enif_alloc_env(); + res = enif_send(env, &to, menv, result); + enif_free_env(menv); + +#ifdef __WIN32__ + Sleep(1000); +#else + sleep(1); +#endif + + return enif_make_atom(env, "ok"); +} + +static ERL_NIF_TERM dirty_heap_access_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + ERL_NIF_TERM res = enif_make_list(env, 0); + int i; + assert(enif_is_on_dirty_scheduler(env)); + for (i = 0; i < 1000; i++) + res = enif_make_list_cell(env, enif_make_copy(env, argv[0]), res); + + return res; +} + + +static ErlNifFunc nif_funcs[] = +{ + {"lib_loaded", 0, lib_loaded}, + {"call_dirty_nif", 3, call_dirty_nif}, + {"send_from_dirty_nif", 1, send_from_dirty_nif, ERL_NIF_DIRTY_JOB_CPU_BOUND}, + {"call_dirty_nif_exception", 1, call_dirty_nif_exception, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"call_dirty_nif_zero_args", 0, call_dirty_nif_zero_args, ERL_NIF_DIRTY_JOB_CPU_BOUND}, + {"dirty_sleeper", 0, dirty_sleeper, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"dirty_call_while_terminated_nif", 1, dirty_call_while_terminated_nif, ERL_NIF_DIRTY_JOB_CPU_BOUND}, + {"dirty_heap_access_nif", 1, dirty_heap_access_nif, ERL_NIF_DIRTY_JOB_CPU_BOUND} +}; + +ERL_NIF_INIT(dirty_nif_SUITE,nif_funcs,load,NULL,NULL,NULL) diff --git a/erts/emulator/test/gc_SUITE.erl b/erts/emulator/test/gc_SUITE.erl index 79c229a34d..8a600b7d9f 100644 --- a/erts/emulator/test/gc_SUITE.erl +++ b/erts/emulator/test/gc_SUITE.erl @@ -25,13 +25,13 @@ -include_lib("common_test/include/ct.hrl"). -export([all/0, suite/0]). --export([grow_heap/1, grow_stack/1, grow_stack_heap/1]). +-export([grow_heap/1, grow_stack/1, grow_stack_heap/1, max_heap_size/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [grow_heap, grow_stack, grow_stack_heap]. + [grow_heap, grow_stack, grow_stack_heap, max_heap_size]. %% Produce a growing list of elements, @@ -163,3 +163,30 @@ show_heap(String) -> {stack_size, SSize}=process_info(self(), stack_size), io:format("Heap/Stack "++String++"~p/~p", [HSize, SSize]). +%% Test that doing a remote GC that triggers the max heap size +%% kills the process. +max_heap_size(_Config) -> + + Pid = spawn_opt(fun long_receive/0,[{max_heap_size, 1024}, + {message_queue_data, on_heap}]), + [Pid ! lists:duplicate(I,I) || I <- lists:seq(1,100)], + Ref = erlang:monitor(process, Pid), + + %% Force messages to be viewed as part of heap + erlang:process_info(Pid, messages), + + %% Do the GC that triggers max heap + erlang:garbage_collect(Pid), + + %% Verify that max heap was triggered + receive + {'DOWN', Ref, process, Pid, killed} -> ok + after 5000 -> + ct:fail({process_did_not_die, Pid, erlang:process_info(Pid)}) + end. + +long_receive() -> + receive + after 10000 -> + ok + end. diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index a7767132ee..a0e9f1bad6 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -38,15 +38,15 @@ is_checks/1, get_length/1, make_atom/1, make_string/1, reverse_list_test/1, otp_9828/1, - otp_9668/1, consume_timeslice/1, dirty_nif/1, dirty_nif_send/1, - dirty_nif_exception/1, call_dirty_nif_exception/1, nif_schedule/1, + otp_9668/1, consume_timeslice/1, nif_schedule/1, nif_exception/1, call_nif_exception/1, nif_nan_and_inf/1, nif_atom_too_long/1, nif_monotonic_time/1, nif_time_offset/1, nif_convert_time_unit/1, nif_now_time/1, nif_cpu_time/1, nif_unique_integer/1, nif_is_process_alive/1, nif_is_port_alive/1, nif_term_to_binary/1, nif_binary_to_term/1, - nif_port_command/1 + nif_port_command/1, + nif_snprintf/1 ]). -export([many_args_100/100]). @@ -75,14 +75,13 @@ all() -> make_string,reverse_list_test, otp_9828, otp_9668, consume_timeslice, - nif_schedule, dirty_nif, dirty_nif_send, dirty_nif_exception, - nif_exception, nif_nan_and_inf, nif_atom_too_long, + nif_schedule, nif_exception, nif_nan_and_inf, nif_atom_too_long, nif_monotonic_time, nif_time_offset, nif_convert_time_unit, nif_now_time, nif_cpu_time, nif_unique_integer, nif_is_process_alive, nif_is_port_alive, nif_term_to_binary, nif_binary_to_term, - nif_port_command - ]. + nif_port_command, + nif_snprintf]. init_per_testcase(_Case, Config) -> Config. @@ -1541,76 +1540,6 @@ nif_schedule(Config) when is_list(Config) -> end, ok. -dirty_nif(Config) when is_list(Config) -> - try erlang:system_info(dirty_cpu_schedulers) of - N when is_integer(N) -> - ensure_lib_loaded(Config), - Val1 = 42, - Val2 = "Erlang", - Val3 = list_to_binary([Val2, 0]), - {Val1, Val2, Val3} = call_dirty_nif(Val1, Val2, Val3), - LargeArray = lists:duplicate(1000, ok), - LargeArray = call_dirty_nif_zero_args(), - ok - catch - error:badarg -> - {skipped,"No dirty scheduler support"} - end. - -dirty_nif_send(Config) when is_list(Config) -> - try erlang:system_info(dirty_cpu_schedulers) of - N when is_integer(N) -> - ensure_lib_loaded(Config), - Parent = self(), - Pid = spawn_link(fun() -> - Self = self(), - {ok, Self} = receive_any(), - Parent ! {ok, Self} - end), - {ok, Pid} = send_from_dirty_nif(Pid), - {ok, Pid} = receive_any(), - ok - catch - error:badarg -> - {skipped,"No dirty scheduler support"} - end. - -dirty_nif_exception(Config) when is_list(Config) -> - try erlang:system_info(dirty_cpu_schedulers) of - N when is_integer(N) -> - ensure_lib_loaded(Config), - try - %% this checks that the expected exception occurs when the - %% dirty NIF returns the result of enif_make_badarg - %% directly - call_dirty_nif_exception(1), - ct:fail(expected_badarg) - catch - error:badarg -> - [{?MODULE,call_dirty_nif_exception,[1],_}|_] = - erlang:get_stacktrace(), - ok - end, - try - %% this checks that the expected exception occurs when the - %% dirty NIF calls enif_make_badarg at some point but then - %% returns a value that isn't an exception - call_dirty_nif_exception(0), - ct:fail(expected_badarg) - catch - error:badarg -> - [{?MODULE,call_dirty_nif_exception,[0],_}|_] = - erlang:get_stacktrace(), - ok - end, - %% this checks that a dirty NIF can raise various terms as - %% exceptions - ok = nif_raise_exceptions(call_dirty_nif_exception) - catch - error:badarg -> - {skipped,"No dirty scheduler support"} - end. - nif_exception(Config) when is_list(Config) -> ensure_lib_loaded(Config), try @@ -2024,9 +1953,18 @@ nif_port_command(Config) -> port_close(Port), {'EXIT', {badarg, _}} = (catch port_command_nif(Port, "hello\n")), + ok. +nif_snprintf(Config) -> + ensure_lib_loaded(Config), + <<"ok",0>> = format_term_nif(3,ok), + <<"o",0>> = format_term_nif(2,ok), + <<"\"hello world\"",0>> = format_term_nif(14,"hello world"), + <<"{{hello,world,-33},3.14",_/binary>> = format_term_nif(50,{{hello,world, -33}, 3.14, self()}), + <<"{{hello,world,-33},",0>> = format_term_nif(20,{{hello,world, -33}, 3.14, self()}), ok. + %% The NIFs: lib_version() -> undefined. call_history() -> ?nif_stub. @@ -2078,10 +2016,6 @@ otp_9668_nif(_) -> ?nif_stub. otp_9828_nif(_) -> ?nif_stub. consume_timeslice_nif(_,_) -> ?nif_stub. call_nif_schedule(_,_) -> ?nif_stub. -call_dirty_nif(_,_,_) -> ?nif_stub. -send_from_dirty_nif(_) -> ?nif_stub. -call_dirty_nif_exception(_) -> ?nif_stub. -call_dirty_nif_zero_args() -> ?nif_stub. call_nif_exception(_) -> ?nif_stub. call_nif_nan_or_inf(_) -> ?nif_stub. call_nif_atom_too_long(_) -> ?nif_stub. @@ -2091,6 +2025,7 @@ is_port_alive_nif(_) -> ?nif_stub. term_to_binary_nif(_, _) -> ?nif_stub. binary_to_term_nif(_, _, _) -> ?nif_stub. port_command_nif(_, _) -> ?nif_stub. +format_term_nif(_,_) -> ?nif_stub. %% maps is_map_nif(_) -> ?nif_stub. diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index 11e5dab58e..13846244d4 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -23,6 +23,9 @@ #include <string.h> #include <assert.h> #include <limits.h> +#ifndef __WIN32__ +#include <unistd.h> +#endif #include "nif_mod.h" @@ -1574,120 +1577,6 @@ static ERL_NIF_TERM call_nif_schedule(ErlNifEnv* env, int argc, const ERL_NIF_TE return result; } -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT - -static int have_dirty_schedulers(void) -{ - ErlNifSysInfo si; - enif_system_info(&si, sizeof(si)); - return si.dirty_scheduler_support; -} - -static ERL_NIF_TERM dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{ - int n; - char s[10]; - ErlNifBinary b; - ERL_NIF_TERM result; - if (have_dirty_schedulers()) { - assert(enif_is_on_dirty_scheduler(env)); - } - assert(argc == 3); - enif_get_int(env, argv[0], &n); - enif_get_string(env, argv[1], s, sizeof s, ERL_NIF_LATIN1); - enif_inspect_binary(env, argv[2], &b); - return enif_make_tuple3(env, - enif_make_int(env, n), - enif_make_string(env, s, ERL_NIF_LATIN1), - enif_make_binary(env, &b)); -} - -static ERL_NIF_TERM call_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{ - int n; - char s[10]; - ErlNifBinary b; - assert(!enif_is_on_dirty_scheduler(env)); - if (argc != 3) - return enif_make_badarg(env); - if (have_dirty_schedulers()) { - if (enif_get_int(env, argv[0], &n) && - enif_get_string(env, argv[1], s, sizeof s, ERL_NIF_LATIN1) && - enif_inspect_binary(env, argv[2], &b)) - return enif_schedule_nif(env, "call_dirty_nif", ERL_NIF_DIRTY_JOB_CPU_BOUND, dirty_nif, argc, argv); - else - return enif_make_badarg(env); - } else { - return dirty_nif(env, argc, argv); - } -} - -static ERL_NIF_TERM send_from_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{ - ERL_NIF_TERM result; - ErlNifPid pid; - ErlNifEnv* menv; - int res; - - if (!enif_get_local_pid(env, argv[0], &pid)) - return enif_make_badarg(env); - result = enif_make_tuple2(env, enif_make_atom(env, "ok"), enif_make_pid(env, &pid)); - menv = enif_alloc_env(); - res = enif_send(env, &pid, menv, result); - enif_free_env(menv); - if (!res) - return enif_make_badarg(env); - else - return result; -} - -static ERL_NIF_TERM call_dirty_nif_exception(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{ - switch (argc) { - case 1: { - int arg; - if (enif_get_int(env, argv[0], &arg) && arg < 2) { - ERL_NIF_TERM args[255]; - int i; - args[0] = argv[0]; - for (i = 1; i < 255; i++) - args[i] = enif_make_int(env, i); - return enif_schedule_nif(env, "call_dirty_nif_exception", ERL_NIF_DIRTY_JOB_CPU_BOUND, - call_dirty_nif_exception, 255, args); - } else { - return enif_raise_exception(env, argv[0]); - } - } - case 2: { - int return_badarg_directly; - enif_get_int(env, argv[0], &return_badarg_directly); - assert(return_badarg_directly == 1 || return_badarg_directly == 0); - if (return_badarg_directly) - return enif_make_badarg(env); - else { - /* ignore return value */ enif_make_badarg(env); - return enif_make_atom(env, "ok"); - } - } - default: - return enif_schedule_nif(env, "call_dirty_nif_exception", ERL_NIF_DIRTY_JOB_CPU_BOUND, - call_dirty_nif_exception, argc-1, argv); - } -} - -static ERL_NIF_TERM call_dirty_nif_zero_args(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{ - int i; - ERL_NIF_TERM result[1000]; - ERL_NIF_TERM ok = enif_make_atom(env, "ok"); - assert(argc == 0); - for (i = 0; i < sizeof(result)/sizeof(*result); i++) { - result[i] = ok; - } - return enif_make_list_from_array(env, result, i); -} -#endif - /* * If argv[0] is the integer 0, call enif_make_badarg, but don't return its * return value. Instead, return ok. Result should still be a badarg @@ -2109,6 +1998,23 @@ static ERL_NIF_TERM port_command(ErlNifEnv* env, int argc, const ERL_NIF_TERM ar return atom_true; } +static ERL_NIF_TERM format_term(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + ErlNifBinary obin; + unsigned int size; + + if (!enif_get_uint(env, argv[0], &size)) + return enif_make_badarg(env); + if (!enif_alloc_binary(size,&obin)) + return enif_make_badarg(env); + + if (enif_snprintf((char*)obin.data, (size_t)size, "%T", argv[1]) < 0) + return atom_false; + + return enif_make_binary(env,&obin); +} + + static ErlNifFunc nif_funcs[] = { {"lib_version", 0, lib_version}, @@ -2162,12 +2068,6 @@ static ErlNifFunc nif_funcs[] = {"otp_9828_nif", 1, otp_9828_nif}, {"consume_timeslice_nif", 2, consume_timeslice_nif}, {"call_nif_schedule", 2, call_nif_schedule}, -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT - {"call_dirty_nif", 3, call_dirty_nif}, - {"send_from_dirty_nif", 1, send_from_dirty_nif, ERL_NIF_DIRTY_JOB_CPU_BOUND}, - {"call_dirty_nif_exception", 1, call_dirty_nif_exception, ERL_NIF_DIRTY_JOB_IO_BOUND}, - {"call_dirty_nif_zero_args", 0, call_dirty_nif_zero_args, ERL_NIF_DIRTY_JOB_CPU_BOUND}, -#endif {"call_nif_exception", 1, call_nif_exception}, {"call_nif_nan_or_inf", 1, call_nif_nan_or_inf}, {"call_nif_atom_too_long", 1, call_nif_atom_too_long}, @@ -2190,7 +2090,8 @@ static ErlNifFunc nif_funcs[] = {"is_port_alive_nif", 1, is_port_alive}, {"term_to_binary_nif", 2, term_to_binary}, {"binary_to_term_nif", 3, binary_to_term}, - {"port_command_nif", 2, port_command} + {"port_command_nif", 2, port_command}, + {"format_term_nif", 2, format_term} }; ERL_NIF_INIT(nif_SUITE,nif_funcs,load,reload,upgrade,unload) diff --git a/erts/emulator/test/port_trace_SUITE.erl b/erts/emulator/test/port_trace_SUITE.erl index bfdea0761b..5d9a75bcd3 100644 --- a/erts/emulator/test/port_trace_SUITE.erl +++ b/erts/emulator/test/port_trace_SUITE.erl @@ -31,7 +31,8 @@ failure_atom/1, failure_posix/1, failure/1, output_term/1, driver_output_term/1, - send_term/1, driver_send_term/1]). + send_term/1, driver_send_term/1, + driver_remote_send_term/1]). -define(ECHO_DRV_NOOP, 0). -define(ECHO_DRV_OUTPUT, 1). @@ -48,6 +49,7 @@ -define(ECHO_DRV_SEND_TERM, 12). -define(ECHO_DRV_DRIVER_SEND_TERM, 13). -define(ECHO_DRV_SAVE_CALLER, 14). +-define(ECHO_DRV_REMOTE_SEND_TERM, 15). suite() -> [{ct_hooks,[ts_install_cth]}, {timetrap, {seconds, 30}}]. @@ -60,7 +62,8 @@ all() -> failure_atom, failure_posix, failure, output_term, driver_output_term, - send_term, driver_send_term]. + send_term, driver_send_term, + driver_remote_send_term]. init_per_suite(Config) -> Config. @@ -75,6 +78,13 @@ end_per_group(_GroupName, Config) -> Config. +init_per_testcase(driver_remote_send_term, Config) -> + case erlang:system_info(smp_support) of + false -> + {skip,"Only supported on smp systems"}; + true -> + init_per_testcase(driver_remote_send_term_smp, Config) + end; init_per_testcase(Func, Config) when is_atom(Func), is_list(Config) -> erlang:trace(all, false, [all]), os:unsetenv("OUTPUTV"), @@ -543,6 +553,34 @@ driver_send_term(_Config) -> ok. +%% Test that driver_send_term from non-scheduler thread does not +%% generate trace messages. +driver_remote_send_term(_Config) -> + + Flags = [send], + {Prt, S} = trace_and_open(Flags,[binary]), + + erlang:port_command(Prt, <<?ECHO_DRV_REMOTE_SEND_TERM, 123456:32>>), + recv({echo, Prt, <<123456:32>>}), + [] = flush(), + + Pid = spawn_link( + fun() -> + erlang:port_command(Prt, <<?ECHO_DRV_SAVE_CALLER>>), + S ! ok, + receive M -> S ! M end + end), + recv(ok), + erlang:trace(Pid, true, ['receive']), + + erlang:port_command(Prt, <<?ECHO_DRV_REMOTE_SEND_TERM, 123456:32>>), + recv({echo, Prt, <<123456:32>>}), + [{trace, Pid, 'receive', {echo, Prt, <<123456:32>>}}] = flush(), + + close(Prt, Flags), + + ok. + %%%%%%%%%%%%%%%%%%% %% Helper functions %%%%%%%%%%%%%%%%%%% @@ -598,7 +636,7 @@ f(From) -> end. recv(Msg) -> - receive Msg -> ok after 100 -> ct:fail({did_not_get_data,Msg,flush()}) end. + receive Msg -> ok after 1000 -> ct:fail({did_not_get_data,Msg,flush()}) end. load_drv(Config) -> Path = proplists:get_value(data_dir, Config), diff --git a/erts/emulator/test/port_trace_SUITE_data/echo_drv.c b/erts/emulator/test/port_trace_SUITE_data/echo_drv.c index b5ae9389b4..b545523192 100644 --- a/erts/emulator/test/port_trace_SUITE_data/echo_drv.c +++ b/erts/emulator/test/port_trace_SUITE_data/echo_drv.c @@ -14,6 +14,13 @@ typedef struct _erl_drv_data { ErlDrvTermData caller; } EchoDrvData; +struct remote_send_term { + char *buf; + int len; + ErlDrvTermData port; + ErlDrvTermData caller; +}; + #define ECHO_DRV_NOOP 0 #define ECHO_DRV_OUTPUT 1 #define ECHO_DRV_OUTPUT2 2 @@ -29,6 +36,7 @@ typedef struct _erl_drv_data { #define ECHO_DRV_SEND_TERM 12 #define ECHO_DRV_DRIVER_SEND_TERM 13 #define ECHO_DRV_SAVE_CALLER 14 +#define ECHO_DRV_REMOTE_SEND_TERM 15 /* ------------------------------------------------------------------------- @@ -78,6 +86,8 @@ static ErlDrvEntry echo_drv_entry = { NULL }; +static void send_term_thread(void *); + /* ------------------------------------------------------------------------- ** Entry functions **/ @@ -200,6 +210,18 @@ static void echo_drv_output(ErlDrvData drv_data, char *buf, ErlDrvSizeT len) { } break; } + case ECHO_DRV_REMOTE_SEND_TERM: + { + ErlDrvTid tid; + struct remote_send_term *t = malloc(sizeof(struct remote_send_term)); + t->len = len-1; + t->buf = malloc(len-1); + t->port = driver_mk_port(port); + t->caller = data_p->caller; + memcpy(t->buf, buf+1, t->len); + erl_drv_thread_create("tmp_thread", &tid, send_term_thread, t, NULL); + break; + } case ECHO_DRV_SAVE_CALLER: data_p->caller = driver_caller(port); break; @@ -239,3 +261,17 @@ static ErlDrvSSizeT echo_drv_call(ErlDrvData drv_data, memcpy(*rbuf, buf+command, len-command); return len-command; } + +static void send_term_thread(void *a) +{ + struct remote_send_term *t = (struct remote_send_term*)a; + ErlDrvTermData term[] = { + ERL_DRV_ATOM, driver_mk_atom("echo"), + ERL_DRV_PORT, t->port, + ERL_DRV_BUF2BINARY, (ErlDrvTermData)(t->buf), + (ErlDrvTermData)(t->len), + ERL_DRV_TUPLE, 3}; + erl_drv_send_term(t->port, t->caller, + term, sizeof(term) / sizeof(ErlDrvTermData)); + return; +} diff --git a/erts/emulator/test/process_SUITE.erl b/erts/emulator/test/process_SUITE.erl index 61a68f9759..eaa4026a8a 100644 --- a/erts/emulator/test/process_SUITE.erl +++ b/erts/emulator/test/process_SUITE.erl @@ -46,7 +46,7 @@ process_status_exiting/1, otp_4725/1, bad_register/1, garbage_collect/1, otp_6237/1, process_info_messages/1, process_flag_badarg/1, process_flag_heap_size/1, - spawn_opt_heap_size/1, + spawn_opt_heap_size/1, spawn_opt_max_heap_size/1, processes_large_tab/1, processes_default_tab/1, processes_small_tab/1, processes_this_tab/1, processes_apply_trap/1, processes_last_call_trap/1, processes_gc_trap/1, @@ -60,7 +60,7 @@ system_task_on_suspended/1, gc_request_when_gc_disabled/1, gc_request_blast_when_gc_disabled/1]). --export([prio_server/2, prio_client/2]). +-export([prio_server/2, prio_client/2, init/1, handle_event/2]). -export([init_per_testcase/2, end_per_testcase/2]). @@ -84,7 +84,8 @@ all() -> bump_reductions, low_prio, yield, yield2, otp_4725, bad_register, garbage_collect, process_info_messages, process_flag_badarg, process_flag_heap_size, - spawn_opt_heap_size, otp_6237, {group, processes_bif}, + spawn_opt_heap_size, spawn_opt_max_heap_size, otp_6237, + {group, processes_bif}, {group, otp_7738}, garb_other_running, {group, system_task}]. @@ -425,6 +426,8 @@ t_process_info(Config) when is_list(Config) -> {status, running} = process_info(self(), status), {min_heap_size, 233} = process_info(self(), min_heap_size), {min_bin_vheap_size,46422} = process_info(self(), min_bin_vheap_size), + {max_heap_size, #{ size := 0, kill := true, error_logger := true}} = + process_info(self(), max_heap_size), {current_function,{?MODULE,t_process_info,1}} = process_info(self(), current_function), {current_function,{?MODULE,t_process_info,1}} = @@ -564,6 +567,8 @@ process_info_other_msg(Config) when is_list(Config) -> {min_heap_size, 233} = process_info(Pid, min_heap_size), {min_bin_vheap_size, 46422} = process_info(Pid, min_bin_vheap_size), + {max_heap_size, #{ size := 0, kill := true, error_logger := true}} = + process_info(self(), max_heap_size), Pid ! stop, ok. @@ -914,10 +919,14 @@ process_info_garbage_collection(_Config) -> Parent = self(), Pid = spawn_link( fun() -> + %% We set mqd to off_heap and send an tuple + %% to process in order to force mbuf_size + %% to be used + process_flag(message_queue_data, off_heap), receive go -> ok end, (fun F(0) -> Parent ! deep, - receive ok -> ok end, + receive {ok,_} -> ok end, []; F(N) -> timer:sleep(1), @@ -926,31 +935,52 @@ process_info_garbage_collection(_Config) -> Parent ! shallow, receive done -> ok end end), - {garbage_collection_info, Before} = - erlang:process_info(Pid, garbage_collection_info), + [{garbage_collection_info, Before},{total_heap_size, THSBefore}] = + erlang:process_info(Pid, [garbage_collection_info, total_heap_size]), Pid ! go, receive deep -> ok end, - {_, Deep} = erlang:process_info(Pid, garbage_collection_info), - Pid ! ok, receive shallow -> ok end, - {_, After} = erlang:process_info(Pid, garbage_collection_info), + [{_, Deep},{_,THSDeep}] = + erlang:process_info(Pid, [garbage_collection_info, total_heap_size]), + Pid ! {ok, make_ref()}, receive shallow -> ok end, + [{_, After},{_, THSAfter}] = + erlang:process_info(Pid, [garbage_collection_info, total_heap_size]), Pid ! done, %% Do some general checks to see if everything seems to be roughly correct ct:log("Before: ~p",[Before]), ct:log("Deep: ~p",[Deep]), ct:log("After: ~p",[After]), + ct:log("Before THS: ~p",[THSBefore]), + ct:log("Deep THS: ~p",[THSDeep]), + ct:log("After THS: ~p",[THSAfter]), %% Check stack_size - true = proplists:get_value(stack_size, Before) < proplists:get_value(stack_size, Deep), - true = proplists:get_value(stack_size, After) < proplists:get_value(stack_size, Deep), + true = gv(stack_size, Before) < gv(stack_size, Deep), + true = gv(stack_size, After) < gv(stack_size, Deep), %% Check used heap size - true = proplists:get_value(heap_size, Before) + proplists:get_value(old_heap_size, Before) - < proplists:get_value(heap_size, Deep) + proplists:get_value(old_heap_size, Deep), - true = proplists:get_value(heap_size, Before) + proplists:get_value(old_heap_size, Before) - < proplists:get_value(heap_size, After) + proplists:get_value(old_heap_size, After), + true = gv(heap_size, Before) + gv(old_heap_size, Before) + < gv(heap_size, Deep) + gv(old_heap_size, Deep), + true = gv(heap_size, Before) + gv(old_heap_size, Before) + < gv(heap_size, After) + gv(old_heap_size, After), + + %% Check that total_heap_size == heap_block_size + old_heap_block_size + mbuf_size + THSBefore = gv(heap_block_size, Before) + + gv(old_heap_block_size, Before) + + gv(mbuf_size, Before), + + THSDeep = gv(heap_block_size, Deep) + + gv(old_heap_block_size, Deep) + + gv(mbuf_size, Deep), + + THSAfter = gv(heap_block_size, After) + + gv(old_heap_block_size, After) + + gv(mbuf_size, After), ok. +gv(Key,List) -> + proplists:get_value(Key,List). + %% Tests erlang:bump_reductions/1. bump_reductions(Config) when is_list(Config) -> erlang:garbage_collect(), @@ -1323,6 +1353,28 @@ process_flag_badarg(Config) when is_list(Config) -> chk_badarg(fun () -> process_flag(min_heap_size, gurka) end), chk_badarg(fun () -> process_flag(min_bin_vheap_size, gurka) end), chk_badarg(fun () -> process_flag(min_bin_vheap_size, -1) end), + + chk_badarg(fun () -> process_flag(max_heap_size, gurka) end), + chk_badarg(fun () -> process_flag(max_heap_size, -1) end), + chk_badarg(fun () -> + {_,Min} = process_info(self(), min_heap_size), + process_flag(max_heap_size, Min - 1) + end), + chk_badarg(fun () -> + {_,Min} = process_info(self(), min_heap_size), + process_flag(max_heap_size, #{size => Min - 1}) + end), + chk_badarg(fun () -> process_flag(max_heap_size, #{}) end), + chk_badarg(fun () -> process_flag(max_heap_size, #{ kill => true }) end), + chk_badarg(fun () -> process_flag(max_heap_size, #{ size => 233, + kill => gurka }) end), + chk_badarg(fun () -> process_flag(max_heap_size, #{ size => 233, + error_logger => gurka }) end), + chk_badarg(fun () -> process_flag(max_heap_size, #{ size => 233, + kill => true, + error_logger => gurka }) end), + chk_badarg(fun () -> process_flag(max_heap_size, #{ size => 1 bsl 64 }) end), + chk_badarg(fun () -> process_flag(priority, 4711) end), chk_badarg(fun () -> process_flag(save_calls, hmmm) end), P= spawn_link(fun () -> receive die -> ok end end), @@ -1923,6 +1975,110 @@ spawn_opt_heap_size(Config) when is_list(Config) -> Pid ! stop, ok. +spawn_opt_max_heap_size(_Config) -> + + error_logger:add_report_handler(?MODULE, self()), + + %% Test that numerical limit works + max_heap_size_test(1024, 1024, true, true), + + %% Test that map limit works + max_heap_size_test(#{ size => 1024 }, 1024, true, true), + + %% Test that no kill is sent + max_heap_size_test(#{ size => 1024, kill => false }, 1024, false, true), + + %% Test that no error_logger report is sent + max_heap_size_test(#{ size => 1024, error_logger => false }, 1024, true, false), + + %% Test that system_flag works + erlang:system_flag(max_heap_size, #{ size => 0, kill => false, + error_logger => true}), + max_heap_size_test(#{ size => 1024 }, 1024, false, true), + max_heap_size_test(#{ size => 1024, kill => true }, 1024, true, true), + + erlang:system_flag(max_heap_size, #{ size => 0, kill => true, + error_logger => false}), + max_heap_size_test(#{ size => 1024 }, 1024, true, false), + max_heap_size_test(#{ size => 1024, error_logger => true }, 1024, true, true), + + erlang:system_flag(max_heap_size, #{ size => 1 bsl 20, kill => true, + error_logger => true}), + max_heap_size_test(#{ }, 1 bsl 20, true, true), + + erlang:system_flag(max_heap_size, #{ size => 0, kill => true, + error_logger => true}), + + %% Test that ordinary case works as expected again + max_heap_size_test(1024, 1024, true, true), + + ok. + +max_heap_size_test(Option, Size, Kill, ErrorLogger) + when map_size(Option) == 0 -> + max_heap_size_test([], Size, Kill, ErrorLogger); +max_heap_size_test(Option, Size, Kill, ErrorLogger) + when is_map(Option); is_integer(Option) -> + max_heap_size_test([{max_heap_size, Option}], Size, Kill, ErrorLogger); +max_heap_size_test(Option, Size, Kill, ErrorLogger) -> + OomFun = fun F() -> timer:sleep(5),[lists:seq(1,1000)|F()] end, + Pid = spawn_opt(OomFun, Option), + {max_heap_size, MHSz} = erlang:process_info(Pid, max_heap_size), + ct:log("Default: ~p~nOption: ~p~nProc: ~p~n", + [erlang:system_info(max_heap_size), Option, MHSz]), + + #{ size := Size} = MHSz, + + Ref = erlang:monitor(process, Pid), + if Kill -> + receive + {'DOWN', Ref, process, Pid, killed} -> + ok + end; + true -> + ok + end, + if ErrorLogger -> + receive + {error, _, {emulator, _, [Pid|_]}} -> + ok + end; + true -> + ok + end, + if not Kill -> + exit(Pid, die), + receive + {'DOWN', Ref, process, Pid, die} -> + ok + end, + flush(); + true -> + ok + end, + receive + M -> + ct:fail({unexpected_message, M}) + after 10 -> + ok + end. + +flush() -> + receive + _M -> + flush() + after 1000 -> + ok + end. + +%% error_logger report handler proxy +init(Pid) -> + {ok, Pid}. + +handle_event(Event, Pid) -> + Pid ! Event, + {ok, Pid}. + processes_term_proc_list(Config) when is_list(Config) -> Tester = self(), as_expected = processes_term_proc_list_test(false), diff --git a/erts/emulator/test/scheduler_SUITE.erl b/erts/emulator/test/scheduler_SUITE.erl index 6b49b68ec8..f18d79d770 100644 --- a/erts/emulator/test/scheduler_SUITE.erl +++ b/erts/emulator/test/scheduler_SUITE.erl @@ -57,7 +57,6 @@ scheduler_suspend_basic/1, scheduler_suspend/1, dirty_scheduler_threads/1, - dirty_scheduler_exit/1, reader_groups/1]). suite() -> @@ -72,7 +71,7 @@ all() -> bound_process, {group, scheduler_bind}, scheduler_threads, scheduler_suspend_basic, scheduler_suspend, - dirty_scheduler_threads, dirty_scheduler_exit, + dirty_scheduler_threads, reader_groups]. groups() -> @@ -1162,53 +1161,6 @@ get_dsstate(Config, Cmd) -> stop_node(Node), {DSCPU, DSCPUOnln, DSIO}. -dirty_scheduler_exit(Config) when is_list(Config) -> - try - erlang:system_info(dirty_cpu_schedulers), - dirty_scheduler_exit_test(Config) - catch - error:badarg -> - {skipped, "No dirty scheduler support"} - end. - -dirty_scheduler_exit_test(Config) -> - {ok, Node} = start_node(Config, "+SDio 1"), - [ok] = mcall(Node, - [fun() -> - Path = proplists:get_value(data_dir, Config), - Lib = atom_to_list(?MODULE), - ok = erlang:load_nif(filename:join(Path,Lib), []), - ok = test_dirty_scheduler_exit() - end]), - stop_node(Node), - ok. - -test_dirty_scheduler_exit() -> - process_flag(trap_exit,true), - test_dse(10,[]). -test_dse(0,Pids) -> - timer:sleep(100), - kill_dse(Pids,[]); -test_dse(N,Pids) -> - Pid = spawn_link(fun dirty_sleeper/0), - test_dse(N-1,[Pid|Pids]). -kill_dse([],Killed) -> - wait_dse(Killed); -kill_dse([Pid|Pids],AlreadyKilled) -> - exit(Pid,kill), - kill_dse(Pids,[Pid|AlreadyKilled]). -wait_dse([]) -> - ok; -wait_dse([Pid|Pids]) -> - receive - {'EXIT',Pid,killed} -> - ok - end, - wait_dse(Pids). - -dirty_sleeper() -> - erlang:nif_error({error,?MODULE}). - scheduler_suspend_basic(Config) when is_list(Config) -> case erlang:system_info(multi_scheduling) of disabled -> diff --git a/erts/emulator/test/scheduler_SUITE_data/Makefile.src b/erts/emulator/test/scheduler_SUITE_data/Makefile.src deleted file mode 100644 index 859112cf19..0000000000 --- a/erts/emulator/test/scheduler_SUITE_data/Makefile.src +++ /dev/null @@ -1,8 +0,0 @@ - -SCHEDULER_LIBS = scheduler_SUITE@dll@ - -all: $(SCHEDULER_LIBS) - -@SHLIB_RULES@ - -$(SCHEDULER_LIBS): scheduler_SUITE.c diff --git a/erts/emulator/test/scheduler_SUITE_data/scheduler_SUITE.c b/erts/emulator/test/scheduler_SUITE_data/scheduler_SUITE.c deleted file mode 100644 index ab4863337f..0000000000 --- a/erts/emulator/test/scheduler_SUITE_data/scheduler_SUITE.c +++ /dev/null @@ -1,37 +0,0 @@ -#ifndef __WIN32__ -#include <unistd.h> -#endif -#include "erl_nif.h" - -static int -load(ErlNifEnv* env, void** priv, ERL_NIF_TERM info) -{ - ErlNifSysInfo sys_info; - enif_system_info(&sys_info, sizeof(ErlNifSysInfo)); - if (!sys_info.smp_support || !sys_info.dirty_scheduler_support) - return 1; - return 0; -} - -static ERL_NIF_TERM -dirty_sleeper(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{ -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT -#ifdef __WIN32__ - Sleep(3000); -#else - sleep(3); -#endif -#endif - return enif_make_atom(env, "ok"); -} - -static ErlNifFunc funcs[] = { -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT - {"dirty_sleeper", 0, dirty_sleeper, ERL_NIF_DIRTY_JOB_IO_BOUND} -#else - {"dirty_sleeper", 0, dirty_sleeper, 0} -#endif -}; - -ERL_NIF_INIT(scheduler_SUITE, funcs, &load, NULL, NULL, NULL); diff --git a/erts/emulator/test/trace_bif_SUITE.erl b/erts/emulator/test/trace_bif_SUITE.erl index 8c3ffccc45..491b37ae46 100644 --- a/erts/emulator/test/trace_bif_SUITE.erl +++ b/erts/emulator/test/trace_bif_SUITE.erl @@ -232,6 +232,7 @@ do_trace_bif_return(TsType, TsFlags) -> {?MODULE, bif_process,0}}, Ts11, TsType), check_ts(TsType, Ts12, make_ts(TsType)), + erlang:trace_pattern({erlang,'_','_'}, false, [local]), ok. diff --git a/erts/emulator/test/tracer_SUITE.erl b/erts/emulator/test/tracer_SUITE.erl index de44d6656a..20fb7e475e 100644 --- a/erts/emulator/test/tracer_SUITE.erl +++ b/erts/emulator/test/tracer_SUITE.erl @@ -44,6 +44,8 @@ groups() -> gc_start, gc_end]}]. init_per_suite(Config) -> + erlang:trace_pattern({'_','_','_'}, false, [local]), + erlang:trace_pattern({'_','_','_'}, false, []), purge(), Config. @@ -119,23 +121,24 @@ unload(_Config) -> end, 1 = erlang:trace(Pid, true, [{tracer, tracer_test, - {#{ call => trace}, self(), []}}, + {#{ call => trace }, self(), []}}, call]), 1 = erlang:trace_pattern({?MODULE, all, 0}, [], []), Tc(1), - receive _ -> ok after 0 -> ct:fail(timeout) end, + receive _M -> ok after 0 -> ct:fail(timeout) end, + receive M0 -> ct:fail({unexpected_message0, M0}) after 0 -> ok end, code:purge(tracer_test), code:delete(tracer_test), Tc(1), - receive M1 -> ct:fail({unexpected_message, M1}) after 0 -> ok end, + receive M1 -> ct:fail({unexpected_message1, M1}) after 0 -> ok end, code:purge(tracer_test), Tc(1), - receive M2 -> ct:fail({unexpected_message, M2}) after 0 -> ok end, + receive M2 -> ct:fail({unexpected_message2, M2}) after 0 -> ok end, ok. @@ -167,13 +170,19 @@ reload(_Config) -> false = code:purge(tracer_test), true = code:delete(tracer_test), - false = code:purge(tracer_test) + false = code:purge(tracer_test), + timer:sleep(10) end || _ <- lists:seq(1,15)], ok. reload_loop() -> ?MODULE:all(), + ?MODULE:all(), + ?MODULE:all(), + ?MODULE:all(), + ?MODULE:all(), + timer:sleep(1), reload_loop(). invalid_tracers(_Config) -> @@ -214,7 +223,7 @@ send(_Config) -> Expect = fun(Pid, State, EOpts) -> receive Msg -> - {Pid, send, State, Pid, ok, Self, Opts} = Msg, + {send, State, Pid, ok, Self, Opts} = Msg, check_opts(EOpts, Opts) end end, @@ -230,7 +239,7 @@ recv(_Config) -> Expect = fun(Pid, State, EOpts) -> receive Msg -> - {undefined, 'receive', State, Pid, ok, undefined, Opts} = Msg, + {'receive', State, Pid, ok, undefined, Opts} = Msg, check_opts(EOpts, Opts) end end, @@ -247,14 +256,14 @@ spawn(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {Pid, spawn, State, Pid, NewPid, + {spawn, State, Pid, NewPid, {lists,seq,[1,10]}, Opts} = Msg, check_opts(EOpts, Opts), true = is_pid(NewPid) andalso NewPid /= Pid end end, - test(spawn, procs, Tc, Expect, true). + test(spawn, procs, Tc, Expect, false). exit(_Config) -> Tc = fun(Pid) -> @@ -265,7 +274,7 @@ exit(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {Pid, exit, State, Pid, normal, undefined, Opts} = Msg, + {exit, State, Pid, normal, undefined, Opts} = Msg, check_opts(EOpts, Opts) end end, @@ -286,13 +295,13 @@ link(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {Pid, link, State, Pid, NewPid, undefined, Opts} = Msg, + {link, State, Pid, NewPid, undefined, Opts} = Msg, check_opts(EOpts, Opts), true = is_pid(NewPid) andalso NewPid /= Pid end end, - test(link, procs, Tc, Expect, true). + test(link, procs, Tc, Expect, false). unlink(_Config) -> @@ -309,13 +318,13 @@ unlink(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {Pid, unlink, State, Pid, NewPid, undefined, Opts} = Msg, + {unlink, State, Pid, NewPid, undefined, Opts} = Msg, check_opts(EOpts, Opts), true = is_pid(NewPid) andalso NewPid /= Pid end end, - test(unlink, procs, Tc, Expect, true). + test(unlink, procs, Tc, Expect, false). getting_linked(_Config) -> @@ -331,7 +340,7 @@ getting_linked(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {NewPid, getting_linked, State, Pid, NewPid, undefined, Opts} = Msg, + {getting_linked, State, Pid, NewPid, undefined, Opts} = Msg, check_opts(EOpts, Opts), true = is_pid(NewPid) andalso NewPid /= Pid end @@ -355,7 +364,7 @@ getting_unlinked(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {NewPid, getting_unlinked, State, Pid, NewPid, undefined, Opts} = Msg, + {getting_unlinked, State, Pid, NewPid, undefined, Opts} = Msg, check_opts(EOpts, Opts), true = is_pid(NewPid) andalso NewPid /= Pid end @@ -377,12 +386,12 @@ register(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {Pid, register, State, Pid, ?MODULE, undefined, Opts} = Msg, + {register, State, Pid, ?MODULE, undefined, Opts} = Msg, check_opts(EOpts, Opts) end end, - test(register, procs, Tc, Expect, true). + test(register, procs, Tc, Expect, false). unregister(_Config) -> @@ -398,18 +407,18 @@ unregister(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {Pid, unregister, State, Pid, ?MODULE, undefined, Opts} = Msg, + {unregister, State, Pid, ?MODULE, undefined, Opts} = Msg, check_opts(EOpts, Opts) end end, - test(unregister, procs, Tc, Expect, true). + test(unregister, procs, Tc, Expect, false). in(_Config) -> Tc = fun(Pid) -> Self = self(), - Pid ! fun() -> receive after 1 -> Self ! ok end end, + Pid ! fun() -> receive after 10 -> Self ! ok end end, receive ok -> ok end end, @@ -418,7 +427,7 @@ in(_Config) -> N = (fun F(N) -> receive Msg -> - {Pid, in, State, Pid, _, + {in, State, Pid, _, undefined, Opts} = Msg, check_opts(EOpts, Opts), F(N+1) @@ -428,7 +437,7 @@ in(_Config) -> true = N > 0 end, - test(in, running, Tc, Expect, true). + test(in, running, Tc, Expect, false). out(_Config) -> Tc = fun(Pid) -> @@ -443,7 +452,7 @@ out(_Config) -> N = (fun F(N) -> receive Msg -> - {Pid, out, State, Pid, _, + {out, State, Pid, _, undefined, Opts} = Msg, check_opts(EOpts, Opts), F(N+1) @@ -453,7 +462,7 @@ out(_Config) -> true = N > 0 end, - test(out, running, Tc, Expect, true, true). + test(out, running, Tc, Expect, false, true). gc_start(_Config) -> @@ -468,12 +477,12 @@ gc_start(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {Pid, gc_major_start, State, Pid, _, undefined, Opts} = Msg, + {gc_major_start, State, Pid, _, undefined, Opts} = Msg, check_opts(EOpts, Opts) end end, - test(gc_major_start, garbage_collection, Tc, Expect, true). + test(gc_major_start, garbage_collection, Tc, Expect, false). gc_end(_Config) -> @@ -488,20 +497,20 @@ gc_end(_Config) -> fun(Pid, State, EOpts) -> receive Msg -> - {Pid, gc_major_end, State, Pid, _, undefined, Opts} = Msg, + {gc_major_end, State, Pid, _, undefined, Opts} = Msg, check_opts(EOpts, Opts) end end, - test(gc_major_end, garbage_collection, Tc, Expect, true). + test(gc_major_end, garbage_collection, Tc, Expect, false). test(Event, Tc, Expect) -> - test(Event, Tc, Expect, true). + test(Event, Tc, Expect, false). test(Event, Tc, Expect, Removes) -> test(Event, Event, Tc, Expect, Removes). test(Event, TraceFlag, Tc, Expect, Removes) -> test(Event, TraceFlag, Tc, Expect, Removes, false). -test(Event, TraceFlag, Tc, Expect, Removes, Dies) -> +test(Event, TraceFlag, Tc, Expect, _Removes, Dies) -> ComplexState = {fun() -> ok end, <<0:(128*8)>>}, Opts = #{ timestamp => undefined, @@ -557,25 +566,6 @@ test(Event, TraceFlag, Tc, Expect, Removes, Dies) -> ok end, - %% Test that remove works - Pid3 = start_tracee(), - State3 = {#{ Event => remove }, self(), ComplexState}, - 1 = erlang:trace(Pid3, true, [TraceFlag, {tracer, tracer_test, State3}]), - Tc(Pid3), - ok = trace_delivered(Pid3), - receive M3 -> ct:fail({unexpected, M3}) after 0 -> ok end, - if not Dies -> - if Removes -> - {flags, []} = erlang:trace_info(Pid3, flags), - {tracer, []} = erlang:trace_info(Pid3, tracer); - true -> - {flags, [TraceFlag]} = erlang:trace_info(Pid3, flags), - {tracer, {tracer_test, State3}} = erlang:trace_info(Pid3, tracer) - end, - erlang:trace(Pid3, false, [TraceFlag]); - true -> - ok - end, ok. check_opts(#{ scheduler_id := number } = E, #{ scheduler_id := N } = O) diff --git a/erts/emulator/test/tracer_SUITE_data/tracer_test.c b/erts/emulator/test/tracer_SUITE_data/tracer_test.c index 8b4be1345d..908f35da9c 100644 --- a/erts/emulator/test/tracer_SUITE_data/tracer_test.c +++ b/erts/emulator/test/tracer_SUITE_data/tracer_test.c @@ -104,16 +104,10 @@ static ERL_NIF_TERM trace(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) enif_get_tuple(env, argv[1], &state_arity, &state_tuple); - tuple = enif_alloc(sizeof(ERL_NIF_TERM)*(argc+1)); - memcpy(tuple+1,argv,sizeof(ERL_NIF_TERM)*argc); + tuple = enif_alloc(sizeof(ERL_NIF_TERM)*(argc)); + memcpy(tuple,argv,sizeof(ERL_NIF_TERM)*argc); - if (enif_self(env, &self)) { - tuple[0] = enif_make_pid(env, &self); - } else { - tuple[0] = enif_make_atom(env, "undefined"); - } - - msg = enif_make_tuple_from_array(env, tuple, argc + 1); + msg = enif_make_tuple_from_array(env, tuple, argc); enif_get_local_pid(env, state_tuple[1], &to); enif_send(env, &to, NULL, msg); enif_free(tuple); diff --git a/erts/epmd/test/epmd_SUITE.erl b/erts/epmd/test/epmd_SUITE.erl index 763984267a..0f0a5acde7 100644 --- a/erts/epmd/test/epmd_SUITE.erl +++ b/erts/epmd/test/epmd_SUITE.erl @@ -23,64 +23,58 @@ % Timeout for test cases (rather long to work on slow machines) --define(SHORT_TEST_TIMEOUT, ?t:seconds(30)). % Default --define(MEDIUM_TEST_TIMEOUT, ?t:minutes(3)). --define(LONG_TEST_TIMEOUT, ?t:minutes(10)). +-define(MEDIUM_TEST_TIMEOUT, {minutes,3}). +-define(LONG_TEST_TIMEOUT, {minutes,10}). % Delay inserted into code -define(SHORT_PAUSE, 100). --define(MEDIUM_PAUSE, ?t:seconds(1)). --define(LONG_PAUSE, ?t:seconds(5)). +-define(MEDIUM_PAUSE, 1000). +-define(LONG_PAUSE, 5000). % Information about nodes -record(node_info, {port, node_type, prot, lvsn, hvsn, node_name, extra}). % Test server specific exports --export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, - init_per_group/2,end_per_group/2, - init_per_testcase/2, end_per_testcase/2]). - --export( - [ - register_name/1, - register_name_ipv6/1, - register_names_1/1, - register_names_2/1, - register_duplicate_name/1, - unicode_name/1, - long_unicode_name/1, - get_port_nr/1, - slow_get_port_nr/1, - unregister_others_name_1/1, - unregister_others_name_2/1, - register_overflow/1, - name_with_null_inside/1, - name_null_terminated/1, - stupid_names_req/1, - - no_data/1, - one_byte/1, - two_bytes/1, - partial_packet/1, - zero_length/1, - too_large/1, - alive_req_too_small_1/1, - alive_req_too_small_2/1, - alive_req_too_large/1, - - returns_valid_empty_extra/1, - returns_valid_populated_extra_with_nulls/1, - - names_stdout/1, - - buffer_overrun_1/1, - buffer_overrun_2/1, - no_nonlocal_register/1, - no_nonlocal_kill/1, - no_live_killing/1, - - socket_reset_before_alive2_reply_is_written/1 - ]). +-export([all/0, suite/0, groups/0, init_per_testcase/2, end_per_testcase/2]). + +-export([register_name/1, + register_name_ipv6/1, + register_names_1/1, + register_names_2/1, + register_duplicate_name/1, + unicode_name/1, + long_unicode_name/1, + get_port_nr/1, + slow_get_port_nr/1, + unregister_others_name_1/1, + unregister_others_name_2/1, + register_overflow/1, + name_with_null_inside/1, + name_null_terminated/1, + stupid_names_req/1, + + no_data/1, + one_byte/1, + two_bytes/1, + partial_packet/1, + zero_length/1, + too_large/1, + alive_req_too_small_1/1, + alive_req_too_small_2/1, + alive_req_too_large/1, + + returns_valid_empty_extra/1, + returns_valid_populated_extra_with_nulls/1, + + names_stdout/1, + + buffer_overrun_1/1, + buffer_overrun_2/1, + no_nonlocal_register/1, + no_nonlocal_kill/1, + no_live_killing/1, + + socket_reset_before_alive2_reply_is_written/1]). % Port we use for testing @@ -88,7 +82,7 @@ -define(EPMDARGS,"-packet_timeout 1"). -define(DUMMY_PORT, 1000). % Port number to register - % not in real use. +% not in real use. % Timeouts etc inside test cases. Time is in milliseconds. -define(CONN_RETRY, 4). % Times to retry connecting @@ -111,7 +105,9 @@ %% all/1 %% -suite() -> [{ct_hooks,[ts_install_cth]}]. +suite() -> + [{ct_hooks,[ts_install_cth]}, + {timetrap, ?MEDIUM_TEST_TIMEOUT}]. all() -> [register_name, register_name_ipv6, @@ -134,105 +130,71 @@ groups() -> [{buffer_overrun, [], [buffer_overrun_1, buffer_overrun_2]}]. -init_per_suite(Config) -> - Config. - -end_per_suite(_Config) -> - ok. - -init_per_group(_GroupName, Config) -> - Config. - -end_per_group(_GroupName, Config) -> - Config. - - %% %% Run before and after each test case %% init_per_testcase(_Func, Config) -> - Dog = test_server:timetrap(?MEDIUM_TEST_TIMEOUT), cleanup(), - [{watchdog, Dog} | Config]. + Config. -end_per_testcase(_Func, Config) -> +end_per_testcase(_Func, _Config) -> cleanup(), - Dog = ?config(watchdog, Config), - catch test_server:timetrap_cancel(Dog), % We may have canceled already ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -register_name(doc) -> - ["Register a name"]; -register_name(suite) -> - []; +%% Register a name register_name(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = register_node("foobar"), - ?line ok = close(Sock), % Unregister + ok = epmdrun(), + {ok,Sock} = register_node("foobar"), + ok = close(Sock), % Unregister ok. -register_name_ipv6(doc) -> - ["Register a name over IPv6"]; -register_name_ipv6(suite) -> - []; +%% Register a name over IPv6 register_name_ipv6(Config) when is_list(Config) -> % Test if the host has an IPv6 loopback address Res = gen_tcp:listen(0, [inet6, {ip, {0,0,0,0,0,0,0,1}}]), case Res of - {ok,LSock} -> - gen_tcp:close(LSock), - ?line ok = epmdrun(), - ?line {ok,Sock} = register_node6("foobar6"), - ?line ok = close(Sock), % Unregister - ok; - _Error -> - {skip, "Host does not have an IPv6 loopback address"} + {ok,LSock} -> + gen_tcp:close(LSock), + ok = epmdrun(), + {ok,Sock} = register_node6("foobar6"), + ok = close(Sock), % Unregister + ok; + _Error -> + {skip, "Host does not have an IPv6 loopback address"} end. -register_names_1(doc) -> - ["Register and unregister two nodes"]; -register_names_1(suite) -> - []; +%% Register and unregister two nodes register_names_1(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock1} = register_node("foobar"), - ?line {ok,Sock2} = register_node("foozap"), - ?line ok = close(Sock1), % Unregister - ?line ok = close(Sock2), % Unregister + ok = epmdrun(), + {ok,Sock1} = register_node("foobar"), + {ok,Sock2} = register_node("foozap"), + ok = close(Sock1), % Unregister + ok = close(Sock2), % Unregister ok. -register_names_2(doc) -> - ["Register and unregister two nodes"]; -register_names_2(suite) -> - []; +%% Register and unregister two nodes register_names_2(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock1} = register_node("foobar"), - ?line {ok,Sock2} = register_node("foozap"), - ?line ok = close(Sock2), % Unregister - ?line ok = close(Sock1), % Unregister + ok = epmdrun(), + {ok,Sock1} = register_node("foobar"), + {ok,Sock2} = register_node("foozap"), + ok = close(Sock2), % Unregister + ok = close(Sock1), % Unregister ok. -register_duplicate_name(doc) -> - ["Two nodes with the same name"]; -register_duplicate_name(suite) -> - []; +%% Two nodes with the same name register_duplicate_name(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = register_node("foobar"), - ?line error = register_node("foobar"), - ?line ok = close(Sock), % Unregister + ok = epmdrun(), + {ok,Sock} = register_node("foobar"), + error = register_node("foobar"), + ok = close(Sock), % Unregister ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -unicode_name(doc) -> - ["Check that we can register and lookup a unicode name"]; -unicode_name(suite) -> - []; +%% Check that we can register and lookup a unicode name unicode_name(Config) when is_list(Config) -> ok = epmdrun(), NodeName = [16#1f608], @@ -244,10 +206,7 @@ unicode_name(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -long_unicode_name(doc) -> - ["Check that we can register and lookup a long unicode name"]; -long_unicode_name(suite) -> - []; +%% Check that we can register and lookup a long unicode name long_unicode_name(Config) when is_list(Config) -> ok = epmdrun(), BaseChar = 16#1f600, @@ -273,101 +232,85 @@ register_node_v2(Port, NodeType, Prot, HVsn, LVsn, Name, Extra) -> register_node_v2(Addr, Port, NodeType, Prot, HVsn, LVsn, Name, Extra) -> Req = alive2_req(Port, NodeType, Prot, HVsn, LVsn, Name, Extra), case send_req(Req, Addr) of - {ok,Sock} -> - case recv(Sock,4) of - {ok, [?EPMD_ALIVE2_RESP,_Res=0,_C0,_C1]} -> - {ok,Sock}; - Other -> - test_server:format("recv on sock ~w: ~p~n", - [Sock,Other]), - error - end; - error -> - error + {ok,Sock} -> + case recv(Sock,4) of + {ok, [?EPMD_ALIVE2_RESP,_Res=0,_C0,_C1]} -> + {ok,Sock}; + Other -> + io:format("recv on sock ~w: ~p~n", [Sock,Other]), + error + end; + error -> + error end. % Internal function to fetch information about a node port_please_v2(Name) -> case send_req([?EPMD_PORT_PLEASE2_REQ, - binary_to_list(unicode:characters_to_binary(Name))]) of - {ok,Sock} -> - case recv_until_sock_closes(Sock) of - {ok, Resp} -> - parse_port2_resp(Resp); - Other -> - test_server:format("recv on sock ~w: ~p~n", - [Sock,Other]), - error - end; - error -> - error + binary_to_list(unicode:characters_to_binary(Name))]) of + {ok,Sock} -> + case recv_until_sock_closes(Sock) of + {ok, Resp} -> + parse_port2_resp(Resp); + Other -> + io:format("recv on sock ~w: ~p~n", [Sock,Other]), + error + end; + error -> + error end. parse_port2_resp(Resp) -> case list_to_binary(Resp) of - <<?EPMD_PORT2_RESP,Res,Port:16,NodeType,Prot,HVsn:16,LVsn:16, - NLen:16,NodeName:NLen/binary, - ELen:16,Extra:ELen/binary>> when Res =:= 0 -> - {ok, #node_info{port=Port,node_type=NodeType,prot=Prot, - hvsn=HVsn,lvsn=LVsn, - node_name=unicode:characters_to_list(NodeName), - extra=binary_to_list(Extra)}}; - _Other -> - test_server:format("invalid port2 resp: ~p~n", - [Resp]), - error + <<?EPMD_PORT2_RESP,Res,Port:16,NodeType,Prot,HVsn:16,LVsn:16, + NLen:16,NodeName:NLen/binary, + ELen:16,Extra:ELen/binary>> when Res =:= 0 -> + {ok, #node_info{port=Port,node_type=NodeType,prot=Prot, + hvsn=HVsn,lvsn=LVsn, + node_name=unicode:characters_to_list(NodeName), + extra=binary_to_list(Extra)}}; + _Other -> + io:format("invalid port2 resp: ~p~n", [Resp]), + error end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -name_with_null_inside(doc) -> - ["Register a name with a null char in it"]; -name_with_null_inside(suite) -> - []; +%% Register a name with a null char in it name_with_null_inside(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line error = register_node("foo\000bar"), + ok = epmdrun(), + error = register_node("foo\000bar"), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -name_null_terminated(doc) -> - ["Register a name with terminating null byte"]; -name_null_terminated(suite) -> - []; +%% Register a name with terminating null byte name_null_terminated(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line error = register_node("foobar\000"), + ok = epmdrun(), + error = register_node("foobar\000"), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -stupid_names_req(doc) -> - ["Read names from epmd in a stupid way"]; -stupid_names_req(suite) -> - []; +%% Read names from epmd in a stupid way stupid_names_req(Config) when is_list(Config) -> - Dog = ?config(watchdog, Config), - test_server:timetrap_cancel(Dog), - LongDog = test_server:timetrap(?MEDIUM_TEST_TIMEOUT), - ?line ok = epmdrun(), - ?line [FirstConn | Conn] = register_many(1, ?REG_REPEAT_LIM, "foo"), - ?line unregister_many([FirstConn]), + ok = epmdrun(), + [FirstConn | Conn] = register_many(1, ?REG_REPEAT_LIM, "foo"), + unregister_many([FirstConn]), sleep(?MEDIUM_PAUSE), - ?line ok = check_names(Conn), - ?line ok = unregister_many(Conn), - test_server:timetrap_cancel(LongDog), + ok = check_names(Conn), + ok = unregister_many(Conn), ok. check_names(Conn) -> - ?line {ok,Sock} = connect_active(), - ?line {ok,Reply} = do_get_names(Sock), - ?line SortConn = lists:sort(Conn), - ?line SortReply = lists:sort(Reply), - ?line ok = check_names_cmp(SortConn, SortReply), + {ok,Sock} = connect_active(), + {ok,Reply} = do_get_names(Sock), + SortConn = lists:sort(Conn), + SortReply = lists:sort(Reply), + ok = check_names_cmp(SortConn, SortReply), ok. - + % Compare if the result was the same as was registered @@ -381,43 +324,43 @@ check_names_cmp([{Name,Port,_Sock} | Conn], [{Name,Port} | Reply]) -> -define(int16(X), [(X bsr 8) band 16#ff, X band 16#ff]). -define(u32(X1,X2,X3,X4), - (((X1) bsl 24) bor ((X2) bsl 16) bor ((X3) bsl 8) bor X4)). + (((X1) bsl 24) bor ((X2) bsl 16) bor ((X3) bsl 8) bor X4)). do_get_names(Socket) -> inet_tcp:send(Socket, [?int16(1),?EPMD_NAMES_REQ]), receive - {tcp, Socket, [P0,P1,P2,P3 | T]} -> - EpmdPort = ?u32(P0,P1,P2,P3), - if EpmdPort == ?PORT -> - names_loop(Socket, T, []); - true -> - close(Socket), - {error, address} - end; - {tcp_closed, Socket} -> - {ok, []} + {tcp, Socket, [P0,P1,P2,P3 | T]} -> + EpmdPort = ?u32(P0,P1,P2,P3), + if EpmdPort == ?PORT -> + names_loop(Socket, T, []); + true -> + close(Socket), + {error, address} + end; + {tcp_closed, Socket} -> + {ok, []} end. names_loop(Socket, Acc, Ps) -> receive - {tcp, Socket, Bytes} -> - {NAcc, NPs} = scan_names(Acc ++ Bytes, Ps), - names_loop(Socket, NAcc, NPs); - {tcp_closed, Socket} -> - {_, NPs} = scan_names(Acc, Ps), % Really needed? - {ok, NPs} + {tcp, Socket, Bytes} -> + {NAcc, NPs} = scan_names(Acc ++ Bytes, Ps), + names_loop(Socket, NAcc, NPs); + {tcp_closed, Socket} -> + {_, NPs} = scan_names(Acc, Ps), % Really needed? + {ok, NPs} end. scan_names(Buf, Ps) -> case scan_line(Buf, []) of - {Line, NBuf} -> - case parse_line(Line) of - {ok, Entry} -> - scan_names(NBuf, [Entry | Ps]); - error -> - scan_names(NBuf, Ps) - end; - [] -> {Buf, Ps} + {Line, NBuf} -> + case parse_line(Line) of + {ok, Entry} -> + scan_names(NBuf, [Entry | Ps]); + error -> + scan_names(NBuf, Ps) + end; + [] -> {Buf, Ps} end. scan_line([$\n | Buf], Line) -> {lists:reverse(Line), Buf}; @@ -426,16 +369,16 @@ scan_line([], _) -> []. parse_line([$n,$a,$m,$e,$ | Buf0]) -> case parse_name(Buf0, []) of - {Name, Buf1} -> - case Buf1 of - [$a,$t,$ ,$p,$o,$r,$t,$ | Buf2] -> - case catch list_to_integer(Buf2) of - {'EXIT', _} -> error; - Port -> {ok, {Name, Port}} - end; - _ -> error - end; - error -> error + {Name, Buf1} -> + case Buf1 of + [$a,$t,$ ,$p,$o,$r,$t,$ | Buf2] -> + case catch list_to_integer(Buf2) of + {'EXIT', _} -> error; + Port -> {ok, {Name, Port}} + end; + _ -> error + end; + error -> error end; parse_line(_) -> error. @@ -447,17 +390,11 @@ parse_name([], _Name) -> error. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -get_port_nr(doc) -> - ["Register a name on a port and ask about port nr"]; -get_port_nr(suite) -> - []; +%% Register a name on a port and ask about port nr get_port_nr(Config) when is_list(Config) -> port_request([?EPMD_PORT_PLEASE2_REQ,"foo"]). -slow_get_port_nr(doc) -> - ["Register with slow write and ask about port nr"]; -slow_get_port_nr(suite) -> - []; +%% Register with slow write and ask about port nr slow_get_port_nr(Config) when is_list(Config) -> port_request([?EPMD_PORT_PLEASE2_REQ,d,$f,d,$o,d,$o]). @@ -465,142 +402,127 @@ slow_get_port_nr(Config) when is_list(Config) -> % Internal function used above port_request(M) -> - ?line ok = epmdrun(), + ok = epmdrun(), Port = 1042, - ?line {ok,RSock} = register_node("foo", Port), - ?line {ok,Sock} = connect(), - ?line ok = send(Sock,[size16(M),M]), - ?line case recv_until_sock_closes(Sock) of - {ok, Resp} -> - ?line close(RSock), - ?line {ok,Rec} = parse_port2_resp(Resp), - ?line Port = Rec#node_info.port, - ok; - Other -> - ?line close(RSock), - ?line test_server:format("recv on sock ~w: ~p~n", - [Sock,Other]), - ?line throw({error,Other}) - end, + {ok,RSock} = register_node("foo", Port), + {ok,Sock} = connect(), + ok = send(Sock,[size16(M),M]), + case recv_until_sock_closes(Sock) of + {ok, Resp} -> + close(RSock), + {ok,Rec} = parse_port2_resp(Resp), + Port = Rec#node_info.port, + ok; + Other -> + close(RSock), + io:format("recv on sock ~w: ~p~n", [Sock,Other]), + throw({error,Other}) + end, ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -unregister_others_name_1(doc) -> - ["Unregister name of other node"]; -unregister_others_name_1(suite) -> - []; +%% Unregister name of other node unregister_others_name_1(Config) when is_list(Config) -> - ?line ok = epmdrun("-relaxed_command_check"), - ?line {ok,RSock} = register_node("foo"), - ?line {ok,Sock} = connect(), + ok = epmdrun("-relaxed_command_check"), + {ok,RSock} = register_node("foo"), + {ok,Sock} = connect(), M = [?EPMD_STOP_REQ,"foo"], - ?line ok = send(Sock,[size16(M),M]), + ok = send(Sock,[size16(M),M]), R = "STOPPED", - ?line {ok,R} = recv(Sock,length(R)), - ?line ok = close(RSock), + {ok,R} = recv(Sock,length(R)), + ok = close(RSock), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -unregister_others_name_2(doc) -> - ["Unregister name of other node"]; -unregister_others_name_2(suite) -> - []; +%% Unregister name of other node unregister_others_name_2(Config) when is_list(Config) -> - ?line ok = epmdrun("-relaxed_command_check"), - ?line {ok,Sock} = connect(), + ok = epmdrun("-relaxed_command_check"), + {ok,Sock} = connect(), M = [?EPMD_STOP_REQ,"xxx42"], - ?line ok = send(Sock,[size16(M),M]), + ok = send(Sock,[size16(M),M]), R = "NOEXIST", - ?line {ok,R} = recv(Sock,length(R)), + {ok,R} = recv(Sock,length(R)), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -register_overflow(doc) -> - ["Register too many, clean and redo 10 times"]; -register_overflow(suite) -> - []; +%% Register too many, clean and redo 10 times register_overflow(Config) when is_list(Config) -> - Dog = ?config(watchdog, Config), - test_server:timetrap_cancel(Dog), - LongDog = test_server:timetrap(?LONG_TEST_TIMEOUT), - ?line ok = epmdrun(), - ?line Conn = register_many(1, ?REG_REPEAT_LIM, "foo"), + ct:timetrap(?LONG_TEST_TIMEOUT), + ok = epmdrun(), + Conn = register_many(1, ?REG_REPEAT_LIM, "foo"), Count = length(Conn), - ?line ok = unregister_many(Conn), + ok = unregister_many(Conn), sleep(?MEDIUM_PAUSE), - test_server:format("Limit was ~w names, now reg/unreg all 10 times~n", - [Count]), - ?line ok = register_repeat(Count), + io:format("Limit was ~w names, now reg/unreg all 10 times~n", [Count]), + ok = register_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = rregister_repeat(Count), + ok = rregister_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = register_repeat(Count), + ok = register_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = rregister_repeat(Count), + ok = rregister_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = register_repeat(Count), + ok = register_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = rregister_repeat(Count), + ok = rregister_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = register_repeat(Count), + ok = register_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = rregister_repeat(Count), + ok = rregister_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = register_repeat(Count), + ok = register_repeat(Count), sleep(?MEDIUM_PAUSE), - ?line ok = rregister_repeat(Count), - test_server:timetrap_cancel(LongDog), + ok = rregister_repeat(Count), ok. register_repeat(Count) -> Conn = register_many(1, ?REG_REPEAT_LIM, "foo"), ok = unregister_many(Conn), if - length(Conn) == Count -> - ok; - true -> - error + length(Conn) == Count -> + ok; + true -> + error end. rregister_repeat(Count) -> Conn = register_many(1, ?REG_REPEAT_LIM, "foo"), ok = unregister_many(lists:reverse(Conn)), if - length(Conn) == Count -> - ok; - true -> - error + length(Conn) == Count -> + ok; + true -> + error end. % Return count of successful registrations register_many(I, N, _Prefix) when I > N -> - test_server:format("Done with all ~n", []), + io:format("Done with all ~n", []), []; register_many(I, N, Prefix) -> Name = gen_name(Prefix, I), Port = ?DUMMY_PORT + I, % Just make it up case register_node(Name, Port) of - {ok,Sock} -> - [{Name,Port,Sock} | register_many(I + 1, N, Prefix)]; - Any -> - test_server:format("Can't register: ~w of 1..~w ~w~n", - [Name,N,Any]), - [] + {ok,Sock} -> + [{Name,Port,Sock} | register_many(I + 1, N, Prefix)]; + Any -> + test_server:format("Can't register: ~w of 1..~w ~w~n", [Name,N,Any]), + [] end. unregister_many([]) -> ok; unregister_many([{Name,_Port,Sock} | Socks]) -> case close(Sock) of - ok -> - unregister_many(Socks); - Any -> - test_server:format("Can't unregister: ~w reason ~w~n", [Name,Any]), - error + ok -> + unregister_many(Socks); + Any -> + test_server:format("Can't unregister: ~w reason ~w~n", [Name,Any]), + error end. gen_name(Str,Int) -> @@ -608,246 +530,203 @@ gen_name(Str,Int) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -no_data(doc) -> - ["Open but send no data"]; -no_data(suite) -> - []; +%% Open but send no data no_data(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), + ok = epmdrun(), + {ok,Sock} = connect(), sleep(?LONG_PAUSE), - ?line closed = recv(Sock,1), + closed = recv(Sock,1), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -one_byte(doc) -> - ["Send one byte only"]; -one_byte(suite) -> - []; +%% Send one byte only one_byte(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), - ?line ok = send(Sock,[0]), + ok = epmdrun(), + {ok,Sock} = connect(), + ok = send(Sock,[0]), sleep(?LONG_PAUSE), - ?line closed = recv(Sock,1), + closed = recv(Sock,1), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -two_bytes(doc) -> - ["Send packet size only"]; -two_bytes(suite) -> - []; +%% Send packet size only two_bytes(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), - ?line ok = send(Sock,[put16(3)]), + ok = epmdrun(), + {ok,Sock} = connect(), + ok = send(Sock,[put16(3)]), sleep(?LONG_PAUSE), - ?line closed = recv(Sock,1), + closed = recv(Sock,1), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -partial_packet(doc) -> - ["Got only part of a packet"]; -partial_packet(suite) -> - []; +%% Got only part of a packet partial_packet(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), - ?line ok = send(Sock,[put16(100),"only a few bytes"]), + ok = epmdrun(), + {ok,Sock} = connect(), + ok = send(Sock,[put16(100),"only a few bytes"]), sleep(?LONG_PAUSE), - ?line closed = recv(Sock,1), + closed = recv(Sock,1), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -zero_length(doc) -> - ["Invalid zero packet size"]; -zero_length(suite) -> - []; +%% Invalid zero packet size zero_length(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), - ?line ok = send(Sock,[0,0,0,0,0,0,0,0,0,0]), + ok = epmdrun(), + {ok,Sock} = connect(), + ok = send(Sock,[0,0,0,0,0,0,0,0,0,0]), sleep(?MEDIUM_PAUSE), - ?line closed = recv(Sock,1), + closed = recv(Sock,1), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -too_large(doc) -> - ["Invalid large packet"]; -too_large(suite) -> - []; +%% Invalid large packet too_large(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), + ok = epmdrun(), + {ok,Sock} = connect(), Size = 63000, M = lists:duplicate(Size, $z), - ?line ok = send(Sock,[put16(Size),M]), + ok = send(Sock,[put16(Size),M]), sleep(?MEDIUM_PAUSE), % With such a large packet, even the writes can fail as the % daemon closes before everything is delivered -> econnaborted case recv(Sock,1) of - closed -> ok; - {error,econnaborted} -> ok; - Other -> exit({unexpected,Other}) + closed -> ok; + {error,econnaborted} -> ok; + Other -> exit({unexpected,Other}) end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -alive_req_too_small_1(doc) -> - ["Try to register but not enough data"]; -alive_req_too_small_1(suite) -> - []; +%% Try to register but not enough data alive_req_too_small_1(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), + ok = epmdrun(), + {ok,Sock} = connect(), M = [?EPMD_ALIVE2_REQ, put16(?DUMMY_PORT),$M,0, put16(5), - put16(5),put16(0)], - ?line ok = send(Sock, [size16(M), M]), + put16(5),put16(0)], + ok = send(Sock, [size16(M), M]), sleep(?MEDIUM_PAUSE), - ?line closed = recv(Sock,1), + closed = recv(Sock,1), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -alive_req_too_small_2(doc) -> - ["Try to register but not enough data"]; -alive_req_too_small_2(suite) -> - []; +%% Try to register but not enough data alive_req_too_small_2(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), + ok = epmdrun(), + {ok,Sock} = connect(), M = [?EPMD_ALIVE2_REQ, put16(?DUMMY_PORT),$M,0, put16(5), - put16(5)], - ?line ok = send(Sock, [size16(M), M]), + put16(5)], + ok = send(Sock, [size16(M), M]), sleep(?MEDIUM_PAUSE), - ?line closed = recv(Sock,1), + closed = recv(Sock,1), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -alive_req_too_large(doc) -> - ["Try to register but node name too large"]; -alive_req_too_large(suite) -> - []; +%% Try to register but node name too large alive_req_too_large(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = connect(), - L = [ - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" - ], + ok = epmdrun(), + {ok,Sock} = connect(), + L = ["aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"], S = length(lists:flatten(L)), M = [?EPMD_ALIVE2_REQ, put16(?DUMMY_PORT),$M,0, put16(5), - put16(5), put16(S),L,put16(0)], - ?line ok = send(Sock, [size16(M), M]), + put16(5), put16(S),L,put16(0)], + ok = send(Sock, [size16(M), M]), sleep(?MEDIUM_PAUSE), - ?line {ok,[?EPMD_ALIVE2_RESP,1]} = recv(Sock,2), + {ok,[?EPMD_ALIVE2_RESP,1]} = recv(Sock,2), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -returns_valid_empty_extra(doc) -> - ["Check that an empty extra is prefixed by a two byte length"]; -returns_valid_empty_extra(suite) -> - []; +%% Check that an empty extra is prefixed by a two byte length returns_valid_empty_extra(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = register_node_v2(4711, 72, 0, 5, 5, "foo", []), - ?line {ok,#node_info{extra=[]}} = port_please_v2("foo"), - ?line ok = close(Sock), + ok = epmdrun(), + {ok,Sock} = register_node_v2(4711, 72, 0, 5, 5, "foo", []), + {ok,#node_info{extra=[]}} = port_please_v2("foo"), + ok = close(Sock), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -returns_valid_populated_extra_with_nulls(doc) -> - ["Check a populated extra with embedded null characters"]; -returns_valid_populated_extra_with_nulls(suite) -> - []; +%% Check a populated extra with embedded null characters returns_valid_populated_extra_with_nulls(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = register_node_v2(4711, 72, 0, 5, 5, "foo", "ABC\000\000"), - ?line {ok,#node_info{extra="ABC\000\000"}} = port_please_v2("foo"), - ?line ok = close(Sock), + ok = epmdrun(), + {ok,Sock} = register_node_v2(4711, 72, 0, 5, 5, "foo", "ABC\000\000"), + {ok,#node_info{extra="ABC\000\000"}} = port_please_v2("foo"), + ok = close(Sock), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -names_stdout(doc) -> - ["Test that epmd -names prints registered nodes to stdout"]; -names_stdout(suite) -> - []; +%% Test that epmd -names prints registered nodes to stdout names_stdout(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,Sock} = register_node("foobar"), - ?line ok = epmdrun("-names"), - ?line {ok, Data} = receive {_Port, {data, D}} -> {ok, D} - after 10000 -> {error, timeout} - end, - ?line {match,_} = re:run(Data, "^epmd: up and running", [multiline]), - ?line {match,_} = re:run(Data, "^name foobar at port", [multiline]), - ?line ok = close(Sock), + ok = epmdrun(), + {ok,Sock} = register_node("foobar"), + ok = epmdrun("-names"), + {ok, Data} = receive {_Port, {data, D}} -> {ok, D} + after 10000 -> {error, timeout} + end, + {match,_} = re:run(Data, "^epmd: up and running", [multiline]), + {match,_} = re:run(Data, "^name foobar at port", [multiline]), + ok = close(Sock), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -buffer_overrun_1(suite) -> - []; -buffer_overrun_1(doc) -> - ["Test security vulnerability in fake extra lengths in alive2_req"]; +%% Test security vulnerability in fake extra lengths in alive2_req buffer_overrun_1(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line true = alltrue([hostile(N) || N <- lists:seq(1,10000)]), + ok = epmdrun(), + true = alltrue([hostile(N) || N <- lists:seq(1,10000)]), ok. -buffer_overrun_2(suite) -> - []; -buffer_overrun_2(doc) -> - ["Test security vulnerability in fake extra lengths in alive2_req"]; + +%% Test security vulnerability in fake extra lengths in alive2_req buffer_overrun_2(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line [false | Rest] = [hostile2(N) || N <- lists:seq(255*4,10000)], - ?line true = alltrue(Rest), + ok = epmdrun(), + [false | Rest] = [hostile2(N) || N <- lists:seq(255*4,10000)], + true = alltrue(Rest), ok. hostile(N) -> try - Bin= <<$x:8,4747:16,$M:8,0:8,5:16,5:16,5:16,"gurka",N:16>>, - S = size(Bin), - {ok,E}=connect_sturdy(), - gen_tcp:send(E,[<<S:16>>,Bin]), - closed = recv(E,1), - gen_tcp:close(E), - true + Bin= <<$x:8,4747:16,$M:8,0:8,5:16,5:16,5:16,"gurka",N:16>>, + S = size(Bin), + {ok,E}=connect_sturdy(), + gen_tcp:send(E,[<<S:16>>,Bin]), + closed = recv(E,1), + gen_tcp:close(E), + true catch - _:_ -> - false + _:_ -> + false end. hostile2(N) -> try - B2 = list_to_binary(lists:duplicate(N,255)), - Bin= <<$x:8,4747:16,$M:8,0:8,5:16,5:16,5:16,"gurka",N:16,B2/binary>>, - S = size(Bin), - {ok,E}=connect_sturdy(), - gen_tcp:send(E,[<<S:16>>,Bin]), - Z = recv(E,2), - gen_tcp:close(E), - (Z =:= closed) or (Z =:= {ok, [$y,1]}) + B2 = list_to_binary(lists:duplicate(N,255)), + Bin= <<$x:8,4747:16,$M:8,0:8,5:16,5:16,5:16,"gurka",N:16,B2/binary>>, + S = size(Bin), + {ok,E}=connect_sturdy(), + gen_tcp:send(E,[<<S:16>>,Bin]), + Z = recv(E,2), + gen_tcp:close(E), + (Z =:= closed) or (Z =:= {ok, [$y,1]}) catch - _A:_B -> - false + _A:_B -> + false end. alltrue([]) -> @@ -857,134 +736,124 @@ alltrue([true|T]) -> alltrue([_|_]) -> false. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -no_nonlocal_register(suite) -> - []; -no_nonlocal_register(doc) -> - ["Ensure that we cannot register throug a nonlocal connection"]; + +%% Ensure that we cannot register throug a nonlocal connection no_nonlocal_register(Config) when is_list(Config) -> - ?line case {os:find_executable("ssh"),ct:get_config(ssh_proxy_host)} of - {SSH,Name} when is_list(Name), is_list(SSH) -> - do_no_nonlocal_register(Config,Name); - {false,_} -> - {skip, "No ssh command found to create proxy"}; - _ -> - {skip, "No ssh_proxy_host configured in ts.config"} - end. + case {os:find_executable("ssh"),ct:get_config(ssh_proxy_host)} of + {SSH,Name} when is_list(Name), is_list(SSH) -> + do_no_nonlocal_register(Config,Name); + {false,_} -> + {skip, "No ssh command found to create proxy"}; + _ -> + {skip, "No ssh_proxy_host configured in ts.config"} + end. do_no_nonlocal_register(Config,SSHHost) when is_list(Config) -> - ?line ok = epmdrun(), - ?line ProxyPort = proxy_port(), - ?line ok = ssh_proxy(SSHHost,ProxyPort), + ok = epmdrun(), + ProxyPort = proxy_port(), + ok = ssh_proxy(SSHHost,ProxyPort), Res = try - ?line Name = "gurka_" - %++ - %integer_to_list(A1)++"_"++ - %integer_to_list(A2)++"_"++ - %integer_to_list(A3)++"_"++ - %integer_to_list(A4) - , - ?line Bname = list_to_binary(Name), - ?line NameS = byte_size(Bname), - ?line Bin= <<$x:8,4747:16,$M:8,0:8,5:16, - 5:16,NameS:16,Bname/binary, - 0:16>>, - ?line S = size(Bin), - ?line {ok, E} = connect("localhost",ProxyPort,passive), - ?line gen_tcp:send(E,[<<S:16>>,Bin]), - ?line closed = recv(E,1), - ?line gen_tcp:close(E), - true - catch - _:_ -> - false - end, + Name = "gurka_" + %++ + %integer_to_list(A1)++"_"++ + %integer_to_list(A2)++"_"++ + %integer_to_list(A3)++"_"++ + %integer_to_list(A4) + , + Bname = list_to_binary(Name), + NameS = byte_size(Bname), + Bin= <<$x:8,4747:16,$M:8,0:8,5:16, + 5:16,NameS:16,Bname/binary, + 0:16>>, + S = size(Bin), + {ok, E} = connect("localhost",ProxyPort,passive), + gen_tcp:send(E,[<<S:16>>,Bin]), + closed = recv(E,1), + gen_tcp:close(E), + true + catch + _:_ -> + false + end, %erlang:display(Res), true = Res, ok. -no_nonlocal_kill(suite) -> - []; -no_nonlocal_kill(doc) -> - ["Ensure that we cannot kill through nonlocal connection"]; +%% Ensure that we cannot kill through nonlocal connection no_nonlocal_kill(Config) when is_list(Config) -> - ?line case {os:find_executable("ssh"),ct:get_config(ssh_proxy_host)} of - {SSH,Name} when is_list(Name), is_list(SSH) -> - do_no_nonlocal_kill(Config,Name); - {false,_} -> - {skip, "No ssh command found to create proxy"}; - _ -> - {skip, "No ssh_proxy_host configured in ts.config"} - end. + case {os:find_executable("ssh"),ct:get_config(ssh_proxy_host)} of + {SSH,Name} when is_list(Name), is_list(SSH) -> + do_no_nonlocal_kill(Config,Name); + {false,_} -> + {skip, "No ssh command found to create proxy"}; + _ -> + {skip, "No ssh_proxy_host configured in ts.config"} + end. do_no_nonlocal_kill(Config,SSHHost) when is_list(Config) -> - ?line ok = epmdrun(), - ?line ProxyPort = proxy_port(), - ?line ok = ssh_proxy(SSHHost,ProxyPort), + ok = epmdrun(), + ProxyPort = proxy_port(), + ok = ssh_proxy(SSHHost,ProxyPort), Res = try - {ok, E} = connect("localhost",ProxyPort,passive), - M = [?EPMD_KILL_REQ], - send(E, [size16(M), M]), - closed = recv(E,2), - gen_tcp:close(E), - sleep(?MEDIUM_PAUSE), - {ok, E2} = connect("localhost",ProxyPort,passive), - gen_tcp:close(E2), - true - catch - _:_ -> - false - end, + {ok, E} = connect("localhost",ProxyPort,passive), + M = [?EPMD_KILL_REQ], + send(E, [size16(M), M]), + closed = recv(E,2), + gen_tcp:close(E), + sleep(?MEDIUM_PAUSE), + {ok, E2} = connect("localhost",ProxyPort,passive), + gen_tcp:close(E2), + true + catch + _:_ -> + false + end, %erlang:display(Res), true = Res, ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -no_live_killing(doc) -> - ["Dont allow killing with live nodes or any unregistering w/o -relaxed_command_check"]; -no_live_killing(suite) -> - []; + +%% Dont allow killing with live nodes or any unregistering w/o -relaxed_command_check no_live_killing(Config) when is_list(Config) -> - ?line ok = epmdrun(), - ?line {ok,RSock} = register_node("foo"), - ?line {ok,Sock} = connect(), - ?line M = [?EPMD_KILL_REQ], - ?line ok = send(Sock,[size16(M),M]), - ?line {ok,"NO"} = recv(Sock,2), - ?line close(Sock), - ?line {ok,Sock2} = connect(), - ?line M2 = [?EPMD_STOP_REQ,"foo"], - ?line ok = send(Sock2,[size16(M2),M2]), - ?line closed = recv(Sock2,1), - ?line close(Sock2), - ?line close(RSock), - ?line sleep(?MEDIUM_PAUSE), - ?line {ok,Sock3} = connect(), - ?line M3 = [?EPMD_KILL_REQ], - ?line ok = send(Sock3,[size16(M3),M3]), - ?line {ok,"OK"} = recv(Sock3,2), - ?line close(Sock3), + ok = epmdrun(), + {ok,RSock} = register_node("foo"), + {ok,Sock} = connect(), + M = [?EPMD_KILL_REQ], + ok = send(Sock,[size16(M),M]), + {ok,"NO"} = recv(Sock,2), + close(Sock), + {ok,Sock2} = connect(), + M2 = [?EPMD_STOP_REQ,"foo"], + ok = send(Sock2,[size16(M2),M2]), + closed = recv(Sock2,1), + close(Sock2), + close(RSock), + sleep(?MEDIUM_PAUSE), + {ok,Sock3} = connect(), + M3 = [?EPMD_KILL_REQ], + ok = send(Sock3,[size16(M3),M3]), + {ok,"OK"} = recv(Sock3,2), + close(Sock3), ok. -socket_reset_before_alive2_reply_is_written(doc) -> - ["Check for regression - don't make zombie from node which " - "sends TCP RST at wrong time"]; -socket_reset_before_alive2_reply_is_written(suite) -> - []; +%% Check for regression - don't make zombie from node which +%% sends TCP RST at wrong time socket_reset_before_alive2_reply_is_written(Config) when is_list(Config) -> %% - delay_write for easier triggering of race condition %% - relaxed_command_check for gracefull shutdown of epmd even if there %% is stuck node. - ?line ok = epmdrun("-delay_write 1 -relaxed_command_check"), + ok = epmdrun("-delay_write 1 -relaxed_command_check"), %% We can't use send_req/1 directly as we want to do inet:setopts/2 %% on our socket. - ?line {ok, Sock} = connect(), + {ok, Sock} = connect(), %% Issuing close/1 on such socket will result in immediate RST packet. - ?line ok = inet:setopts(Sock, [{linger, {true, 0}}]), + ok = inet:setopts(Sock, [{linger, {true, 0}}]), Req = alive2_req(4711, 77, 0, 5, 5, "test", []), - ?line ok = send(Sock, [size16(Req), Req]), + ok = send(Sock, [size16(Req), Req]), timer:sleep(500), %% Wait for the first 1/2 of delay_write before closing - ?line ok = close(Sock), + ok = close(Sock), timer:sleep(500 + ?SHORT_PAUSE), %% Wait for the other 1/2 of delay_write @@ -992,11 +861,11 @@ socket_reset_before_alive2_reply_is_written(Config) when is_list(Config) -> %% Should be removed when this is issue is fixed there. timer:sleep(1000), - ?line {ok, SockForNames} = connect_active(), + {ok, SockForNames} = connect_active(), %% And there should be no stuck nodes - ?line {ok, []} = do_get_names(SockForNames), - ?line ok = close(SockForNames), + {ok, []} = do_get_names(SockForNames), + ok = close(SockForNames), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1005,14 +874,14 @@ socket_reset_before_alive2_reply_is_written(Config) when is_list(Config) -> cleanup() -> sleep(?MEDIUM_PAUSE), case connect() of - {ok,Sock} -> - M = [?EPMD_KILL_REQ], - send(Sock, [size16(M), M]), - recv(Sock,length("OK")), - close(Sock), - sleep(?MEDIUM_PAUSE); - _ -> - true + {ok,Sock} -> + M = [?EPMD_KILL_REQ], + send(Sock, [size16(M), M]), + recv(Sock,length("OK")), + close(Sock), + sleep(?MEDIUM_PAUSE); + _ -> + true end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1022,11 +891,11 @@ proxy_port() -> ?PORT+1. ssh_proxy(SSHHost,ProxyPort) -> - ?line Host = lists:nth(2,string:tokens(atom_to_list(node()),"@")), + Host = lists:nth(2,string:tokens(atom_to_list(node()),"@")), % Requires proxy to be a unix host with the command 'read' accessible - ?line osrun("ssh -L "++integer_to_list(ProxyPort)++":"++Host++":" - ++integer_to_list(?PORT)++" "++SSHHost++" read"). - + osrun("ssh -L "++integer_to_list(ProxyPort)++":"++Host++":" + ++integer_to_list(?PORT)++" "++SSHHost++" read"). + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1036,21 +905,21 @@ epmdrun() -> epmdrun([]). epmdrun(Args) -> case os:find_executable(epmd) of - false -> - {error, {could_not_find_epmd_in_path}}; - Path -> - epmdrun(Path,Args) + false -> + {error, {could_not_find_epmd_in_path}}; + Path -> + epmdrun(Path,Args) end. epmdrun(Epmd,Args0) -> - %% test_server:format("epmdrun() => Epmd = ~p",[Epmd]), + %% test_server:format("epmdrun() => Epmd = ~p",[Epmd]), Args = case Args0 of - [] -> - []; - O -> - " "++O - end, - osrun("\"" ++ Epmd ++ "\"" ++ " " ?EPMDARGS " -port " ++ integer_to_list(?PORT) ++ Args). + [] -> + []; + O -> + " "++O + end, + osrun("\"" ++ Epmd ++ "\"" ++ " " ?EPMDARGS " -port " ++ integer_to_list(?PORT) ++ Args). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1059,7 +928,7 @@ epmdrun(Epmd,Args0) -> osrun(Cmd) -> _ = open_port({spawn, Cmd}, []), ok. - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Wrappers of TCP functions @@ -1084,16 +953,16 @@ connect(Addr, Port, Mode) -> connect(Addr, Port, Mode, ?CONN_SLEEP, ?CONN_RETRY). connect(Addr, Port, Mode, Sleep, Retry) -> case connect_repeat(Addr, Retry, Port, Mode, Sleep) of - {ok,Sock} -> - {ok,Sock}; - {error,timeout} -> - timeout; - {error,Reason} -> - test_server:format("connect: error: ~w~n",[Reason]), - error; - Any -> - test_server:format("connect: unknown message: ~w~n",[Any]), - exit(1) + {ok,Sock} -> + {ok,Sock}; + {error,timeout} -> + timeout; + {error,Reason} -> + test_server:format("connect: error: ~w~n",[Reason]), + error; + Any -> + test_server:format("connect: unknown message: ~w~n",[Any]), + exit(1) end. @@ -1104,33 +973,33 @@ connect_repeat(Addr, 1, Port, Mode, _Sleep) -> connect_mode(Addr,Port, Mode); connect_repeat(Addr,Retry, Port, Mode, Sleep) -> case connect_mode(Addr,Port, Mode) of - {ok,Sock} -> - {ok,Sock}; - {error,Reason} -> - test_server:format("connect: error: ~w~n",[Reason]), - timer:sleep(Sleep), - connect_repeat(Addr, Retry - 1, Port, Mode, Sleep); - Any -> - test_server:format("connect: unknown message: ~w~n",[Any]), - exit(1) + {ok,Sock} -> + {ok,Sock}; + {error,Reason} -> + test_server:format("connect: error: ~w~n",[Reason]), + timer:sleep(Sleep), + connect_repeat(Addr, Retry - 1, Port, Mode, Sleep); + Any -> + test_server:format("connect: unknown message: ~w~n",[Any]), + exit(1) end. connect_mode(Addr,Port, active) -> gen_tcp:connect(Addr, Port, [{packet, 0}], ?CONN_TIMEOUT); connect_mode(Addr, Port, passive) -> gen_tcp:connect(Addr, Port, [{packet, 0}, {active, false}], - ?CONN_TIMEOUT). + ?CONN_TIMEOUT). close(Sock) -> case gen_tcp:close(Sock) of - {error,_} -> - error; - ok -> - ok; - Any -> - test_server:format("unknown message: ~w~n",[Any]), - exit(1) + {error,_} -> + error; + ok -> + ok; + Any -> + test_server:format("unknown message: ~w~n",[Any]), + exit(1) end. recv(Sock, Len) -> @@ -1138,19 +1007,19 @@ recv(Sock, Len) -> recv(Sock, Len, Timeout) -> case gen_tcp:recv(Sock, Len, Timeout) of - {ok,[]} -> % Should not be the case - recv(Sock, 1, 1); % any longer - {ok,Data} -> - {ok,Data}; - {error,timeout} -> - timeout; - {error,closed} -> - closed; - {error,_}=Error -> - Error; - Any -> - test_server:format("unknown message: ~w~n",[Any]), - exit(1) + {ok,[]} -> % Should not be the case + recv(Sock, 1, 1); % any longer + {ok,Data} -> + {ok,Data}; + {error,timeout} -> + timeout; + {error,closed} -> + closed; + {error,_}=Error -> + Error; + Any -> + test_server:format("unknown message: ~w~n",[Any]), + exit(1) end. %% Send data to socket. The list can be non flat and contain @@ -1159,12 +1028,12 @@ recv(Sock, Len, Timeout) -> send(Sock, SendSpec) -> case send(SendSpec, [], Sock) of - {ok,[]} -> - ok; - {ok,RevBytes} -> - send_direct(Sock, lists:reverse(RevBytes)); - Any -> - Any + {ok,[]} -> + ok; + {ok,RevBytes} -> + send_direct(Sock, lists:reverse(RevBytes)); + Any -> + Any end. @@ -1179,54 +1048,54 @@ send([Byte | Spec], RevBytes, Sock) when is_integer(Byte) -> send(Spec, [Byte | RevBytes], Sock); send([List | Spec], RevBytes, Sock) when is_list(List) -> case send(List, RevBytes, Sock) of - {ok,Left} -> - send(Spec, Left, Sock); - Other -> - Other + {ok,Left} -> + send(Spec, Left, Sock); + Other -> + Other end; send([d | Spec], RevBytes, Sock) -> send([{d,1000} | Spec], RevBytes, Sock); send([{d,S} | Spec], RevBytes, Sock) -> case send_direct(Sock, lists:reverse(RevBytes)) of - ok -> - timer:sleep(S), - send(Spec, [], Sock); - Any -> - Any + ok -> + timer:sleep(S), + send(Spec, [], Sock); + Any -> + Any end. %%%% send_direct(Sock, Bytes) -> case gen_tcp:send(Sock, Bytes) of - ok -> - ok; - {error, closed} -> - closed; - {error, _Reason} -> - error; - Any -> - test_server:format("unknown message: ~w~n",[Any]), - Any + ok -> + ok; + {error, closed} -> + closed; + {error, _Reason} -> + error; + Any -> + test_server:format("unknown message: ~w~n",[Any]), + Any end. send_req(Req) -> send_req(Req, "localhost"). send_req(Req, Addr) -> case connect(Addr) of - {ok,Sock} -> - case send(Sock, [size16(Req), Req]) of - ok -> - {ok,Sock}; - Other -> - test_server:format("Failed to send ~w on sock ~w: ~w~n", - [Req,Sock,Other]), - error - end; - Other -> - test_server:format("Connect failed when sending ~w: ~p~n", - [Req, Other]), - error + {ok,Sock} -> + case send(Sock, [size16(Req), Req]) of + ok -> + {ok,Sock}; + Other -> + test_server:format("Failed to send ~w on sock ~w: ~w~n", + [Req,Sock,Other]), + error + end; + Other -> + test_server:format("Connect failed when sending ~w: ~p~n", + [Req, Other]), + error end. recv_until_sock_closes(Sock) -> @@ -1234,12 +1103,12 @@ recv_until_sock_closes(Sock) -> recv_until_sock_closes_2(Sock,AccData) -> case recv(Sock,0) of - {ok,Data} -> - recv_until_sock_closes_2(Sock,AccData++Data); - closed -> - {ok,AccData}; - Other -> - Other + {ok,Data} -> + recv_until_sock_closes_2(Sock,AccData++Data); + closed -> + {ok,AccData}; + Other -> + Other end. sleep(MilliSeconds) -> @@ -1261,7 +1130,7 @@ flat_count([H|T], N) when is_list(H) -> flat_count([_|T], N) -> flat_count(T, N); flat_count([], N) -> N. - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index 5a5021b003..42da05b1f7 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -150,6 +150,9 @@ static char *plush_val_switches[] = { "ms", "mbs", "pds", + "max", + "maxk", + "maxel", "mqd", "", NULL @@ -842,7 +845,6 @@ int main(int argc, char **argv) if (argv[i][3] != '\0') goto the_default; } -#ifdef ERTS_DIRTY_SCHEDULERS else if (argv[i][2] == 'D') { char* type = argv[i]+3; if (strncmp(type, "cpu", 3) != 0 && @@ -854,7 +856,6 @@ int main(int argc, char **argv) (argv[i][3] == 'i' && argv[i][5] != '\0')) goto the_default; } -#endif else if (argv[i][2] != '\0') goto the_default; if (i+1 >= argc) diff --git a/erts/include/erl_native_features_config.h.in b/erts/include/erl_native_features_config.h.in deleted file mode 100644 index 59a5dde09e..0000000000 --- a/erts/include/erl_native_features_config.h.in +++ /dev/null @@ -1,22 +0,0 @@ -/* - * %CopyrightBegin% - * - * Copyright Ericsson AB 2004-2016. All Rights Reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - * %CopyrightEnd% - */ - -/* Dirty scheduler support */ -#undef ERL_NIF_DIRTY_SCHEDULER_SUPPORT diff --git a/erts/lib_src/Makefile.in b/erts/lib_src/Makefile.in index 6e2f236bdf..601f3917a8 100644 --- a/erts/lib_src/Makefile.in +++ b/erts/lib_src/Makefile.in @@ -465,7 +465,6 @@ RELSYSDIR = $(RELEASE_PATH)/erts-$(VSN) RELEASE_INCLUDES= \ $(ERTS_INCL)/erl_memory_trace_parser.h \ $(ERTS_INCL)/$(TARGET)/erl_int_sizes_config.h \ - $(ERTS_INCL)/$(TARGET)/erl_native_features_config.h \ $(ERTS_INCL)/erl_fixed_size_int_types.h RELEASE_LIBS=$(ERTS_LIBS) diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex 3e82b8e20e..de2693472a 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erl_tracer.beam b/erts/preloaded/ebin/erl_tracer.beam Binary files differindex 67d96d0f0d..69804540c9 100644 --- a/erts/preloaded/ebin/erl_tracer.beam +++ b/erts/preloaded/ebin/erl_tracer.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 321e4e7e1b..8379bf1768 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/erts_code_purger.beam b/erts/preloaded/ebin/erts_code_purger.beam Binary files differindex 0c667663fd..b1da0aa861 100644 --- a/erts/preloaded/ebin/erts_code_purger.beam +++ b/erts/preloaded/ebin/erts_code_purger.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differindex ed2e328f5b..cc4f3dbdaf 100644 --- a/erts/preloaded/ebin/erts_internal.beam +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex 23ebd2cad9..7b5797e90a 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/otp_ring0.beam b/erts/preloaded/ebin/otp_ring0.beam Binary files differindex b63fa24848..f893b9c181 100644 --- a/erts/preloaded/ebin/otp_ring0.beam +++ b/erts/preloaded/ebin/otp_ring0.beam diff --git a/erts/preloaded/ebin/prim_eval.beam b/erts/preloaded/ebin/prim_eval.beam Binary files differindex 7c8fdc82c0..e4ce601c03 100644 --- a/erts/preloaded/ebin/prim_eval.beam +++ b/erts/preloaded/ebin/prim_eval.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex d4559bc9ed..babba3081f 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex 5a0678632e..0521060e34 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/prim_zip.beam b/erts/preloaded/ebin/prim_zip.beam Binary files differindex 3d32dd8449..e1faca7d96 100644 --- a/erts/preloaded/ebin/prim_zip.beam +++ b/erts/preloaded/ebin/prim_zip.beam diff --git a/erts/preloaded/ebin/zlib.beam b/erts/preloaded/ebin/zlib.beam Binary files differindex ced9f9c952..5d3cbc1b36 100644 --- a/erts/preloaded/ebin/zlib.beam +++ b/erts/preloaded/ebin/zlib.beam diff --git a/erts/preloaded/src/erl_tracer.erl b/erts/preloaded/src/erl_tracer.erl index de1e9ca01e..fe15812535 100644 --- a/erts/preloaded/src/erl_tracer.erl +++ b/erts/preloaded/src/erl_tracer.erl @@ -41,10 +41,14 @@ on_load() -> %%% NIF placeholders %%% --spec enabled(Tag :: trace_tag() | seq_trace | trace_status, +-spec enabled(Tag :: trace_status, TracerState :: tracer_state(), Tracee :: tracee()) -> - trace | discard | remove. + trace | remove; + (Tag :: trace_tag() | seq_trace, + TracerState :: tracer_state(), + Tracee :: tracee()) -> + trace | discard. enabled(_, _, _) -> erlang:nif_error(nif_not_loaded). diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index 90fd536b15..3d152c4e92 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -2073,6 +2073,9 @@ open_port(PortName, PortSettings) -> (min_bin_vheap_size, MinBinVHeapSize) -> OldMinBinVHeapSize when MinBinVHeapSize :: non_neg_integer(), OldMinBinVHeapSize :: non_neg_integer(); + (max_heap_size, MaxHeapSize) -> OldMaxHeapSize when + MaxHeapSize :: max_heap_size(), + OldMaxHeapSize :: max_heap_size(); (message_queue_data, MQD) -> OldMQD when MQD :: message_queue_data(), OldMQD :: message_queue_data(); @@ -2154,6 +2157,7 @@ process_flag(_Flag, _Value) -> {messages, MessageQueue :: [term()]} | {min_heap_size, MinHeapSize :: non_neg_integer()} | {min_bin_vheap_size, MinBinVHeapSize :: non_neg_integer()} | + {max_heap_size, MaxHeapSize :: max_heap_size()} | {monitored_by, Pids :: [pid()]} | {monitors, Monitors :: [{process, Pid :: pid() | @@ -2238,6 +2242,7 @@ setelement(_Index, _Tuple1, _Value) -> | {priority, Level :: priority_level()} | {fullsweep_after, Number :: non_neg_integer()} | {min_heap_size, Size :: non_neg_integer()} + | {max_heap_size, Size :: max_heap_size()} | {min_bin_vheap_size, VSize :: non_neg_integer()}. spawn_opt(_Tuple) -> erlang:nif_error(undefined). @@ -2330,6 +2335,9 @@ subtract(_,_) -> OldMinBinVHeapSize when MinBinVHeapSize :: non_neg_integer(), OldMinBinVHeapSize :: non_neg_integer(); + (max_heap_size, MaxHeapSize) -> OldMaxHeapSize when + MaxHeapSize :: max_heap_size(), + OldMaxHeapSize :: max_heap_size(); (multi_scheduling, BlockState) -> OldBlockState when BlockState :: block | unblock | block_normal | unblock_normal, OldBlockState :: blocked | disabled | enabled; @@ -2511,6 +2519,7 @@ tuple_to_list(_Tuple) -> logical_processors_available | logical_processors_online) -> unknown | pos_integer(); (machine) -> string(); + (max_heap_size) -> {max_heap_size, MaxHeapSize :: max_heap_size()}; (message_queue_data) -> message_queue_data(); (min_heap_size) -> {min_heap_size, MinHeapSize :: pos_integer()}; (min_bin_vheap_size) -> {min_bin_vheap_size, @@ -2648,6 +2657,13 @@ spawn_monitor(M, F, A) -> erlang:error(badarg, [M,F,A]). +-type max_heap_size() :: + Size :: non_neg_integer() + %% TODO change size => to := when -type maps support is finalized + | #{ size => non_neg_integer(), + kill => boolean(), + error_logger => boolean() }. + -type spawn_opt_option() :: link | monitor @@ -2655,6 +2671,7 @@ spawn_monitor(M, F, A) -> | {fullsweep_after, Number :: non_neg_integer()} | {min_heap_size, Size :: non_neg_integer()} | {min_bin_vheap_size, VSize :: non_neg_integer()} + | {max_heap_size, Size :: max_heap_size()} | {message_queue_data, MQD :: message_queue_data()}. -spec spawn_opt(Fun, Options) -> pid() | {pid(), reference()} when diff --git a/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl b/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl index 8f4cf2b9ec..325bea5879 100644 --- a/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl +++ b/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl @@ -500,9 +500,7 @@ gen_decode_sof(Erules,TypeName,_InnerTypeName,D) when is_record(D,type) -> Atom when is_atom(Atom) -> Atom; _ -> TypeNameSuffix end, - ObjFun = false, - gen_dec_line(Erules,TypeName,ContName,[],Cont,mandatory,ObjFun), - %% gen_dec_line_sof(Erules,Typename,ContName,Cont,ObjFun), + gen_dec_line(Erules,TypeName,ContName,[],Cont,mandatory), emit([" || ",{curr,v}," <- ",{curr,tlv},"].",nl,nl,nl]). @@ -869,7 +867,7 @@ gen_dec_choice_cases(Erules,TopType, [H|T]) -> (?ASN1CT_GEN_BER:decode_class(T1class) bsl 10) + T1number,",_} -> ",nl]), emit([indent(8),"{",{asis,Cname},", "]), - gen_dec_line(Erules,TopType,Cname,[],Type,Prop,false), + gen_dec_line(Erules,TopType,Cname,[],Type,Prop), emit(["};",nl,nl]), Fun(Tail,Fun); ([],_) -> @@ -896,7 +894,7 @@ gen_dec_choice_cases(Erules,TopType, [H|T]) -> (?ASN1CT_GEN_BER:decode_class(FirstT#tag.class) bsl 10) + FirstT#tag.number,", ",{curr,v},"} -> ",nl]), emit([indent(8),"{",{asis,Cname},", "]), - gen_dec_line(Erules,TopType,Cname,[],Type#type{tag=RestT},Prop,false), + gen_dec_line(Erules,TopType,Cname,[],Type#type{tag=RestT},Prop), emit(["};",nl,nl]) end, gen_dec_choice_cases(Erules,TopType, T). @@ -1060,8 +1058,14 @@ gen_optormand_case({'DEFAULT',DefaultValue}, Erules, _TopType, end, emit([indent(9),"_ ->",nl,indent(12)]) end. - +%% Use for SEQUENCE OF and CHOICE. +gen_dec_line(Erules,TopType,Cname,CTags,Type,OptOrMand) -> + %% The matching on the next line is an assertion. + {[],[]} = gen_dec_line(Erules,TopType,Cname,CTags,Type,OptOrMand,false), + ok. + +%% Use for SEQUENCE. gen_dec_line(Erules,TopType,Cname,CTags,Type,OptOrMand,DecObjInf) -> BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(v)), Tag = diff --git a/lib/common_test/doc/src/ct_netconfc.xml b/lib/common_test/doc/src/ct_netconfc.xml index 9d1ff25a3a..e6930b30d5 100644 --- a/lib/common_test/doc/src/ct_netconfc.xml +++ b/lib/common_test/doc/src/ct_netconfc.xml @@ -201,7 +201,8 @@ information, see module <seealso marker="ct"><c>ct</c></seealso>.</p> </item> - <tag><c>host() = <seealso marker="kernel:inet#type-hostname"><c>inet:hostname()</c></seealso> | <seealso marker="kernel:inet#type-ip_address"><c>inet:ip_address()</c></seealso></c></tag> + <tag><c>host() = </c><seealso marker="kernel:inet#type-hostname"><c>inet:hostname()</c></seealso> + <c> | </c><seealso marker="kernel:inet#type-ip_address"><c>inet:ip_address()</c></seealso></tag> <item><marker id="type-host"/></item> <tag><c>key_or_name() = server_id() | target_name()</c></tag> @@ -218,7 +219,9 @@ <tag><c>notification_content() = [event_time() | simple_xml()]</c></tag> <item><marker id="type-notification_content"/> </item> - <tag><c>option() = {ssh, host()} | {port, <seealso marker="kernel:inet#type-port_number"><c>inet:port_number()</c></seealso>} | {timeout, timeout()} | SshConnectOption</c></tag> + <tag><c>option() = {ssh, host()} | {port, </c> + <seealso marker="kernel:inet#type-port_number"><c>inet:port_number()</c></seealso> + <c>} | {timeout, timeout()} | SshConnectOption</c></tag> <item><marker id="type-option"/> <p><c>SshConnectOption</c> is any valid option to diff --git a/lib/common_test/src/ct_logs.erl b/lib/common_test/src/ct_logs.erl index 29b38e9748..e6d683c8a9 100644 --- a/lib/common_test/src/ct_logs.erl +++ b/lib/common_test/src/ct_logs.erl @@ -269,7 +269,7 @@ cast(Msg) -> %%% <p>This function is called by ct_framework:init_tc/3</p> init_tc(RefreshLog) -> call({init_tc,self(),group_leader(),RefreshLog}), - io:format(["$tc_html",xhtml("", "<br />")]), + tc_io_format(group_leader(), xhtml("", "<br />"), []), ok. %%%----------------------------------------------------------------- @@ -800,7 +800,8 @@ logger_loop(State) -> %% make sure no IO for this test case from the %% CT logger gets rejected test_server:permit_io(GL, self()), - print_style(GL,GL,State#logger_state.stylesheet), + IoFormat = fun tc_io_format/3, + print_style(GL, IoFormat, State#logger_state.stylesheet), set_evmgr_gl(GL), TCGLs = add_tc_gl(TCPid,GL,State), if not RefreshLog -> @@ -931,7 +932,7 @@ print_to_log(sync, FromPid, Category, TCGL, Content, EscChars, State) -> if FromPid /= TCGL -> IoFun = create_io_fun(FromPid, CtLogFd, EscChars), IoList = lists:foldl(IoFun, [], Content), - try io:format(TCGL,["$tc_html","~ts"], [IoList]) of + try tc_io_format(TCGL, "~ts", [IoList]) of ok -> ok catch _:_ -> @@ -962,7 +963,7 @@ print_to_log(async, FromPid, Category, TCGL, Content, EscChars, State) -> case erlang:is_process_alive(TCGL) of true -> - try io:format(TCGL, ["$tc_html","~ts"], + try tc_io_format(TCGL, "~ts", [lists:foldl(IoFun,[],Content)]) of _ -> ok catch @@ -1113,27 +1114,25 @@ open_ctlog(MiscIoName) -> "View I/O logged after the test run</a></li>\n</ul>\n", [MiscIoName,MiscIoName]), - print_style(Fd,group_leader(),undefined), + print_style(Fd, fun io:format/3, undefined), io:format(Fd, xhtml("<br><h2>Progress Log</h2>\n<pre>\n", "<br />\n<h4>PROGRESS LOG</h4>\n<pre>\n"), []), Fd. -print_style(Fd,GL,undefined) -> +print_style(Fd, IoFormat, undefined) -> case basic_html() of true -> Style = "<style>\n div.ct_internal { background:lightgrey; color:black; }\n div.default { background:lightgreen; color:black; }\n </style>\n", - if Fd == GL -> io:format(["$tc_html",Style], []); - true -> io:format(Fd, Style, []) - end; + IoFormat(Fd, Style, []); _ -> ok end; -print_style(Fd,GL,StyleSheet) -> +print_style(Fd, IoFormat, StyleSheet) -> case file:read_file(StyleSheet) of {ok,Bin} -> Str = b2s(Bin,encoding(StyleSheet)), @@ -1146,36 +1145,55 @@ print_style(Fd,GL,StyleSheet) -> N1 -> N1 end, if (Pos0 == 0) and (Pos1 /= 0) -> - print_style_error(Fd,GL,StyleSheet,missing_style_start_tag); + print_style_error(Fd, IoFormat, + StyleSheet, missing_style_start_tag); (Pos0 /= 0) and (Pos1 == 0) -> - print_style_error(Fd,GL,StyleSheet,missing_style_end_tag); + print_style_error(Fd, IoFormat, + StyleSheet,missing_style_end_tag); Pos0 /= 0 -> Style = string:sub_string(Str,Pos0,Pos1+7), - if Fd == GL -> io:format(Fd,["$tc_html","~ts\n"],[Style]); - true -> io:format(Fd,"~ts\n",[Style]) - end; + IoFormat(Fd,"~ts\n",[Style]); Pos0 == 0 -> - if Fd == GL -> io:format(Fd,["$tc_html","<style>\n~ts</style>\n"],[Str]); - true -> io:format(Fd,"<style>\n~ts</style>\n",[Str]) - end + IoFormat(Fd,"<style>\n~ts</style>\n",[Str]) end; {error,Reason} -> - print_style_error(Fd,GL,StyleSheet,Reason) + print_style_error(Fd,IoFormat,StyleSheet,Reason) end. -print_style_error(Fd,GL,StyleSheet,Reason) -> +print_style_error(Fd, IoFormat, StyleSheet, Reason) -> IO = io_lib:format("\n<!-- Failed to load stylesheet ~ts: ~p -->\n", [StyleSheet,Reason]), - if Fd == GL -> io:format(Fd,["$tc_html",IO],[]); - true -> io:format(Fd,IO,[]) - end, - print_style(Fd,GL,undefined). + IoFormat(Fd, IO, []), + print_style(Fd, IoFormat, undefined). close_ctlog(Fd) -> io:format(Fd, "\n</pre>\n", []), io:format(Fd, [xhtml("<br><br>\n", "<br /><br />\n") | footer()], []), file:close(Fd). +%%%----------------------------------------------------------------- +%%% tc_io_format/3 +%%% Tell common_test's IO server (group leader) not to escape +%%% HTML characters. + +-spec tc_io_format(io:device(), io:format(), [term()]) -> 'ok'. + +tc_io_format(Fd, Format0, Args) -> + %% We know that the specially wrapped format string is handled + %% by our IO server, but Dialyzer does not and would tell us + %% that the call to io:format/3 would fail. Therefore, we must + %% fool dialyzer. + + Format = case cloaked_true() of + true -> ["$tc_html",Format0]; + false -> Format0 %Never happens. + end, + io:format(Fd, Format, Args). + +%% Return 'true', but let dialyzer think that a boolean is returned. +cloaked_true() -> + is_process_alive(self()). + %%%----------------------------------------------------------------- %%% Make an index page for the last run diff --git a/lib/compiler/src/beam_bool.erl b/lib/compiler/src/beam_bool.erl index f9a08f8718..359fdb6d3c 100644 --- a/lib/compiler/src/beam_bool.erl +++ b/lib/compiler/src/beam_bool.erl @@ -238,9 +238,9 @@ extend_block(BlAcc0, Fail, [{block,Is0}|OldAcc]) -> end; extend_block(BlAcc, _, OldAcc) -> {BlAcc,OldAcc}. -extend_block_1([{set,[_],_,{bif,_,{f,Fail}}}=I|Is], Fail, Acc) -> +extend_block_1([{set,[{x,_}],_,{bif,_,{f,Fail}}}=I|Is], Fail, Acc) -> extend_block_1(Is, Fail, [I|Acc]); -extend_block_1([{set,[_],As,{bif,Bif,_}}=I|Is]=Is0, Fail, Acc) -> +extend_block_1([{set,[{x,_}],As,{bif,Bif,_}}=I|Is]=Is0, Fail, Acc) -> case safe_bool_op(Bif, length(As)) of false -> {Acc,reverse(Is0)}; true -> extend_block_1(Is, Fail, [I|Acc]) diff --git a/lib/compiler/src/beam_utils.erl b/lib/compiler/src/beam_utils.erl index 37f89dd677..47703b4aa3 100644 --- a/lib/compiler/src/beam_utils.erl +++ b/lib/compiler/src/beam_utils.erl @@ -25,9 +25,8 @@ is_not_used/3,is_not_used_at/3, empty_label_index/0,index_label/3,index_labels/1, code_at/2,bif_to_test/3,is_pure_test/1, - live_opt/1,delete_live_annos/1,combine_heap_needs/2]). - --export([join_even/2,split_even/1]). + live_opt/1,delete_live_annos/1,combine_heap_needs/2, + join_even/2,split_even/1]). -import(lists, [member/2,sort/1,reverse/1,splitwith/2]). @@ -67,8 +66,7 @@ is_killed(R, Is, D) -> St = #live{bl=check_killed_block_fun(),lbl=D,res=gb_trees:empty()}, case check_liveness(R, Is, St) of {killed,_} -> true; - {used,_} -> false; - {unknown,_} -> false + {used,_} -> false end. %% is_killed_at(Reg, Lbl, State) -> true|false @@ -78,8 +76,7 @@ is_killed_at(R, Lbl, D) when is_integer(Lbl) -> St0 = #live{bl=check_killed_block_fun(),lbl=D,res=gb_trees:empty()}, case check_liveness_at(R, Lbl, St0) of {killed,_} -> true; - {used,_} -> false; - {unknown,_} -> false + {used,_} -> false end. %% is_not_used(Register, [Instruction], State) -> true|false @@ -93,8 +90,7 @@ is_not_used(R, Is, D) -> St = #live{bl=fun check_used_block/3,lbl=D,res=gb_trees:empty()}, case check_liveness(R, Is, St) of {killed,_} -> true; - {used,_} -> false; - {unknown,_} -> false + {used,_} -> false end. %% is_not_used(Register, [Instruction], State) -> true|false @@ -108,8 +104,7 @@ is_not_used_at(R, Lbl, D) -> St = #live{bl=fun check_used_block/3,lbl=D,res=gb_trees:empty()}, case check_liveness_at(R, Lbl, St) of {killed,_} -> true; - {used,_} -> false; - {unknown,_} -> false + {used,_} -> false end. %% index_labels(FunctionIs) -> State @@ -137,10 +132,7 @@ index_label(Lbl, Is0, Acc) -> %% Retrieve the code at the given label. code_at(L, Ll) -> - case gb_trees:lookup(L, Ll) of - {value,Code} -> Code; - none -> none - end. + gb_trees:get(L, Ll). %% bif_to_test(Bif, [Op], Fail) -> {test,Test,Fail,[Op]} %% Convert a BIF to a test. Fail if not possible. @@ -164,10 +156,10 @@ bif_to_test('=<', [A,B], Fail) -> {test,is_ge,Fail,[B,A]}; bif_to_test('>', [A,B], Fail) -> {test,is_lt,Fail,[B,A]}; bif_to_test('<', [_,_]=Ops, Fail) -> {test,is_lt,Fail,Ops}; bif_to_test('>=', [_,_]=Ops, Fail) -> {test,is_ge,Fail,Ops}; -bif_to_test('==', [A,[]], Fail) -> {test,is_nil,Fail,[A]}; +bif_to_test('==', [A,nil], Fail) -> {test,is_nil,Fail,[A]}; bif_to_test('==', [_,_]=Ops, Fail) -> {test,is_eq,Fail,Ops}; bif_to_test('/=', [_,_]=Ops, Fail) -> {test,is_ne,Fail,Ops}; -bif_to_test('=:=', [A,[]], Fail) -> {test,is_nil,Fail,[A]}; +bif_to_test('=:=', [A,nil], Fail) -> {test,is_nil,Fail,[A]}; bif_to_test('=:=', [_,_]=Ops, Fail) -> {test,is_eq_exact,Fail,Ops}; bif_to_test('=/=', [_,_]=Ops, Fail) -> {test,is_ne_exact,Fail,Ops}; bif_to_test(is_record, [_,_,_]=Ops, Fail) -> {test,is_record,Fail,Ops}. @@ -235,21 +227,28 @@ combine_heap_needs(Words, {alloc,Alloc}) when is_integer(Words) -> combine_heap_needs(H1, H2) when is_integer(H1), is_integer(H2) -> H1+H2. +%% split_even/1 +%% [1,2,3,4,5,6] -> {[1,3,5],[2,4,6]} + +split_even(Rs) -> split_even(Rs, [], []). + +%% join_even/1 +%% {[1,3,5],[2,4,6]} -> [1,2,3,4,5,6] + +join_even([], []) -> []; +join_even([S|Ss], [D|Ds]) -> [S,D|join_even(Ss, Ds)]. + %%% %%% Local functions. %%% -%% check_liveness(Reg, [Instruction], {State,BlockCheckFun}) -> -%% {killed | used | unknown,UpdateState} -%% Finds out how Reg is used in the instruction sequence. Returns one of: -%% killed - Reg is assigned a new value or killed by an allocation instruction -%% used - Reg is used (or possibly referenced by an allocation instruction) -%% unknown - not possible to determine (perhaps because of an instruction -%% that we don't recognize) +%% check_liveness(Reg, [Instruction], #live{}) -> +%% {killed | used, #live{}} +%% Find out whether Reg is used or killed in instruction sequence. +%% 'killed' means that Reg is assigned a new value or killed by an +%% allocation instruction. 'used' means that Reg is used in some way. -check_liveness(R, [{set,_,_,_}=I|_], St) -> - erlang:error(only_allowed_in_blocks, [R,I,St]); check_liveness(R, [{block,Blk}|Is], #live{bl=BlockCheck}=St0) -> case BlockCheck(R, Blk, St0) of {transparent,St} -> check_liveness(R, Is, St); @@ -461,8 +460,9 @@ check_liveness(R, [{loop_rec,{f,_},{x,0}}|_], St) -> {x,_} -> {killed,St}; _ -> - %% y register. Rarely happens. Be very conversative. - {unknown,St} + %% y register. Rarely happens. Be very conversative and + %% assume it's used. + {used,St} end; check_liveness(R, [{loop_rec_end,{f,Fail}}|_], St) -> check_liveness_at(R, Fail, St); @@ -493,7 +493,8 @@ check_liveness(R, [{put_map,{f,_},_,Src,_D,Live,{list,_}}|_], St0) -> {x,_} -> {killed,St0}; {y,_} -> - {unknown,St0} + %% Conservatively mark it as used. + {used,St0} end; check_liveness(R, [{test_heap,N,Live}|Is], St) -> I = {block,[{set,[],[],{alloc,Live,{nozero,nostack,N,[]}}}]}, @@ -505,12 +506,8 @@ check_liveness(R, [{get_list,S,D1,D2}|Is], St) -> I = {block,[{set,[D1,D2],[S],get_list}]}, check_liveness(R, [I|Is], St); check_liveness(_R, Is, St) when is_list(Is) -> -%% case Is of -%% [I|_] -> -%% io:format("~p ~p\n", [_R,I]); -%% _ -> ok -%% end, - {unknown,St}. + %% Not implemented. Conservatively assume that the register is used. + {used,St}. check_liveness_everywhere(R, [{f,Lbl}|T], St0) -> case check_liveness_at(R, Lbl, St0) of @@ -529,7 +526,7 @@ check_liveness_at(R, Lbl, #live{lbl=Ll,res=ResMemorized}=St0) -> none -> {Res,St} = case gb_trees:lookup(Lbl, Ll) of {value,Is} -> check_liveness(R, Is, St0); - none -> {unknown,St0} + none -> {used,St0} end, {Res,St#live{res=gb_trees:insert(Lbl, Res, St#live.res)}} end. @@ -594,8 +591,10 @@ check_killed_block(_, []) -> transparent. %% killed - Reg is assigned a new value or killed by an allocation instruction %% transparent - Reg is neither used nor killed %% used - Reg is explicitly used by an instruction -%% -%% (Unknown instructions will cause an exception.) +%% +%% '%live' annotations are not allowed. +%% +%% (Unknown instructions will cause an exception.) check_used_block({x,X}=R, [{set,Ds,Ss,{alloc,Live,Op}}|Is], St) -> if @@ -604,11 +603,6 @@ check_used_block({x,X}=R, [{set,Ds,Ss,{alloc,Live,Op}}|Is], St) -> end; check_used_block(R, [{set,Ds,Ss,Op}|Is], St) -> check_used_block_1(R, Ss, Ds, Op, Is, St); -check_used_block(R, [{'%live',Live,_}|Is], St) -> - case R of - {x,X} when X >= Live -> {killed,St}; - _ -> check_used_block(R, Is, St) - end; check_used_block(_, [], St) -> {transparent,St}. check_used_block_1(R, Ss, Ds, Op, Is, St0) -> @@ -639,8 +633,7 @@ is_reg_used_at_1(_, 0, St) -> is_reg_used_at_1(R, Lbl, St0) -> case check_liveness_at(R, Lbl, St0) of {killed,St} -> {false,St}; - {used,St} -> {true,St}; - {unknown,St} -> {true,St} + {used,St} -> {true,St} end. index_labels_1([{label,Lbl}|Is0], Acc) -> @@ -756,11 +749,6 @@ live_opt([{select,_,Src,Fail,List}=I|Is], Regs0, D, Acc) -> Regs1 = x_live([Src], Regs0), Regs = live_join_labels([Fail|List], D, Regs1), live_opt(Is, Regs, D, [I|Acc]); -live_opt([{'try',_,_}=I|Is], Regs, D, Acc) -> - %% If an exeption happens, all x registers will be killed. - %% Therefore, we should only base liveness of the code inside - %% the try. - live_opt(Is, Regs, D, [I|Acc]); live_opt([{try_case,_}=I|Is], _, D, Acc) -> live_opt(Is, live_call(1), D, [I|Acc]); live_opt([{loop_rec,_Fail,_Dst}=I|Is], _, D, Acc) -> @@ -860,14 +848,7 @@ x_live([], Regs) -> Regs. is_live(X, Regs) -> ((Regs bsr X) band 1) =:= 1. -%% split_even/1 -%% [1,2,3,4,5,6] -> {[1,3,5],[2,4,6]} -split_even(Rs) -> split_even(Rs,[],[]). -split_even([],Ss,Ds) -> {reverse(Ss),reverse(Ds)}; -split_even([S,D|Rs],Ss,Ds) -> - split_even(Rs,[S|Ss],[D|Ds]). - -%% join_even/1 -%% {[1,3,5],[2,4,6]} -> [1,2,3,4,5,6] -join_even([],[]) -> []; -join_even([S|Ss],[D|Ds]) -> [S,D|join_even(Ss,Ds)]. +split_even([], Ss, Ds) -> + {reverse(Ss),reverse(Ds)}; +split_even([S,D|Rs], Ss, Ds) -> + split_even(Rs, [S|Ss], [D|Ds]). diff --git a/lib/compiler/src/core_pp.erl b/lib/compiler/src/core_pp.erl index f34a5c034f..67209d06be 100644 --- a/lib/compiler/src/core_pp.erl +++ b/lib/compiler/src/core_pp.erl @@ -33,8 +33,7 @@ %% Prettyprint-formats (naively) an abstract Core Erlang syntax %% tree. --record(ctxt, {class = term :: 'clause' | 'def' | 'expr' | 'term', - indent = 0 :: integer(), +-record(ctxt, {indent = 0 :: integer(), item_indent = 2 :: integer(), body_indent = 4 :: integer(), line = 0 :: integer(), @@ -132,14 +131,11 @@ format_1(#c_literal{anno=A,val=Bitstring}, Ctxt) when is_bitstring(Bitstring) -> format_1(#c_binary{anno=A,segments=Segs}, Ctxt); format_1(#c_literal{anno=A,val=M},Ctxt) when is_map(M) -> Pairs = maps:to_list(M), - Op = case Ctxt of - #ctxt{ class = clause } -> exact; - _ -> assoc - end, - Cpairs = [#c_map_pair{op=#c_literal{val=Op}, + Op = #c_literal{val=assoc}, + Cpairs = [#c_map_pair{op=Op, key=#c_literal{val=K}, val=#c_literal{val=V}} || {K,V} <- Pairs], - format_1(#c_map{anno=A,arg=#c_literal{val=#{}},es=Cpairs},Ctxt); + format_1(#c_map{anno=A,arg=#c_literal{val=#{}},es=Cpairs},Ctxt); format_1(#c_var{name={I,A}}, _) -> [core_atom(I),$/,integer_to_list(A)]; format_1(#c_var{name=V}, _) -> @@ -340,35 +336,30 @@ format_1(#c_module{name=N,exports=Es,attrs=As,defs=Ds}, Ctxt) -> [Mod," [", format_vseq(Es, "", ",", - add_indent(set_class(Ctxt, term), width(Mod, Ctxt)+2), + add_indent(Ctxt, width(Mod, Ctxt)+2), fun format/2), "]", nl_indent(Ctxt), " attributes [", format_vseq(As, "", ",", - add_indent(set_class(Ctxt, def), 16), + add_indent(Ctxt, 16), fun format_def/2), "]", nl_indent(Ctxt), format_funcs(Ds, Ctxt), nl_indent(Ctxt) | "end" - ]; -format_1(Type, _) -> - ["** Unsupported type: ", - io_lib:write(Type) - | " **" ]. format_funcs(Fs, Ctxt) -> format_vseq(Fs, "", "", - set_class(Ctxt, def), + Ctxt, fun format_def/2). format_def({N,V}, Ctxt0) -> - Ctxt1 = add_indent(set_class(Ctxt0, expr), Ctxt0#ctxt.body_indent), + Ctxt1 = add_indent(Ctxt0, Ctxt0#ctxt.body_indent), [format(N, Ctxt0), " =", nl_indent(Ctxt1) @@ -392,8 +383,7 @@ do_format_bitstr(#c_bitstr{val=V,size=S,unit=U,type=T,flags=Fs}, Ctxt0) -> ["#<", Val, ">(", format_hseq(Vs,",", Ctxt2, fun format/2), $)]. format_clauses(Cs, Ctxt) -> - format_vseq(Cs, "", "", set_class(Ctxt, clause), - fun format_clause/2). + format_vseq(Cs, "", "", Ctxt, fun format_clause/2). format_clause(Node, Ctxt) -> maybe_anno(Node, fun format_clause_1/2, Ctxt). @@ -405,15 +395,13 @@ format_clause_1(#c_clause{pats=Ps,guard=G,body=B}, Ctxt) -> case is_trivial_guard(G) of true -> [" when ", - format_guard(G, add_indent(set_class(Ctxt, expr), - width(Ptxt, Ctxt) + 6))]; + format_guard(G, add_indent(Ctxt, width(Ptxt, Ctxt) + 6))]; false -> [nl_indent(Ctxt2), "when ", - format_guard(G, add_indent(set_class(Ctxt2, expr), 2))] + format_guard(G, add_indent(Ctxt2, 2))] end++ " ->", - nl_indent(Ctxt2) - | format(B, set_class(Ctxt2, expr)) + nl_indent(Ctxt2) | format(B, Ctxt2) ]. is_trivial_guard(#c_literal{val=Val}) when is_atom(Val) -> true; @@ -467,7 +455,7 @@ format_list_tail(Tail, Ctxt) -> format_map_pair(Op, K, V, Ctxt0) -> Ctxt1 = add_indent(Ctxt0, 1), - Txt = format(K, set_class(Ctxt1, expr)), + Txt = format(K, Ctxt1), Ctxt2 = add_indent(Ctxt0, width(Txt, Ctxt1)), [Txt,Op,format(V, Ctxt2)]. @@ -490,6 +478,7 @@ spaces(5) -> " "; spaces(6) -> " "; spaces(7) -> " ". +%% Undo indentation done by nl_indent/1. unindent(T, Ctxt) -> unindent(T, Ctxt#ctxt.indent, []). @@ -505,18 +494,11 @@ unindent([$\t|T], N, C) -> unindent([spaces(Tab - N)|T], 0, C) end; unindent([L|T], N, C) when is_list(L) -> - unindent(L, N, [T|C]); -unindent([H|T], _, C) -> - [H|[T|C]]; -unindent([], N, [H|T]) -> - unindent(H, N, T); -unindent([], _, []) -> []. + unindent(L, N, [T|C]). width(Txt, Ctxt) -> - try width(Txt, 0, Ctxt, []) - catch error:_ -> exit({bad_text,Txt}) - end. + width(Txt, 0, Ctxt, []). width([$\t|T], A, Ctxt, C) -> width(T, A + ?TAB_WIDTH, Ctxt, C); @@ -533,14 +515,9 @@ width([], A, _, []) -> A. add_indent(Ctxt, Dx) -> Ctxt#ctxt{indent = Ctxt#ctxt.indent + Dx}. -set_class(Ctxt, Class) -> - Ctxt#ctxt{class = Class}. - core_atom(A) -> io_lib:write_string(atom_to_list(A), $'). -is_simple_term(#c_values{es=Es}) -> - length(Es) < 3 andalso lists:all(fun is_simple_term/1, Es); is_simple_term(#c_tuple{es=Es}) -> length(Es) < 4 andalso lists:all(fun is_simple_term/1, Es); is_simple_term(#c_var{}) -> true; diff --git a/lib/compiler/src/sys_core_fold.erl b/lib/compiler/src/sys_core_fold.erl index b5b8d8a8ec..dbc27db377 100644 --- a/lib/compiler/src/sys_core_fold.erl +++ b/lib/compiler/src/sys_core_fold.erl @@ -374,10 +374,21 @@ expr(#c_receive{clauses=Cs0,timeout=T0,action=A0}=Recv, Ctxt, Sub) -> T1 = expr(T0, value, Sub), A1 = body(A0, Ctxt, Sub), Recv#c_receive{clauses=Cs1,timeout=T1,action=A1}; -expr(#c_apply{op=Op0,args=As0}=App, _, Sub) -> +expr(#c_apply{anno=Anno,op=Op0,args=As0}=App, _, Sub) -> Op1 = expr(Op0, value, Sub), As1 = expr_list(As0, value, Sub), - App#c_apply{op=Op1,args=As1}; + case Op1 of + #c_var{} -> + App#c_apply{op=Op1,args=As1}; + _ -> + add_warning(App, invalid_call), + Err = #c_call{anno=Anno, + module=#c_literal{val=erlang}, + name=#c_literal{val=error}, + args=[#c_tuple{es=[#c_literal{val='badfun'}, + Op1]}]}, + make_effect_seq(As1++[Err], Sub) + end; expr(#c_call{module=M0,name=N0}=Call0, Ctxt, Sub) -> M1 = expr(M0, value, Sub), N1 = expr(N0, value, Sub), @@ -3395,6 +3406,8 @@ format_error({no_effect,{erlang,F,A}}) -> format_error(result_ignored) -> "the result of the expression is ignored " "(suppress the warning by assigning the expression to the _ variable)"; +format_error(invalid_call) -> + "invalid function call"; format_error(useless_building) -> "a term is constructed, but never used"; format_error(bin_opt_alias) -> diff --git a/lib/compiler/test/Makefile b/lib/compiler/test/Makefile index 85118502e3..203a50db55 100644 --- a/lib/compiler/test/Makefile +++ b/lib/compiler/test/Makefile @@ -9,6 +9,7 @@ MODULES= \ andor_SUITE \ apply_SUITE \ beam_block_SUITE \ + beam_bool_SUITE \ beam_validator_SUITE \ beam_disasm_SUITE \ beam_except_SUITE \ @@ -46,6 +47,7 @@ NO_OPT= \ andor \ apply \ beam_block \ + beam_bool \ beam_except \ beam_reorder \ beam_type \ @@ -70,6 +72,7 @@ INLINE= \ andor \ apply \ beam_block \ + beam_bool \ beam_utils \ bs_bincomp \ bs_bit_binaries \ diff --git a/lib/compiler/test/andor_SUITE.erl b/lib/compiler/test/andor_SUITE.erl index f6e7b09010..05c087104d 100644 --- a/lib/compiler/test/andor_SUITE.erl +++ b/lib/compiler/test/andor_SUITE.erl @@ -22,8 +22,7 @@ -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, init_per_group/2,end_per_group/2, t_case/1,t_and_or/1,t_andalso/1,t_orelse/1,inside/1,overlap/1, - combined/1,in_case/1,before_and_inside_if/1, - slow_compilation/1]). + combined/1,in_case/1,slow_compilation/1]). -include_lib("common_test/include/ct.hrl"). @@ -36,7 +35,7 @@ all() -> groups() -> [{p,[parallel], [t_case,t_and_or,t_andalso,t_orelse,inside,overlap, - combined,in_case,before_and_inside_if]}]. + combined,in_case,slow_compilation]}]. init_per_suite(Config) -> Config. @@ -450,64 +449,6 @@ in_case_1_guard(LenUp, LenDw, LenN, Rotation, Count) -> false -> loop end. -before_and_inside_if(Config) when is_list(Config) -> - no = before_and_inside_if([a], [b], delete), - no = before_and_inside_if([a], [b], x), - no = before_and_inside_if([a], [], delete), - no = before_and_inside_if([a], [], x), - no = before_and_inside_if([], [], delete), - yes = before_and_inside_if([], [], x), - yes = before_and_inside_if([], [b], delete), - yes = before_and_inside_if([], [b], x), - - {ch1,ch2} = before_and_inside_if_2([a], [b], blah), - {ch1,ch2} = before_and_inside_if_2([a], [b], xx), - {ch1,ch2} = before_and_inside_if_2([a], [], blah), - {ch1,ch2} = before_and_inside_if_2([a], [], xx), - {no,no} = before_and_inside_if_2([], [b], blah), - {no,no} = before_and_inside_if_2([], [b], xx), - {ch1,no} = before_and_inside_if_2([], [], blah), - {no,ch2} = before_and_inside_if_2([], [], xx), - ok. - -%% Thanks to Simon Cornish and Kostis Sagonas. -%% Used to crash beam_bool. -before_and_inside_if(XDo1, XDo2, Do3) -> - Do1 = (XDo1 =/= []), - Do2 = (XDo2 =/= []), - if - %% This expression occurs in a try/catch (protected) - %% block, which cannot refer to variables outside of - %% the block that are boolean expressions. - Do1 =:= true; - Do1 =:= false, Do2 =:= false, Do3 =:= delete -> - no; - true -> - yes - end. - -%% Thanks to Simon Cornish. -%% Used to generate code that would not set {y,0} on -%% all paths before its use (and therefore fail -%% validation by the beam_validator). -before_and_inside_if_2(XDo1, XDo2, Do3) -> - Do1 = (XDo1 =/= []), - Do2 = (XDo2 =/= []), - CH1 = if Do1 == true; - Do1 == false,Do2==false,Do3 == blah -> - ch1; - true -> - no - end, - CH2 = if Do1 == true; - Do1 == false,Do2==false,Do3 == xx -> - ch2; - true -> - no - end, - {CH1,CH2}. - - -record(state, {stack = []}). slow_compilation(_) -> diff --git a/lib/compiler/test/beam_block_SUITE.erl b/lib/compiler/test/beam_block_SUITE.erl index 81f8d10687..d343e26737 100644 --- a/lib/compiler/test/beam_block_SUITE.erl +++ b/lib/compiler/test/beam_block_SUITE.erl @@ -35,7 +35,8 @@ all() -> groups() -> [{p,[parallel], - [get_map_elements + [get_map_elements, + otp_7345 ]}]. init_per_suite(Config) -> diff --git a/lib/compiler/test/beam_bool_SUITE.erl b/lib/compiler/test/beam_bool_SUITE.erl new file mode 100644 index 0000000000..84d634e5ca --- /dev/null +++ b/lib/compiler/test/beam_bool_SUITE.erl @@ -0,0 +1,160 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2015-2016. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(beam_bool_SUITE). + +-export([all/0,suite/0,groups/0,init_per_suite/1,end_per_suite/1, + init_per_group/2,end_per_group/2, + before_and_inside_if/1, + scotland/1,y_registers/1]). + +suite() -> + [{ct_hooks,[ts_install_cth]}]. + +all() -> + test_lib:recompile(?MODULE), + [{group,p}]. + +groups() -> + [{p,[parallel], + [before_and_inside_if, + scotland, + y_registers + ]}]. + +init_per_suite(Config) -> + Config. + +end_per_suite(_Config) -> + ok. + +init_per_group(_GroupName, Config) -> + Config. + +end_per_group(_GroupName, Config) -> + Config. + +before_and_inside_if(_Config) -> + no = before_and_inside_if([a], [b], delete), + no = before_and_inside_if([a], [b], x), + no = before_and_inside_if([a], [], delete), + no = before_and_inside_if([a], [], x), + no = before_and_inside_if([], [], delete), + yes = before_and_inside_if([], [], x), + yes = before_and_inside_if([], [b], delete), + yes = before_and_inside_if([], [b], x), + + {ch1,ch2} = before_and_inside_if_2([a], [b], blah), + {ch1,ch2} = before_and_inside_if_2([a], [b], xx), + {ch1,ch2} = before_and_inside_if_2([a], [], blah), + {ch1,ch2} = before_and_inside_if_2([a], [], xx), + {no,no} = before_and_inside_if_2([], [b], blah), + {no,no} = before_and_inside_if_2([], [b], xx), + {ch1,no} = before_and_inside_if_2([], [], blah), + {no,ch2} = before_and_inside_if_2([], [], xx), + ok. + +%% Thanks to Simon Cornish and Kostis Sagonas. +%% Used to crash beam_bool. +before_and_inside_if(XDo1, XDo2, Do3) -> + Do1 = (XDo1 =/= []), + Do2 = (XDo2 =/= []), + if + %% This expression occurs in a try/catch (protected) + %% block, which cannot refer to variables outside of + %% the block that are boolean expressions. + Do1 =:= true; + Do1 =:= false, Do2 =:= false, Do3 =:= delete -> + no; + true -> + yes + end. + +%% Thanks to Simon Cornish. +%% Used to generate code that would not set {y,0} on +%% all paths before its use (and therefore fail +%% validation by the beam_validator). +before_and_inside_if_2(XDo1, XDo2, Do3) -> + Do1 = (XDo1 =/= []), + Do2 = (XDo2 =/= []), + CH1 = if Do1 == true; + Do1 == false,Do2==false,Do3 == blah -> + ch1; + true -> + no + end, + CH2 = if Do1 == true; + Do1 == false,Do2==false,Do3 == xx -> + ch2; + true -> + no + end, + {CH1,CH2}. + + +%% beam_bool would remove the initialization of {y,0}. +%% (Thanks to Thomas Arts and QuickCheck.) + +scotland(_Config) -> + million = do_scotland(placed), + {'EXIT',{{badmatch,placed},_}} = (catch do_scotland(false)), + {'EXIT',{{badmatch,placed},_}} = (catch do_scotland(true)), + {'EXIT',{{badmatch,placed},_}} = (catch do_scotland(echo)), + ok. + +do_scotland(Echo) -> + found(case Echo of + Echo when true; Echo, Echo, Echo -> + Echo; + echo -> + [] + end, + Echo = placed). + +found(_, _) -> million. + + +%% ERL-143: beam_bool could not handle Y registers as a destination. +y_registers(_Config) -> + {'EXIT',{badarith,[_|_]}} = (catch baker(valentine)), + {'EXIT',{badarith,[_|_]}} = (catch baker(clementine)), + + {not_ok,true} = potter([]), + {ok,false} = potter([{encoding,any}]), + + ok. + +%% Thanks to Quickcheck. +baker(Baker) -> + (valentine == Baker) + + case Baker of + Baker when Baker; Baker -> + Baker; + Baker -> + [] + end. + +%% Thanks to Jose Valim. +potter(Modes) -> + Raw = lists:keyfind(encoding, 1, Modes) == false, + Final = case Raw of + X when X == false; X == nil -> ok; + _ -> not_ok + end, + {Final,Raw}. diff --git a/lib/compiler/test/compile_SUITE.erl b/lib/compiler/test/compile_SUITE.erl index e634f0fcc2..a15efc2a00 100644 --- a/lib/compiler/test/compile_SUITE.erl +++ b/lib/compiler/test/compile_SUITE.erl @@ -693,8 +693,7 @@ core(Config) when is_list(Config) -> Outdir = filename:join(PrivDir, "core"), ok = file:make_dir(Outdir), - Wc = filename:join(filename:dirname(code:which(?MODULE)), "*.beam"), - TestBeams = filelib:wildcard(Wc), + TestBeams = get_unique_beam_files(), Abstr = [begin {ok,{Mod,[{abstract_code, {raw_abstract_v1,Abstr}}]}} = beam_lib:chunks(Beam, [abstract_code]), @@ -755,8 +754,7 @@ core_roundtrip(Config) -> Outdir = filename:join(PrivDir, atom_to_list(?FUNCTION_NAME)), ok = file:make_dir(Outdir), - Wc = filename:join(filename:dirname(code:which(?MODULE)), "*.beam"), - TestBeams = filelib:wildcard(Wc), + TestBeams = get_unique_beam_files(), test_lib:p_run(fun(F) -> do_core_roundtrip(F, Outdir) end, TestBeams). do_core_roundtrip(Beam, Outdir) -> @@ -781,8 +779,13 @@ do_core_roundtrip_1(Mod, Abstr, Outdir) -> %% Primarily, test that annotations are accepted for all %% constructs. Secondarily, smoke test cerl_trees:label/1. - {Core,_} = cerl_trees:label(Core0), - do_core_roundtrip_2(Mod, Core, Outdir). + {Core1,_} = cerl_trees:label(Core0), + do_core_roundtrip_2(Mod, Core1, Outdir), + + %% Run the inliner to force generation of variables + %% with numeric names. + {ok,Mod,Core2} = compile:forms(Abstr, [inline,to_core]), + do_core_roundtrip_2(Mod, Core2, Outdir). do_core_roundtrip_2(M, Core0, Outdir) -> CoreFile = filename:join(Outdir, atom_to_list(M)++".core"), @@ -870,8 +873,7 @@ asm(Config) when is_list(Config) -> Outdir = filename:join(PrivDir, "asm"), ok = file:make_dir(Outdir), - Wc = filename:join(filename:dirname(code:which(?MODULE)), "*.beam"), - TestBeams = filelib:wildcard(Wc), + TestBeams = get_unique_beam_files(), Res = test_lib:p_run(fun(F) -> do_asm(F, Outdir) end, TestBeams), Res. @@ -947,8 +949,7 @@ dialyzer(Config) -> %% Test that warnings contain filenames and line numbers. warnings(_Config) -> - TestDir = filename:dirname(code:which(?MODULE)), - Files = filelib:wildcard(filename:join(TestDir, "*.erl")), + Files = get_unique_files(".erl"), test_lib:p_run(fun do_warnings/1, Files). do_warnings(F) -> @@ -1102,3 +1103,14 @@ compile_and_verify(Name, Target, Opts) -> beam_lib:chunks(Target, [compile_info]), {options,BeamOpts} = lists:keyfind(options, 1, CInfo), Opts = BeamOpts. + +get_unique_beam_files() -> + get_unique_files(".beam"). + +get_unique_files(Ext) -> + Wc = filename:join(filename:dirname(code:which(?MODULE)), "*"++Ext), + [F || F <- filelib:wildcard(Wc), not is_cloned(F, Ext)]. + +is_cloned(File, Ext) -> + Mod = list_to_atom(filename:basename(File, Ext)), + test_lib:is_cloned_mod(Mod). diff --git a/lib/compiler/test/fun_SUITE.erl b/lib/compiler/test/fun_SUITE.erl index 17ff8601d9..16474adf5b 100644 --- a/lib/compiler/test/fun_SUITE.erl +++ b/lib/compiler/test/fun_SUITE.erl @@ -22,7 +22,7 @@ -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, init_per_group/2,end_per_group/2, test1/1,overwritten_fun/1,otp_7202/1,bif_fun/1, - external/1,eep37/1,eep37_dup/1,badarity/1]). + external/1,eep37/1,eep37_dup/1,badarity/1,badfun/1]). %% Internal exports. -export([call_me/1,dup1/0,dup2/0]). @@ -33,10 +33,12 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> test_lib:recompile(?MODULE), - [test1,overwritten_fun,otp_7202,bif_fun,external,eep37,eep37_dup,badarity]. + [{group,p}]. -groups() -> - []. +groups() -> + [{p,[parallel], + [test1,overwritten_fun,otp_7202,bif_fun,external,eep37, + eep37_dup,badarity,badfun]}]. init_per_suite(Config) -> Config. @@ -221,5 +223,25 @@ badarity(Config) when is_list(Config) -> {'EXIT',{{badarity,{_,[]}},_}} = (catch (fun badarity/1)()), ok. +badfun(_Config) -> + X = not_a_fun, + expect_badfun(42, catch 42()), + expect_badfun(42.0, catch 42.0(1)), + expect_badfun(X, catch X()), + expect_badfun(X, catch X(1)), + Len = length(atom_to_list(X)), + expect_badfun(Len, catch begin length(atom_to_list(X)) end(1)), + + expect_badfun(42, catch 42(put(?FUNCTION_NAME, yes))), + yes = erase(?FUNCTION_NAME), + + expect_badfun(X, catch X(put(?FUNCTION_NAME, of_course))), + of_course = erase(?FUNCTION_NAME), + + ok. + +expect_badfun(Term, Exit) -> + {'EXIT',{{badfun,Term},_}} = Exit. + id(I) -> I. diff --git a/lib/compiler/test/guard_SUITE.erl b/lib/compiler/test/guard_SUITE.erl index 13d274d98a..83298e546e 100644 --- a/lib/compiler/test/guard_SUITE.erl +++ b/lib/compiler/test/guard_SUITE.erl @@ -34,7 +34,7 @@ tricky/1,rel_ops/1,rel_op_combinations/1,literal_type_tests/1, basic_andalso_orelse/1,traverse_dcd/1, check_qlc_hrl/1,andalso_semi/1,t_tuple_size/1,binary_part/1, - bad_constants/1,bad_guards/1,scotland/1, + bad_constants/1,bad_guards/1, guard_in_catch/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. @@ -54,7 +54,7 @@ groups() -> rel_ops,rel_op_combinations, literal_type_tests,basic_andalso_orelse,traverse_dcd, check_qlc_hrl,andalso_semi,t_tuple_size,binary_part, - bad_constants,bad_guards,scotland,guard_in_catch]}]. + bad_constants,bad_guards,guard_in_catch]}]. init_per_suite(Config) -> Config. @@ -1852,27 +1852,6 @@ bad_guards_2(M, [_]) when M#{a := 0, b => 0}, map_size(M) -> bad_guards_3(M, [_]) when is_map(M) andalso M#{a := 0, b => 0}, length(M) -> ok. -%% beam_bool would remove the initialization of {y,0}. -%% (Thanks to Thomas Arts and QuickCheck.) - -scotland(_Config) -> - million = do_scotland(placed), - {'EXIT',{{badmatch,placed},_}} = (catch do_scotland(false)), - {'EXIT',{{badmatch,placed},_}} = (catch do_scotland(true)), - {'EXIT',{{badmatch,placed},_}} = (catch do_scotland(echo)), - ok. - -do_scotland(Echo) -> - found(case Echo of - Echo when true; Echo, Echo, Echo -> - Echo; - echo -> - [] - end, - Echo = placed). - -found(_, _) -> million. - %% Building maps in a guard in a 'catch' would crash v3_codegen. guard_in_catch(_Config) -> diff --git a/lib/compiler/test/test_lib.erl b/lib/compiler/test/test_lib.erl index d141d86021..d5b79e2357 100644 --- a/lib/compiler/test/test_lib.erl +++ b/lib/compiler/test/test_lib.erl @@ -22,7 +22,7 @@ -include_lib("common_test/include/ct.hrl"). -compile({no_auto_import,[binary_part/2]}). -export([id/1,recompile/1,parallel/0,uniq/0,opt_opts/1,get_data_dir/1, - smoke_disasm/1,p_run/2,binary_part/2]). + is_cloned_mod/1,smoke_disasm/1,p_run/2,binary_part/2]). id(I) -> I. @@ -91,6 +91,17 @@ get_data_dir(Config) -> Data = re:replace(Data1, "_post_opt_SUITE", "_SUITE", Opts), re:replace(Data, "_inline_SUITE", "_SUITE", Opts). +is_cloned_mod(Mod) -> + is_cloned_mod_1(atom_to_list(Mod)). + +%% Test whether Mod is a cloned module. + +is_cloned_mod_1("no_opt_SUITE") -> true; +is_cloned_mod_1("post_opt_SUITE") -> true; +is_cloned_mod_1("inline_SUITE") -> true; +is_cloned_mod_1([_|T]) -> is_cloned_mod_1(T); +is_cloned_mod_1([]) -> false. + %% p_run(fun(Data) -> ok|error, List) -> ok %% Will fail the test case if there were any errors. @@ -106,8 +117,9 @@ p_run(Test, List) -> %% slightly faster than using 3. Using more than %% 4 would not buy us much and could actually be %% slower. - max(S, 4) + min(S, 4) end, + io:format("p_run: ~p parallel processes\n", [N]), p_run_loop(Test, List, N, [], 0, 0). p_run_loop(_, [], _, [], Errors, Ws) -> diff --git a/lib/compiler/test/trycatch_SUITE.erl b/lib/compiler/test/trycatch_SUITE.erl index f7ad78cb8d..a591d6cc93 100644 --- a/lib/compiler/test/trycatch_SUITE.erl +++ b/lib/compiler/test/trycatch_SUITE.erl @@ -1034,6 +1034,9 @@ grab_bag(_Config) -> end end, + %% Unnecessary catch. + 22 = (catch 22), + ok. diff --git a/lib/dialyzer/src/dialyzer_contracts.erl b/lib/dialyzer/src/dialyzer_contracts.erl index 116260778c..d1ffa07706 100644 --- a/lib/dialyzer/src/dialyzer_contracts.erl +++ b/lib/dialyzer/src/dialyzer_contracts.erl @@ -479,7 +479,8 @@ initialize_constraints([], _MFA, _RecDict, _ExpTypes, _AllRecords, Acc) -> initialize_constraints([Constr|Rest], MFA, RecDict, ExpTypes, AllRecords, Acc) -> case Constr of {type, _, constraint, [{atom, _, is_subtype}, [Type1, Type2]]} -> - T1 = final_form(Type1, ExpTypes, MFA, AllRecords, maps:new()), + VarTable = erl_types:var_table__new(), + T1 = final_form(Type1, ExpTypes, MFA, AllRecords, VarTable), Entry = {T1, Type2}, initialize_constraints(Rest, MFA, RecDict, ExpTypes, AllRecords, [Entry|Acc]); {type, _, constraint, [{atom,_,Name}, List]} -> @@ -489,8 +490,9 @@ initialize_constraints([Constr|Rest], MFA, RecDict, ExpTypes, AllRecords, Acc) - end. constraints_fixpoint(Constrs, MFA, RecDict, ExpTypes, AllRecords) -> + VarTable = erl_types:var_table__new(), VarDict = - constraints_to_dict(Constrs, MFA, RecDict, ExpTypes, AllRecords, maps:new()), + constraints_to_dict(Constrs, MFA, RecDict, ExpTypes, AllRecords, VarTable), constraints_fixpoint(VarDict, MFA, Constrs, RecDict, ExpTypes, AllRecords). constraints_fixpoint(OldVarDict, MFA, Constrs, RecDict, ExpTypes, AllRecords) -> @@ -498,11 +500,11 @@ constraints_fixpoint(OldVarDict, MFA, Constrs, RecDict, ExpTypes, AllRecords) -> constraints_to_dict(Constrs, MFA, RecDict, ExpTypes, AllRecords, OldVarDict), case NewVarDict of OldVarDict -> - DictFold = + Fun = fun(Key, Value, Acc) -> [{subtype, erl_types:t_var(Key), Value}|Acc] end, - FinalConstrs = maps:fold(DictFold, [], NewVarDict), + FinalConstrs = maps:fold(Fun, [], NewVarDict), {FinalConstrs, NewVarDict}; _Other -> constraints_fixpoint(NewVarDict, MFA, Constrs, RecDict, ExpTypes, AllRecords) @@ -512,12 +514,12 @@ final_form(Form, ExpTypes, MFA, AllRecords, VarDict) -> from_form_with_check(Form, ExpTypes, MFA, AllRecords, VarDict). from_form_with_check(Form, ExpTypes, MFA, AllRecords) -> - from_form_with_check(Form, ExpTypes, MFA, AllRecords, maps:new()). + VarTable = erl_types:var_table__new(), + from_form_with_check(Form, ExpTypes, MFA, AllRecords, VarTable). from_form_with_check(Form, ExpTypes, MFA, AllRecords, VarDict) -> Site = {spec, MFA}, - erl_types:t_check_record_fields(Form, ExpTypes, Site, AllRecords, - VarDict), + erl_types:t_check_record_fields(Form, ExpTypes, Site, AllRecords, VarDict), erl_types:t_from_form(Form, ExpTypes, Site, AllRecords, VarDict). constraints_to_dict(Constrs, MFA, RecDict, ExpTypes, AllRecords, VarDict) -> diff --git a/lib/dialyzer/src/dialyzer_dep.erl b/lib/dialyzer/src/dialyzer_dep.erl index 6678037bc0..273c05c54c 100644 --- a/lib/dialyzer/src/dialyzer_dep.erl +++ b/lib/dialyzer/src/dialyzer_dep.erl @@ -59,8 +59,14 @@ %% separately. %% --spec analyze(cerl:c_module()) -> - {dict:dict(), ordsets:ordset('external' | label()), dict:dict(), dict:dict()}. +-type dep_ordset() :: ordsets:ordset(label() | 'external'). + +-type deps() :: dict:dict(label() | 'external' | 'top', dep_ordset()). +-type esc() :: dep_ordset(). +-type calls() :: dict:dict(label(), ordsets:ordset(label())). +-type letrecs() :: dict:dict(label(), label()). + +-spec analyze(cerl:c_module()) -> {deps(), esc(), calls(), letrecs()}. analyze(Tree) -> %% io:format("Handling ~w\n", [cerl:atom_val(cerl:module_name(Tree))]), @@ -79,22 +85,26 @@ traverse(Tree, Out, State, CurrentFun) -> apply -> Op = cerl:apply_op(Tree), Args = cerl:apply_args(Tree), - %% Op is always a variable and should not be marked as escaping - %% based on its use. case var =:= cerl:type(Op) of - false -> erlang:error({apply_op_not_a_variable, cerl:type(Op)}); - true -> ok - end, - OpFuns = case map__lookup(cerl_trees:get_label(Op), Out) of - none -> output(none); - {value, OF} -> OF - end, - {ArgFuns, State2} = traverse_list(Args, Out, State, CurrentFun), - State3 = state__add_esc(merge_outs(ArgFuns), State2), - State4 = state__add_deps(CurrentFun, OpFuns, State3), - State5 = state__store_callsite(cerl_trees:get_label(Tree), - OpFuns, length(Args), State4), - {output(set__singleton(external)), State5}; + false -> + %% We have discovered an error here, but we ignore it and let + %% later passes handle it; we do not modify the dependencies. + %% erlang:error({apply_op_not_a_variable, cerl:type(Op)}); + {output(none), State}; + true -> + %% Op is a variable and should not be marked as escaping + %% based on its use. + OpFuns = case map__lookup(cerl_trees:get_label(Op), Out) of + none -> output(none); + {value, OF} -> OF + end, + {ArgFuns, State2} = traverse_list(Args, Out, State, CurrentFun), + State3 = state__add_esc(merge_outs(ArgFuns), State2), + State4 = state__add_deps(CurrentFun, OpFuns, State3), + State5 = state__store_callsite(cerl_trees:get_label(Tree), + OpFuns, length(Args), State4), + {output(set__singleton(external)), State5} + end; binary -> {output(none), State}; 'case' -> @@ -481,11 +491,11 @@ all_vars(Tree, AccIn) -> -type local_set() :: 'none' | #set{}. --record(state, {deps :: dict:dict(), +-record(state, {deps :: deps(), esc :: local_set(), - call :: dict:dict(), - arities :: dict:dict(), - letrecs :: dict:dict()}). + calls :: calls(), + arities :: dict:dict(label() | 'top', arity()), + letrecs :: letrecs()}). state__new(Tree) -> Exports = set__from_list([X || X <- cerl:module_exports(Tree)]), @@ -503,7 +513,7 @@ state__new(Tree) -> %% init the escaping function labels to exported + called from on_load InitEsc = set__from_list(OnLoadLs ++ ExpLs), Arities = cerl_trees:fold(fun find_arities/2, dict:new(), Tree), - #state{deps = map__new(), esc = InitEsc, call = map__new(), + #state{deps = map__new(), esc = InitEsc, calls = map__new(), arities = Arities, letrecs = map__new()}. find_arities(Tree, AccMap) -> @@ -518,7 +528,7 @@ find_arities(Tree, AccMap) -> state__add_deps(_From, #output{content = none}, State) -> State; -state__add_deps(From, #output{type = single, content=To}, +state__add_deps(From, #output{type = single, content = To}, #state{deps = Map} = State) -> %% io:format("Adding deps from ~w to ~w\n", [From, set__to_ordsets(To)]), State#state{deps = map__add(From, To, Map)}. @@ -544,16 +554,16 @@ state__esc(#state{esc = Esc}) -> state__store_callsite(_From, #output{content = none}, _CallArity, State) -> State; state__store_callsite(From, To, CallArity, - #state{call = Calls, arities = Arities} = State) -> + #state{calls = Calls, arities = Arities} = State) -> Filter = fun(external) -> true; (Fun) -> CallArity =:= dict:fetch(Fun, Arities) end, case filter_outs(To, Filter) of #output{content = none} -> State; - To1 -> State#state{call = map__store(From, To1, Calls)} + To1 -> State#state{calls = map__store(From, To1, Calls)} end. -state__calls(#state{call = Calls}) -> +state__calls(#state{calls = Calls}) -> Calls. %%------------------------------------------------------------ diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index 77ea9d0413..077fe01e85 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 2.10 +DIALYZER_VSN = 3.0 diff --git a/lib/diameter/doc/src/diameter.xml b/lib/diameter/doc/src/diameter.xml index 66daf6bb0b..d68a78ed6d 100644 --- a/lib/diameter/doc/src/diameter.xml +++ b/lib/diameter/doc/src/diameter.xml @@ -1764,11 +1764,13 @@ An example return value with for a client service with Origin-Host {send_pend,0}]}]}, {statistics,[{{{0,258,0},recv},3}, {{{0,258,1},send},3}, + {{{0,258,0},recv,{'Result-Code',2001}},3}, {{{0,257,0},recv},1}, {{{0,257,1},send},1}, - {{{0,258,0},recv,{'Result-Code',2001}},3}, + {{{0,257,0},recv,{'Result-Code',2001}},1}, {{{0,280,1},recv},2}, - {{{0,280,0},send},2}]}]] + {{{0,280,0},send},2}, + {{{0,280,0},send,{'Result-Code',2001}},2}]}]] </pre> <p> @@ -1844,13 +1846,16 @@ connection might look as follows.</p> [{watchdog,{<0.72.0>,{1346,171491,998404},initial}}]]}, {statistics,[{{{0,280,0},recv},7}, {{{0,280,1},send},7}, - {{{0,258,0},send,{'Result-Code',2001}},3}, + {{{0,280,0},recv,{'Result-Code',2001}},7}, {{{0,258,1},recv},3}, {{{0,258,0},send},3}, + {{{0,258,0},send,{'Result-Code',2001}},3}, {{{0,280,1},recv},5}, {{{0,280,0},send},5}, + {{{0,280,0},send,{'Result-Code',2001}},5}, {{{0,257,1},recv},1}, - {{{0,257,0},send},1}]}]] + {{{0,257,0},send},1}, + {{{0,257,0},send,{'Result-Code',2001}},1}]}]] </pre> <p> @@ -1918,13 +1923,16 @@ A return value for the server above might look as follows.</p> {send_pend,0}]}]}, {statistics,[{{{0,280,0},recv},62}, {{{0,280,1},send},62}, - {{{0,258,0},send,{'Result-Code',2001}},3}, + {{{0,280,0},recv,{'Result-Code',2001}},62}, {{{0,258,1},recv},3}, {{{0,258,0},send},3}, + {{{0,258,0},send,{'Result-Code',2001}},3}, {{{0,280,1},recv},66}, {{{0,280,0},send},66}, + {{{0,280,0},send,{'Result-Code',2001}},66}, {{{0,257,1},recv},1}, - {{{0,257,0},send},1}]}]] + {{{0,257,0},send},1}, + {{{0,257,0},send,{'Result-Code',2001}},1}]}]] </pre> <p> diff --git a/lib/diameter/src/diameter.appup.src b/lib/diameter/src/diameter.appup.src index 12b09889d1..618d5a7f10 100644 --- a/lib/diameter/src/diameter.appup.src +++ b/lib/diameter/src/diameter.appup.src @@ -47,10 +47,9 @@ {"1.9.2.2", [{restart_application, diameter}]}, %% 17.5.6.7 {"1.9.2.3", [{restart_application, diameter}]}, %% 17.5.6.8 {"1.10", [{restart_application, diameter}]}, %% 18.0 - {"1.11", [{load_module, diameter_traffic}, %% 18.1 - {update, diameter_service, {advanced, []}}]}, - {"1.11.1", [{load_module, diameter_traffic}, %% 18.2 - {update, diameter_service, {advanced, []}}]} + {"1.11", [{restart_application, diameter}]}, %% 18.1 + {"1.11.1", [{restart_application, diameter}]}, %% 18.2 + {"1.11.2", [{restart_application, diameter}]} %% 18.3 ], [ {"0.9", [{restart_application, diameter}]}, @@ -80,6 +79,7 @@ {"1.9.2.3", [{restart_application, diameter}]}, {"1.10", [{restart_application, diameter}]}, {"1.11", [{restart_application, diameter}]}, - {"1.11.1", [{restart_application, diameter}]} + {"1.11.1", [{restart_application, diameter}]}, + {"1.11.2", [{restart_application, diameter}]} ] }. diff --git a/lib/diameter/vsn.mk b/lib/diameter/vsn.mk index 836def8447..cb750c69a3 100644 --- a/lib/diameter/vsn.mk +++ b/lib/diameter/vsn.mk @@ -1,6 +1,6 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2010-2015. All Rights Reserved. +# Copyright Ericsson AB 2010-2016. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -17,5 +17,5 @@ # %CopyrightEnd% APPLICATION = diameter -DIAMETER_VSN = 1.11.2 +DIAMETER_VSN = 1.12 APP_VSN = $(APPLICATION)-$(DIAMETER_VSN)$(PRE_VSN) diff --git a/lib/eldap/src/eldap.erl b/lib/eldap/src/eldap.erl index dc236f8a44..625309271b 100644 --- a/lib/eldap/src/eldap.erl +++ b/lib/eldap/src/eldap.erl @@ -564,7 +564,12 @@ loop(Cpid, Data) -> ?MODULE:loop(Cpid, NewData); {_From, close} -> - {no_reply,_NewData} = do_unbind(Data), + % Ignore tcp error if connection is already closed. + try do_unbind(Data) of + {no_reply,_NewData} -> ok + catch + throw:{gen_tcp_error, _TcpErr} -> ok + end, unlink(Cpid), exit(closed); diff --git a/lib/eldap/test/eldap_basic_SUITE.erl b/lib/eldap/test/eldap_basic_SUITE.erl index 9a23f74e22..536e891a1e 100644 --- a/lib/eldap/test/eldap_basic_SUITE.erl +++ b/lib/eldap/test/eldap_basic_SUITE.erl @@ -94,7 +94,8 @@ connection_tests() -> client_side_start_tls_timeout, client_side_bind_timeout, client_side_add_timeout, - client_side_search_timeout + client_side_search_timeout, + close_after_tcp_error ]. @@ -312,6 +313,30 @@ tcp_connection(Config) -> end. %%%---------------------------------------------------------------- + +close_after_tcp_error(Config) -> + Host = proplists:get_value(listen_host, Config), + Port = proplists:get_value(listen_port, Config), + Opts = proplists:get_value(tcp_connect_opts, Config), + T = 1000, + case eldap:open([Host], [{timeout,T},{port,Port}|Opts]) of + {ok,H} -> + Sl = proplists:get_value(listen_socket, Config), + gen_tcp:close(Sl), + {error,{gen_tcp_error,closed}} = eldap:simple_bind(H, anon, anon), + ok = eldap:close(H), + wait_for_close(H); + Other -> ct:fail("eldap:open failed: ~p",[Other]) + end. + +wait_for_close(H) -> + case erlang:is_process_alive(H) of + true -> timer:sleep(100), + wait_for_close(H); + false -> ok + end. + +%%%---------------------------------------------------------------- ssl_connection(Config) -> Host = proplists:get_value(listen_host, Config), Port = proplists:get_value(ssl_listen_port, Config), diff --git a/lib/et/doc/src/et_desc.xmlsrc b/lib/et/doc/src/et_desc.xmlsrc index 96a4a9df31..29e0ab1fe3 100644 --- a/lib/et/doc/src/et_desc.xmlsrc +++ b/lib/et/doc/src/et_desc.xmlsrc @@ -55,13 +55,12 @@ <p></p> <code type="none"><![CDATA[ - % erl -pa et/examples - Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] - [async-threads:0] [kernel-poll:false] +% erl -pa et/examples +Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] - Eshell V5.7.4 (abort with ^G) - 1> {ok, Viewer} = et_viewer:start([]). - {ok,<0.40.0>}]]></code> +Eshell V5.7.4 (abort with ^G) +1> {ok, Viewer} = et_viewer:start([]). +{ok,<0.40.0>}]]></code> <p>A <c>Viewer</c> gets trace <c>Events</c> from its <c>Collector</c> by polling it regularly for more <c>Events</c> to @@ -69,40 +68,38 @@ <c>Collector</c> with <c>et_collector:report_event/6</c>:</p> <code type="none"><![CDATA[ - 2> Collector = et_viewer:get_collector_pid(Viewer). - <0.39.0> - 3> et_collector:report_event(Collector, 60, my_shell, mnesia_tm, start_outer, - 3> "Start outer transaction"), - 3> et_collector:report_event(Collector, 40, mnesia_tm, my_shell, new_tid, - 3> "New transaction id is 4711"), - 3> et_collector:report_event(Collector, 20, my_shell, mnesia_locker, try_write_lock, - 3> "Acquire write lock for {my_tab, key}"), - 3> et_collector:report_event(Collector, 10, mnesia_locker, my_shell, granted, - 3> "You got the write lock for {my_tab, key}"), - 3> et_collector:report_event(Collector, 60, my_shell, do_commit, - 3> "Perform transaction commit"), - 3> et_collector:report_event(Collector, 40, my_shell, mnesia_locker, release_tid, - 3> "Release all locks for transaction 4711"), - 3> et_collector:report_event(Collector, 60, my_shell, mnesia_tm, delete_transaction, - 3> "End of outer transaction"), - 3> et_collector:report_event(Collector, 20, my_shell, end_outer, - 3> "Transaction returned {atomic, ok}"). - {ok,{table_handle,<0.39.0>,16402,trace_ts, - #Fun<et_collector.0.62831470>}}]]></code> +2> Collector = et_viewer:get_collector_pid(Viewer). +<0.39.0> +3> et_collector:report_event(Collector, 60, my_shell, mnesia_tm, start_outer, +3> "Start outer transaction"), +3> et_collector:report_event(Collector, 40, mnesia_tm, my_shell, new_tid, +3> "New transaction id is 4711"), +3> et_collector:report_event(Collector, 20, my_shell, mnesia_locker, try_write_lock, +3> "Acquire write lock for {my_tab, key}"), +3> et_collector:report_event(Collector, 10, mnesia_locker, my_shell, granted, +3> "You got the write lock for {my_tab, key}"), +3> et_collector:report_event(Collector, 60, my_shell, do_commit, +3> "Perform transaction commit"), +3> et_collector:report_event(Collector, 40, my_shell, mnesia_locker, release_tid, +3> "Release all locks for transaction 4711"), +3> et_collector:report_event(Collector, 60, my_shell, mnesia_tm, delete_transaction, +3> "End of outer transaction"), +3> et_collector:report_event(Collector, 20, my_shell, end_outer, +3> "Transaction returned {atomic, ok}"). +{ok,{table_handle,<0.39.0>,16402,trace_ts, + #Fun<et_collector.0.62831470>}}]]></code> <p>This actually is a simulation of the process <c>Events</c> caused by a <c>Mnesia</c> transaction that writes a record in a local table:</p> <code type="none"><![CDATA[ - mnesia:transaction(fun() -> mnesia:write({my_tab, key, val}) end).]]></code> +mnesia:transaction(fun() -> mnesia:write({my_tab, key, val}) end).]]></code> <p>At this stage when we have a couple of <c>Events</c>, it is time to show how it looks like in the graphical interface of <c>et_viewer</c>:</p> - <p></p> - <image file="sim_trans.png"> <icaption>A simulated Mnesia transaction which writes one record</icaption> </image> @@ -144,11 +141,11 @@ <p></p> <code type="none"><![CDATA[ - filter(TraceData) -> false | true | {true, NewEvent} +filter(TraceData) -> false | true | {true, NewEvent} - TraceData = Event | erlang_trace_data() - Event = #event{} - NewEvent = #event{}]]></code> +TraceData = Event | erlang_trace_data() +Event = #event{} +NewEvent = #event{}]]></code> <p>The interface of the filter function is the same as the the filter functions for the good old <c>lists:filtermap/2</c>. If the filter @@ -204,10 +201,10 @@ <p></p> <code type="none"><![CDATA[ - 4> Fun = fun(E) -> et_demo:mgr_actors(E) end. - #Fun<erl_eval.6.13229925> - 5> et_collector:dict_insert(Collector, {filter, mgr_actors}, Fun). - ok]]></code> +4> Fun = fun(E) -> et_demo:mgr_actors(E) end. +#Fun<erl_eval.6.13229925> +5> et_collector:dict_insert(Collector, {filter, mgr_actors}, Fun). +ok]]></code> <p>you will see that the <c>Filter</c> menu in all viewers have got a new entry called <c>mgr_actors</c>. Select it, and a new @@ -228,21 +225,16 @@ <c>Contents Viewer</c> window to pop up, showing the <c>Event</c> in the <c>mgr_actors</c> view:</p> - <p></p> - <image file="sim_trans_contents_viewer_mgr_actors.png"> - <icaption>The trace <c>Event</c> in the mgr_actors view</icaption> + <icaption>The trace Event in the mgr_actors view</icaption> </image> <p>Select the <c>all</c> entry in the <c>Filters</c> menu and a new <c>Contents Viewer window</c> will pop up showing the same trace <c>Event</c> in the collectors view:</p> - <p></p> - <image file="sim_trans_contents_viewer_collector.png"> - <icaption>The same trace <c>Event</c> in the collectors - view</icaption> + <icaption>The same trace Event in the collectors view</icaption> </image> </section> @@ -311,7 +303,7 @@ <c>et_collector</c> or <c>et_viewer</c> in order to activate the global tracing. There is no restriction on how many concurrent (anonymous) collectors you can have, but you can only have one - <b>global</b> <c>Collector</c> as its name is registered in + <em>global</em> <c>Collector</c> as its name is registered in <c>global</c>.</p> <p>In order to further simplify the tracing, you can make use of diff --git a/lib/et/doc/src/et_examples.xmlsrc b/lib/et/doc/src/et_examples.xmlsrc index 6f143a397e..f4d94f7cb0 100644 --- a/lib/et/doc/src/et_examples.xmlsrc +++ b/lib/et/doc/src/et_examples.xmlsrc @@ -55,34 +55,32 @@ <p></p> <code type="none"><![CDATA[ - mnesia:transaction(fun() -> mnesia:write({my_tab, key, val}) end).]]></code> +mnesia:transaction(fun() -> mnesia:write({my_tab, key, val}) end).]]></code> <p>And the viewer window will look like:</p> <p></p> <code type="none"><![CDATA[ - Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] - [async-threads:0] [kernel-poll:false] +Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] - Eshell V5.7.4 (abort with ^G) - 1> {ok, Viewer} = et_viewer:start([]). - {ok,<0.40.0>;} - 2> et_demo:sim_trans(). - {ok,{table_handle,<0.45.0>,24596,trace_ts, - #Fun<et_collector.0.62831470>}}]]></code> +Eshell V5.7.4 (abort with ^G) +1> {ok, Viewer} = et_viewer:start([]). +{ok,<0.40.0>;} +2> et_demo:sim_trans(). +{ok,{table_handle,<0.45.0>,24596,trace_ts, + #Fun<et_collector.0.62831470>}}]]></code> <p></p> <image file="sim_trans.png"> - <icaption>A simulated <c>Mnesia</c> transaction which writes one - record</icaption> + <icaption>A simulated Mnesia transaction which writes one record</icaption> </image> </section> <section> - <title>Some convenient functions used in the <c>Mnesia</c> transaction + <title>Some convenient functions used in the Mnesia transaction example</title> <p>The <c>module_as_actor</c> filter converts the <c>Event @@ -173,21 +171,19 @@ <p></p> <code type="none"><![CDATA[ - erl -pa ../examples - Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] - [async-threads:0] [kernel-poll:false] +erl -pa ../examples +Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] + [async-threads:0] [kernel-poll:false] - Eshell V5.7.4 (abort with ^G) - 1> et_demo:live_trans(). - {atomic,ok}]]></code> +Eshell V5.7.4 (abort with ^G) +1> et_demo:live_trans(). +{atomic,ok}]]></code> <p>Please, explore the different filters in order to see how the traced transaction can be seen from different point of views:</p> - <p></p> - <image file="live_trans.png"> - <icaption>A real <c>Mnesia</c> transaction which writes one record</icaption> + <icaption>A real Mnesia transaction which writes one record</icaption> </image> </section> @@ -215,20 +211,20 @@ <p></p> <code type="none"><![CDATA[ - -module(megaco_filter). - -export([start/0]). - - start() -> - Options = - [{event_order, event_ts}, - {scale, 3}, - {max_actors, infinity}, - {trace_pattern, {megaco, max}}, - {trace_global, true}, - {dict_insert, {filter, megaco_filter}, fun filter/1}, - {active_filter, megaco_filter}, - {title, "Megaco tracer - Erlang/OTP"}], - et_viewer:start(Options).]]></code> +-module(megaco_filter). +-export([start/0]). + +start() -> + Options = + [{event_order, event_ts}, + {scale, 3}, + {max_actors, infinity}, + {trace_pattern, {megaco, max}}, + {trace_global, true}, + {dict_insert, {filter, megaco_filter}, fun filter/1}, + {active_filter, megaco_filter}, + {title, "Megaco tracer - Erlang/OTP"}], + et_viewer:start(Options).]]></code> <p>First we start an Erlang node with a global <c>Collector</c> and its <c>Viewer</c>.</p> @@ -236,13 +232,12 @@ <p></p> <code type="none"><![CDATA[ - erl -sname observer - Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] - [async-threads:0] [kernel-poll:false] +erl -sname observer +Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] - Eshell V5.7.4 (abort with ^G) - (observer@falco)1> megaco_filter:start(). - {ok,<0.48.0>}]]></code> +Eshell V5.7.4 (abort with ^G) +(observer@falco)1> megaco_filter:start(). +{ok,<0.48.0>}]]></code> <p>Secondly we start another Erlang node which we connect the observer node, before we start the application that we want to @@ -253,28 +248,27 @@ <p></p> <code type="none"><![CDATA[ - erl -sname mgc -pa ../../megaco/examples/simple - Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] - [async-threads:0] [kernel-poll:false] - - Eshell V5.7.4 (abort with ^G) - (mgc@falco)1> net:ping(observer@falco). - pong - (mgc@falco)2> megaco:start(). - ok - (mgc@falco)3> megaco_simple_mgc:start(). - {ok,[{ok,2944, - {megaco_receive_handle,{deviceName,"controller"}, - megaco_pretty_text_encoder,[],megaco_tcp,dynamic}}, - {ok,2944, - {megaco_receive_handle,{deviceName,"controller"}, - megaco_pretty_text_encoder,[],megaco_udp,dynamic}}, - {ok,2945, - {megaco_receive_handle,{deviceName,"controller"}, - megaco_binary_encoder,[],megaco_tcp,dynamic}}, - {ok,2945, - {megaco_receive_handle,{deviceName,"controller"}, - megaco_binary_encoder,[],megaco_udp,dynamic}}]}]]></code> +erl -sname mgc -pa ../../megaco/examples/simple +Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] + +Eshell V5.7.4 (abort with ^G) +(mgc@falco)1> net:ping(observer@falco). +pong +(mgc@falco)2> megaco:start(). +ok +(mgc@falco)3> megaco_simple_mgc:start(). +{ok,[{ok,2944, + {megaco_receive_handle,{deviceName,"controller"}, + megaco_pretty_text_encoder,[],megaco_tcp,dynamic}}, + {ok,2944, + {megaco_receive_handle,{deviceName,"controller"}, + megaco_pretty_text_encoder,[],megaco_udp,dynamic}}, + {ok,2945, + {megaco_receive_handle,{deviceName,"controller"}, + megaco_binary_encoder,[],megaco_tcp,dynamic}}, + {ok,2945, + {megaco_receive_handle,{deviceName,"controller"}, + megaco_binary_encoder,[],megaco_udp,dynamic}}]}]]></code> <p>And finally we start an Erlang node for the Media Gateways and connect to the observer node. Each Media Gateway connects to the @@ -288,94 +282,87 @@ <p></p> <code type="none"><![CDATA[ - Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] - [async-threads:0] [kernel-poll:false] - - Eshell V5.7.4 (abort with ^G) - (mg@falco)1> net:ping(observer@falco). - pong - (mg@falco)2> megaco_simple_mg:start(). - [{{deviceName,"gateway_tt"}, - {error,{start_user,megaco_not_started}}}, - {{deviceName,"gateway_tb"}, - {error,{start_user,megaco_not_started}}}, - {{deviceName,"gateway_ut"}, - {error,{start_user,megaco_not_started}}}, - {{deviceName,"gateway_ub"}, - {error,{start_user,megaco_not_started}}}] - (mg@falco)3> megaco:start(). - ok - (mg@falco)4> megaco_simple_mg:start(). - [{{deviceName,"gateway_tt"}, - {1, - {ok,[{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, - [{serviceChangeReply, - {'ServiceChangeReply', - [{megaco_term_id,false,["root"]}], - {serviceChangeResParms, - {'ServiceChangeResParm', - {deviceName,"controller"}, - asn1_NOVALUE,asn1_NOVALUE,asn1_NOVALUE, - asn1_NOVALUE}}}}]}]}}}, - {{deviceName,"gateway_tb"}, - {1, - {ok,[{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, - [{serviceChangeReply, - {'ServiceChangeReply', - [{megaco_term_id,false,["root"]}], - {serviceChangeResParms, - {'ServiceChangeResParm', - {deviceName,"controller"}, - asn1_NOVALUE,asn1_NOVALUE,asn1_NOVALUE, - asn1_NOVALUE}}}}]}]}}}, - {{deviceName,"gateway_ut"}, - {1, - {ok,[{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, - [{serviceChangeReply, - {'ServiceChangeReply', - [{megaco_term_id,false,["root"]}], - {serviceChangeResParms, - {'ServiceChangeResParm', - {deviceName,"controller"}, - asn1_NOVALUE,asn1_NOVALUE,asn1_NOVALUE, - asn1_NOVALUE}}}}]}]}}}, - {{deviceName,"gateway_ub"}, - {1, - {ok,[{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, - [{serviceChangeReply, - {'ServiceChangeReply', - [{megaco_term_id,false,["root"]}], - {serviceChangeResParms, - {'ServiceChangeResParm', - {deviceName,"controller"}, - asn1_NOVALUE,asn1_NOVALUE, - asn1_NOVALUE,...}}}}]}]}}}]]]></code> +Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] + +Eshell V5.7.4 (abort with ^G) +(mg@falco)1> net:ping(observer@falco). +pong +(mg@falco)2> megaco_simple_mg:start(). +[{{deviceName,"gateway_tt"}, + {error,{start_user,megaco_not_started}}}, + {{deviceName,"gateway_tb"}, + {error,{start_user,megaco_not_started}}}, + {{deviceName,"gateway_ut"}, + {error,{start_user,megaco_not_started}}}, + {{deviceName,"gateway_ub"}, + {error,{start_user,megaco_not_started}}}] +(mg@falco)3> megaco:start(). +ok +(mg@falco)4> megaco_simple_mg:start(). +[{{deviceName,"gateway_tt"}, + {1, + {ok,[{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, + [{serviceChangeReply, + {'ServiceChangeReply', + [{megaco_term_id,false,["root"]}], + {serviceChangeResParms, + {'ServiceChangeResParm', + {deviceName,"controller"}, + asn1_NOVALUE,asn1_NOVALUE,asn1_NOVALUE, + asn1_NOVALUE}}}}]}]}}}, + {{deviceName,"gateway_tb"}, + {1, + {ok,[{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, + [{serviceChangeReply, + {'ServiceChangeReply', + [{megaco_term_id,false,["root"]}], + {serviceChangeResParms, + {'ServiceChangeResParm', + {deviceName,"controller"}, + asn1_NOVALUE,asn1_NOVALUE,asn1_NOVALUE, + asn1_NOVALUE}}}}]}]}}}, + {{deviceName,"gateway_ut"}, + {1, + {ok,[{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, + [{serviceChangeReply, + {'ServiceChangeReply', + [{megaco_term_id,false,["root"]}], + {serviceChangeResParms, + {'ServiceChangeResParm', + {deviceName,"controller"}, + asn1_NOVALUE,asn1_NOVALUE,asn1_NOVALUE, + asn1_NOVALUE}}}}]}]}}}, + {{deviceName,"gateway_ub"}, + {1, + {ok,[{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, + [{serviceChangeReply, + {'ServiceChangeReply', + [{megaco_term_id,false,["root"]}], + {serviceChangeResParms, + {'ServiceChangeResParm', + {deviceName,"controller"}, + asn1_NOVALUE,asn1_NOVALUE, + asn1_NOVALUE,...}}}}]}]}}}]]]></code> <p>The <c>Megaco</c> adopted viewer looks like this, when we have clicked - on the <b>[gateway_tt]</b> actor name in order to only display the events + on the <em>[gateway_tt]</em> actor name in order to only display the events regarding that actor:</p> - <p></p> - <image file="megaco_tracer.png"> <icaption>The viewer adopted for Megaco</icaption> </image> <p>A pretty printed <c>Megaco</c> message looks like this:</p> - <p></p> - <image file="megaco_filter.png"> - <icaption>A textual <c>Megaco</c> message</icaption> + <icaption>A textual Megaco message</icaption> </image> <p>And the corresponding internal form for the same <c>Megaco</c> message looks like this:</p> - <p></p> - <image file="megaco_collector.png"> - <icaption>The internal form of a <c>Megaco</c> message</icaption> + <icaption>The internal form of a Megaco message</icaption> </image> </section> diff --git a/lib/et/doc/src/et_tutorial.xmlsrc b/lib/et/doc/src/et_tutorial.xmlsrc index b5d1b815be..b6e1ca141c 100644 --- a/lib/et/doc/src/et_tutorial.xmlsrc +++ b/lib/et/doc/src/et_tutorial.xmlsrc @@ -75,12 +75,10 @@ <codeinclude file="../../examples/et_display_demo.erl" tag="%module" type="erl"></codeinclude> <p>When you run the <c>et_display_demo:test().</c> function in the - example above, the <c>Viewer</c> window will look like this:</p>. - - <p></p> + example above, the <c>Viewer</c> window will look like this:</p> <image file="coffee_order.png"> - <icaption>Screenshot of the <c>Viewer</c> window</icaption> + <icaption>Screenshot of the Viewer window</icaption> </image> </section> @@ -262,14 +260,11 @@ </list> <p>When you run the <c>et_trace_demo:test()</c> function above, the - <c>Viewer</c> window will look like this screenshot:</p>. + <c>Viewer</c> window will look like this screenshot:</p> - <p></p> - <image file="coffee_order.png"> - <icaption>Screenshot of the <c>Viewer</c> window</icaption> + <icaption>Screenshot of the Viewer window</icaption> </image> </section> - </chapter> diff --git a/lib/hipe/cerl/erl_types.erl b/lib/hipe/cerl/erl_types.erl index 84addcc105..b037a4360c 100644 --- a/lib/hipe/cerl/erl_types.erl +++ b/lib/hipe/cerl/erl_types.erl @@ -220,6 +220,7 @@ is_opaque_type/2, is_erl_type/1, atom_to_string/1, + var_table__new/0, map_pairwise_merge/3 ]). @@ -236,7 +237,7 @@ -export([t_is_identifier/1]). -endif. --export_type([erl_type/0, opaques/0, type_table/0]). +-export_type([erl_type/0, opaques/0, type_table/0, var_table/0]). %%-define(DEBUG, true). @@ -380,7 +381,7 @@ -type type_table() :: dict:dict(record_key() | type_key(), record_value() | type_value()). --type var_table() :: #{atom() => erl_type()}. +-opaque var_table() :: #{atom() => erl_type()}. %%----------------------------------------------------------------------------- %% Unions @@ -5464,6 +5465,17 @@ family(L) -> sofs:to_external(F). %%============================================================================= +%% +%% Interface functions for abstract data types defined in this module +%% +%%============================================================================= + +-spec var_table__new() -> var_table(). + +var_table__new() -> + maps:new(). + +%%============================================================================= %% Consistency-testing function(s) below %%============================================================================= diff --git a/lib/inets/src/http_server/httpd_response.erl b/lib/inets/src/http_server/httpd_response.erl index c0b5f09faf..1374b7e85e 100644 --- a/lib/inets/src/http_server/httpd_response.erl +++ b/lib/inets/src/http_server/httpd_response.erl @@ -390,8 +390,7 @@ send_response_old(#mod{socket_type = Type, send_header(ModData, StatusCode, [{content_length, content_length(NewResponse)}]), httpd_socket:deliver(Type, Sock, NewResponse); - - {error, _Reason} -> + _ -> send_status(ModData, 500, "Internal Server Error") end. diff --git a/lib/inets/src/http_server/mod_htaccess.erl b/lib/inets/src/http_server/mod_htaccess.erl index 2bc0d79218..7b742bba24 100644 --- a/lib/inets/src/http_server/mod_htaccess.erl +++ b/lib/inets/src/http_server/mod_htaccess.erl @@ -412,8 +412,8 @@ getAuthenticatingDataFromHeader(Info)-> case httpd_util:split(UnCodedString,":",2) of {ok,[User,PassWord]}-> {user,User,PassWord}; - {error,Error}-> - {error,Error} + Other -> + {error, Other} end end; BadCredentials -> diff --git a/lib/inets/src/inets_app/Makefile b/lib/inets/src/inets_app/Makefile index a294381f67..1d870c14e8 100644 --- a/lib/inets/src/inets_app/Makefile +++ b/lib/inets/src/inets_app/Makefile @@ -49,8 +49,7 @@ MODULES = \ inets_sup \ inets_trace \ inets_lib \ - inets_time_compat \ - inets_regexp + inets_time_compat INTERNAL_HRL_FILES = inets_internal.hrl EXTERNAL_HRL_FILES = ../../include/httpd.hrl \ diff --git a/lib/inets/src/inets_app/inets.app.src b/lib/inets/src/inets_app/inets.app.src index abf8c0cdea..5706a335d7 100644 --- a/lib/inets/src/inets_app/inets.app.src +++ b/lib/inets/src/inets_app/inets.app.src @@ -29,7 +29,6 @@ inets_trace, inets_lib, inets_time_compat, - inets_regexp, %% FTP ftp, diff --git a/lib/inets/src/inets_app/inets_regexp.erl b/lib/inets/src/inets_app/inets_regexp.erl deleted file mode 100644 index fc1608bc5a..0000000000 --- a/lib/inets/src/inets_app/inets_regexp.erl +++ /dev/null @@ -1,414 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2009. All Rights Reserved. -%% -%% Licensed under the Apache License, Version 2.0 (the "License"); -%% you may not use this file except in compliance with the License. -%% You may obtain a copy of the License at -%% -%% http://www.apache.org/licenses/LICENSE-2.0 -%% -%% Unless required by applicable law or agreed to in writing, software -%% distributed under the License is distributed on an "AS IS" BASIS, -%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -%% See the License for the specific language governing permissions and -%% limitations under the License. -%% -%% %CopyrightEnd% -%% - --module(inets_regexp). - --export([parse/1, match/2, first_match/2, split/2, sub/3, gsub/3]). - - -%%%========================================================================= -%%% API -%%%========================================================================= - -%% parse(RegExp) -> {ok, RE} | {error, E}. -%% Parse the regexp described in the string RegExp. - -parse(S) -> - case (catch reg(S)) of - {R, []} -> - {ok, R}; - {_R, [C|_]} -> - {error, {illegal, [C]}}; - {error, E} -> - {error, E} - end. - - -%% Find the longest match of RegExp in String. - -match(S, RegExp) when is_list(RegExp) -> - case parse(RegExp) of - {ok,RE} -> match(S, RE); - {error,E} -> {error,E} - end; -match(S, RE) -> - case match(RE, S, 1, 0, -1) of - {Start,Len} when Len >= 0 -> - {match, Start, Len}; - {_Start,_Len} -> - nomatch - end. - -%% Find the first match of RegExp in String. - -first_match(S, RegExp) when is_list(RegExp) -> - case parse(RegExp) of - {ok, RE} -> - first_match(S, RE); - {error, E} -> - {error, E} - end; -first_match(S, RE) -> - case first_match(RE, S, 1) of - {Start,Len} when Len >= 0 -> - {match, Start,Len}; - nomatch -> - nomatch - end. - -first_match(RE, S, St) when S =/= [] -> - case re_apply(S, St, RE) of - {match, P, _Rest} -> - {St, P-St}; - nomatch -> - first_match(RE, tl(S), St+1) - end; -first_match(_RE, [], _St) -> - nomatch. - - -match(RE, S, St, Pos, L) -> - case first_match(RE, S, St) of - {St1, L1} -> - Nst = St1 + 1, - if L1 > L -> - match(RE, lists:nthtail(Nst-St, S), Nst, St1, L1); - true -> - match(RE, lists:nthtail(Nst-St, S), Nst, Pos, L) - end; - nomatch -> - {Pos, L} - end. - - -%% Split a string into substrings where the RegExp describes the -%% field seperator. The RegExp " " is specially treated. - -split(String, " ") -> %This is really special - {ok, RE} = parse("[ \t]+"), - case split_apply(String, RE, true) of - [[]|Ss] -> - {ok,Ss}; - Ss -> - {ok,Ss} - end; -split(String, RegExp) when is_list(RegExp) -> - case parse(RegExp) of - {ok, RE} -> - {ok, split_apply(String, RE, false)}; - {error, E} -> - {error,E} - end; -split(String, RE) -> - {ok, split_apply(String, RE, false)}. - - -%% Substitute the first match of the regular expression RegExp -%% with the string Replace in String. Accept pre-parsed regular -%% expressions. - -sub(String, RegExp, Rep) when is_list(RegExp) -> - case parse(RegExp) of - {ok, RE} -> - sub(String, RE, Rep); - {error, E} -> - {error, E} - end; -sub(String, RE, Rep) -> - Ss = sub_match(String, RE, 1), - {ok, sub_repl(Ss, Rep, String, 1), length(Ss)}. - - -%% Substitute every match of the regular expression RegExp with -%% the string New in String. Accept pre-parsed regular expressions. - -gsub(String, RegExp, Rep) when is_list(RegExp) -> - case parse(RegExp) of - {ok, RE} -> - gsub(String, RE, Rep); - {error, E} -> - {error, E} - end; -gsub(String, RE, Rep) -> - Ss = matches(String, RE, 1), - {ok, sub_repl(Ss, Rep, String, 1), length(Ss)}. - - -%%%======================================================================== -%%% Internal functions -%%%======================================================================== - -%% This is the regular expression grammar used. It is equivalent to the -%% one used in AWK, except that we allow ^ $ to be used anywhere and fail -%% in the matching. -%% -%% reg -> reg1 : '$1'. -%% reg1 -> reg1 "|" reg2 : {'or','$1','$2'}. -%% reg1 -> reg2 : '$1'. -%% reg2 -> reg2 reg3 : {concat,'$1','$2'}. -%% reg2 -> reg3 : '$1'. -%% reg3 -> reg3 "*" : {kclosure,'$1'}. -%% reg3 -> reg3 "+" : {pclosure,'$1'}. -%% reg3 -> reg3 "?" : {optional,'$1'}. -%% reg3 -> reg4 : '$1'. -%% reg4 -> "(" reg ")" : '$2'. -%% reg4 -> "\\" char : '$2'. -%% reg4 -> "^" : bos. -%% reg4 -> "$" : eos. -%% reg4 -> "." : char. -%% reg4 -> "[" class "]" : {char_class,char_class('$2')} -%% reg4 -> "[" "^" class "]" : {comp_class,char_class('$3')} -%% reg4 -> "\"" chars "\"" : char_string('$2') -%% reg4 -> char : '$1'. -%% reg4 -> empty : epsilon. -%% The grammar of the current regular expressions. The actual parser -%% is a recursive descent implementation of the grammar. - -reg(S) -> reg1(S). - -%% reg1 -> reg2 reg1' -%% reg1' -> "|" reg2 -%% reg1' -> empty - -reg1(S0) -> - {L,S1} = reg2(S0), - reg1p(S1, L). - -reg1p([$||S0], L) -> - {R,S1} = reg2(S0), - reg1p(S1, {'or',L,R}); -reg1p(S, L) -> {L,S}. - -%% reg2 -> reg3 reg2' -%% reg2' -> reg3 -%% reg2' -> empty - -reg2(S0) -> - {L,S1} = reg3(S0), - reg2p(S1, L). - -reg2p([C|S0], L) when (C =/= $|) andalso (C =/= $)) -> - {R,S1} = reg3([C|S0]), - reg2p(S1, {concat,L,R}); -reg2p(S, L) -> {L,S}. - -%% reg3 -> reg4 reg3' -%% reg3' -> "*" reg3' -%% reg3' -> "+" reg3' -%% reg3' -> "?" reg3' -%% reg3' -> empty - -reg3(S0) -> - {L,S1} = reg4(S0), - reg3p(S1, L). - -reg3p([$*|S], L) -> reg3p(S, {kclosure,L}); -reg3p([$+|S], L) -> reg3p(S, {pclosure,L}); -reg3p([$?|S], L) -> reg3p(S, {optional,L}); -reg3p(S, L) -> {L,S}. - -reg4([$(|S0]) -> - case reg(S0) of - {R,[$)|S1]} -> {R,S1}; - {_R,_S} -> throw({error,{unterminated,"("}}) - end; -reg4([$\\,O1,O2,O3|S]) - when ((O1 >= $0) andalso - (O1 =< $7) andalso - (O2 >= $0) andalso - (O2 =< $7) andalso - (O3 >= $0) andalso - (O3 =< $7)) -> - {(O1*8 + O2)*8 + O3 - 73*$0,S}; -reg4([$\\,C|S]) -> - {escape_char(C),S}; -reg4([$\\]) -> - throw({error, {unterminated,"\\"}}); -reg4([$^|S]) -> - {bos,S}; -reg4([$$|S]) -> - {eos,S}; -reg4([$.|S]) -> - {{comp_class,"\n"},S}; -reg4("[^" ++ S0) -> - case char_class(S0) of - {Cc,[$]|S1]} -> {{comp_class,Cc},S1}; - {_Cc,_S} -> throw({error,{unterminated,"["}}) - end; -reg4([$[|S0]) -> - case char_class(S0) of - {Cc,[$]|S1]} -> {{char_class,Cc},S1}; - {_Cc,_S1} -> throw({error,{unterminated,"["}}) - end; -reg4([C|S]) - when (C =/= $*) andalso (C =/= $+) andalso (C =/= $?) andalso (C =/= $]) -> - {C, S}; -reg4([C|_S]) -> - throw({error,{illegal,[C]}}); -reg4([]) -> - {epsilon,[]}. - -escape_char($n) -> $\n; %\n = LF -escape_char($r) -> $\r; %\r = CR -escape_char($t) -> $\t; %\t = TAB -escape_char($v) -> $\v; %\v = VT -escape_char($b) -> $\b; %\b = BS -escape_char($f) -> $\f; %\f = FF -escape_char($e) -> $\e; %\e = ESC -escape_char($s) -> $\s; %\s = SPACE -escape_char($d) -> $\d; %\d = DEL -escape_char(C) -> C. - -char_class([$]|S]) -> char_class(S, [$]]); -char_class(S) -> char_class(S, []). - -char($\\, [O1,O2,O3|S]) when - O1 >= $0, O1 =< $7, O2 >= $0, O2 =< $7, O3 >= $0, O3 =< $7 -> - {(O1*8 + O2)*8 + O3 - 73*$0,S}; -char($\\, [C|S]) -> {escape_char(C),S}; -char(C, S) -> {C,S}. - -char_class([C1|S0], Cc) when C1 =/= $] -> - case char(C1, S0) of - {Cf,[$-,C2|S1]} when C2 =/= $] -> - case char(C2, S1) of - {Cl,S2} when Cf < Cl -> char_class(S2, [{Cf,Cl}|Cc]); - {Cl,_S2} -> throw({error,{char_class,[Cf,$-,Cl]}}) - end; - {C,S1} -> char_class(S1, [C|Cc]) - end; -char_class(S, Cc) -> {Cc,S}. - - -%% re_apply(String, StartPos, RegExp) -> re_app_res(). -%% -%% Apply the (parse of the) regular expression RegExp to String. If -%% there is a match return the position of the remaining string and -%% the string if else return 'nomatch'. BestMatch specifies if we want -%% the longest match, or just a match. -%% -%% StartPos should be the real start position as it is used to decide -%% if we ae at the beginning of the string. -%% -%% Pass two functions to re_apply_or so it can decide, on the basis -%% of BestMatch, whether to just any take any match or try both to -%% find the longest. This is slower but saves duplicatng code. - -re_apply(S, St, RE) -> re_apply(RE, [], S, St). - -re_apply(epsilon, More, S, P) -> %This always matches - re_apply_more(More, S, P); -re_apply({'or',RE1,RE2}, More, S, P) -> - re_apply_or(re_apply(RE1, More, S, P), - re_apply(RE2, More, S, P)); -re_apply({concat,RE1,RE2}, More, S0, P) -> - re_apply(RE1, [RE2|More], S0, P); -re_apply({kclosure,CE}, More, S, P) -> - %% Be careful with the recursion, explicitly do one call before - %% looping. - re_apply_or(re_apply_more(More, S, P), - re_apply(CE, [{kclosure,CE}|More], S, P)); -re_apply({pclosure,CE}, More, S, P) -> - re_apply(CE, [{kclosure,CE}|More], S, P); -re_apply({optional,CE}, More, S, P) -> - re_apply_or(re_apply_more(More, S, P), - re_apply(CE, More, S, P)); -re_apply(bos, More, S, 1) -> re_apply_more(More, S, 1); -re_apply(eos, More, [$\n|S], P) -> re_apply_more(More, S, P); -re_apply(eos, More, [], P) -> re_apply_more(More, [], P); -re_apply({char_class,Cc}, More, [C|S], P) -> - case in_char_class(C, Cc) of - true -> re_apply_more(More, S, P+1); - false -> nomatch - end; -re_apply({comp_class,Cc}, More, [C|S], P) -> - case in_char_class(C, Cc) of - true -> nomatch; - false -> re_apply_more(More, S, P+1) - end; -re_apply(C, More, [C|S], P) when is_integer(C) -> - re_apply_more(More, S, P+1); -re_apply(_RE, _More, _S, _P) -> nomatch. - -%% re_apply_more([RegExp], String, Length) -> re_app_res(). - -re_apply_more([RE|More], S, P) -> re_apply(RE, More, S, P); -re_apply_more([], S, P) -> {match,P,S}. - -%% in_char_class(Char, Class) -> bool(). - -in_char_class(C, [{C1,C2}|_Cc]) when C >= C1, C =< C2 -> true; -in_char_class(C, [C|_Cc]) -> true; -in_char_class(C, [_|Cc]) -> in_char_class(C, Cc); -in_char_class(_C, []) -> false. - -%% re_apply_or(Match1, Match2) -> re_app_res(). -%% If we want the best match then choose the longest match, else just -%% choose one by trying sequentially. - -re_apply_or({match,P1,S1}, {match,P2,_S2}) when P1 >= P2 -> {match,P1,S1}; -re_apply_or({match,_P1,_S1}, {match,P2,S2}) -> {match,P2,S2}; -re_apply_or(nomatch, R2) -> R2; -re_apply_or(R1, nomatch) -> R1. - - -matches(S, RE, St) -> - case first_match(RE, S, St) of - {St1,0} -> - [{St1,0}|matches(string:substr(S, St1+2-St), RE, St1+1)]; - {St1,L1} -> - [{St1,L1}|matches(string:substr(S, St1+L1+1-St), RE, St1+L1)]; - nomatch -> - [] - end. - -sub_match(S, RE, St) -> - case first_match(RE, S, St) of - {St1,L1} -> [{St1,L1}]; - nomatch -> [] - end. - -sub_repl([{St,L}|Ss], Rep, S, Pos) -> - Rs = sub_repl(Ss, Rep, S, St+L), - string:substr(S, Pos, St-Pos) ++ - sub_repl(Rep, string:substr(S, St, L), Rs); -sub_repl([], _Rep, S, Pos) -> - string:substr(S, Pos). - -sub_repl([$&|Rep], M, Rest) -> M ++ sub_repl(Rep, M, Rest); -sub_repl("\\&" ++ Rep, M, Rest) -> [$&|sub_repl(Rep, M, Rest)]; -sub_repl([C|Rep], M, Rest) -> [C|sub_repl(Rep, M, Rest)]; -sub_repl([], _M, Rest) -> Rest. - -split_apply(S, RE, Trim) -> split_apply(S, 1, RE, Trim, []). - -split_apply([], _P, _RE, true, []) -> - []; -split_apply([], _P, _RE, _T, Sub) -> - [lists:reverse(Sub)]; -split_apply(S, P, RE, T, Sub) -> - case re_apply(S, P, RE) of - {match,P,_Rest} -> - split_apply(tl(S), P+1, RE, T, [hd(S)|Sub]); - {match,P1,Rest} -> - [lists:reverse(Sub)|split_apply(Rest, P1, RE, T, [])]; - nomatch -> - split_apply(tl(S), P+1, RE, T, [hd(S)|Sub]) - end. diff --git a/lib/odbc/configure.in b/lib/odbc/configure.in index 2871d91a1c..cb2b23d534 100644 --- a/lib/odbc/configure.in +++ b/lib/odbc/configure.in @@ -73,6 +73,16 @@ AC_CHECK_TOOL(LD, ld, '$(CC)') AC_SUBST(LD) +_search_path=/bin:/usr/bin:/usr/local/bin:$PATH + +AC_PATH_PROG(RM, rm, false, $_search_path) +if test "$ac_cv_path_RM" = false; then + AC_MSG_ERROR([No 'rm' command found]) +fi + +_search_path= + + # Sockets #-------------------------------------------------------------------- # Check for the existence of the -lsocket and -lnsl libraries. @@ -128,7 +138,7 @@ dnl Checks for library functions. AC_CHECK_FUNCS([memset socket]) # ODBC -/bin/rm -f "$ERL_TOP/lib/odbc/SKIP" +$RM -f "$ERL_TOP/lib/odbc/SKIP" LM_CHECK_THR_LIB AC_SUBST(THR_DEFS) diff --git a/lib/os_mon/c_src/cpu_sup.c b/lib/os_mon/c_src/cpu_sup.c index 61bf1b0352..353c7e674e 100644 --- a/lib/os_mon/c_src/cpu_sup.c +++ b/lib/os_mon/c_src/cpu_sup.c @@ -47,6 +47,10 @@ #include <kstat.h> #endif +#if (defined(__APPLE__) && defined(__MACH__)) +#include <mach/mach.h> +#endif + #include <errno.h> #if defined(__sun__) || defined(__linux__) @@ -73,6 +77,10 @@ typedef struct { #endif +#if (defined(__APPLE__) && defined(__MACH__)) +#define CU_OSX_VALUES (5) +#endif + #if defined(__FreeBSD__) #include <sys/resource.h> #include <sys/sysctl.h> @@ -164,7 +172,7 @@ static int processors_online() { } #endif -#if defined(__FreeBSD__) +#if (defined(__APPLE__) && defined(__MACH__)) || defined(__FreeBSD__) void getsysctl(const char *, void *, size_t); #endif @@ -173,7 +181,7 @@ int main(int argc, char** argv) { int rc; int sz; unsigned int *rv; -#if defined(__linux__) || defined(__FreeBSD__) +#if defined(__linux__) || (defined(__APPLE__) && defined(__MACH__)) ||defined(__FreeBSD__) unsigned int no_of_cpus = 0; #endif @@ -190,6 +198,13 @@ int main(int argc, char** argv) { } #endif +#if (defined(__APPLE__) && defined(__MACH__)) + getsysctl("hw.ncpu", &no_of_cpus, sizeof(int)); + if ( (rv = (unsigned int*)malloc(sizeof(unsigned int)*(2 + 2*no_of_cpus*CU_OSX_VALUES))) == NULL) { + error("cpu_sup: malloc error"); + } +#endif + #if defined(__FreeBSD__) getsysctl("hw.ncpu", &no_of_cpus, sizeof(int)); if ( (rv = (unsigned int*)malloc(sizeof(unsigned int)*(2 + 2*no_of_cpus*CU_BSD_VALUES))) == NULL) { @@ -222,7 +237,7 @@ int main(int argc, char** argv) { case AVG5: bsd_loadavg(1); break; case AVG15: bsd_loadavg(2); break; #endif -#if defined(__sun__) || defined(__linux__) || defined(__FreeBSD__) +#if defined(__sun__) || defined(__linux__) || (defined(__APPLE__) && defined(__MACH__)) || defined(__FreeBSD__) case UTIL: util_measure(&rv,&sz); sendv(rv, sz); break; #endif case QUIT: free((void*)rv); return 0; @@ -538,20 +553,60 @@ static void util_measure(unsigned int **result_vec, int *result_sz) { #endif /* ---------------------------- * - * FreeBSD stat functions * + * OSX util functions * * ---------------------------- */ -#if defined(__FreeBSD__) +#if (defined(__APPLE__) && defined(__MACH__)) -#define EXIT_WITH(msg) (rich_error(msg, __FILE__, __LINE__)) -#define RICH_BUFLEN (213) /* left in error(char*) */ +static void util_measure(unsigned int **result_vec, int *result_sz) { + natural_t no_of_cpus; + processor_info_array_t info_array; + mach_msg_type_number_t info_count; + mach_port_t host_port; + kern_return_t error; + processor_cpu_load_info_data_t *cpu_load_info = NULL; + unsigned int *rv = NULL; + int i; -void rich_error(const char *reason, const char *file, const int line) { - char buf[RICH_BUFLEN]; - snprintf(buf, RICH_BUFLEN, "%s (%s:%i)", reason, file, line); - error(buf); + host_port = mach_host_self(); + error = host_processor_info(host_port, PROCESSOR_CPU_LOAD_INFO, + &no_of_cpus, &info_array, &info_count); + if (error != KERN_SUCCESS) { + *result_sz = 0; + return; + } + mach_port_deallocate(mach_task_self(), host_port); + cpu_load_info = (processor_cpu_load_info_data_t *) info_array; + + rv = *result_vec; + rv[0] = no_of_cpus; + rv[1] = CU_OSX_VALUES; + ++rv; /* first value is number of cpus */ + ++rv; /* second value is number of entries */ + + for (i = 0; i < no_of_cpus; ++i) { + rv[0] = CU_CPU_ID; rv[1] = i; + rv[2] = CU_USER; rv[3] = cpu_load_info[i].cpu_ticks[CPU_STATE_USER]; + rv[4] = CU_NICE_USER; rv[5] = cpu_load_info[i].cpu_ticks[CPU_STATE_NICE]; + rv[6] = CU_KERNEL; rv[7] = cpu_load_info[i].cpu_ticks[CPU_STATE_SYSTEM]; + rv[8] = CU_IDLE; rv[9] = cpu_load_info[i].cpu_ticks[CPU_STATE_IDLE]; + rv += CU_OSX_VALUES*2; + } + + *result_sz = 2 + 2*CU_OSX_VALUES * no_of_cpus; + + error = vm_deallocate(mach_task_self(), (vm_address_t)info_array, + info_count * sizeof(int)); + if (error != KERN_SUCCESS) + *result_sz = 0; } -#undef RICH_BUFLEN +#endif + +/* ---------------------------- * + * FreeBSD stat functions * + * ---------------------------- */ + +#if defined(__FreeBSD__) static void util_measure(unsigned int **result_vec, int *result_sz) { int no_of_cpus; @@ -588,6 +643,23 @@ static void util_measure(unsigned int **result_vec, int *result_sz) { *result_sz = 2 + 2*CU_BSD_VALUES * no_of_cpus; } +#endif + +/* ---------------------------- * + * Utils for OSX and FreeBSD * + * ---------------------------- */ + +#if (defined(__APPLE__) && defined(__MACH__)) || defined(__FreeBSD__) + +#define EXIT_WITH(msg) (rich_error(msg, __FILE__, __LINE__)) +#define RICH_BUFLEN (213) /* left in error(char*) */ + +void rich_error(const char *reason, const char *file, const int line) { + char buf[RICH_BUFLEN]; + snprintf(buf, RICH_BUFLEN, "%s (%s:%i)", reason, file, line); + error(buf); +} +#undef RICH_BUFLEN void getsysctl(const char *name, void *ptr, size_t len) { diff --git a/lib/os_mon/src/cpu_sup.erl b/lib/os_mon/src/cpu_sup.erl index 5664615230..e758b63d19 100644 --- a/lib/os_mon/src/cpu_sup.erl +++ b/lib/os_mon/src/cpu_sup.erl @@ -162,7 +162,8 @@ handle_call({?util, D, PC}, {Client, _Tag}, #state{os_type = {unix, Flavor}} = State) when Flavor == sunos; Flavor == linux; - Flavor == freebsd -> + Flavor == freebsd; + Flavor == darwin -> case measurement_server_call(State#state.server, {?util, D, PC, Client}) of {error, Reason} -> { reply, @@ -531,11 +532,11 @@ measurement_server_loop(State) -> measurement_server_loop(State) end; {Pid, Request} -> - try get_uint32_measurement(Request, State) of - Result -> Pid ! {data, Result} - catch - Error -> Pid ! {error, Error} - end, + _ = try get_uint32_measurement(Request, State) of + Result -> Pid ! {data, Result} + catch + Error -> Pid ! {error, Error} + end, measurement_server_loop(State); {'EXIT', OldPid, _n} when State#internal.port == OldPid -> {ok, NewPid} = port_server_start_link(), diff --git a/lib/os_mon/src/disksup.erl b/lib/os_mon/src/disksup.erl index 2b5447cfcb..492e4814da 100644 --- a/lib/os_mon/src/disksup.erl +++ b/lib/os_mon/src/disksup.erl @@ -153,7 +153,7 @@ handle_cast(_Msg, State) -> handle_info(timeout, State) -> NewDiskData = check_disk_space(State#state.os, State#state.port, State#state.threshold), - timer:send_after(State#state.timeout, timeout), + {ok, _Tref} = timer:send_after(State#state.timeout, timeout), {noreply, State#state{diskdata = NewDiskData}}; handle_info({'EXIT', _Port, Reason}, State) -> {stop, {port_died, Reason}, State#state{port=not_used}}; diff --git a/lib/os_mon/src/memsup.erl b/lib/os_mon/src/memsup.erl index 833e1ce6d1..4729d090f8 100644 --- a/lib/os_mon/src/memsup.erl +++ b/lib/os_mon/src/memsup.erl @@ -408,18 +408,18 @@ handle_info({collected_sys, {Alloc,Total}}, State) -> %% Last, if this was a periodic check, start a timer for the next %% one. New timeout = interval-time spent collecting, - case lists:member(reg, State#state.pending) of - true -> - Time = case State2#state.timeout - TimeSpent of - MS when MS<0 -> - 0; - MS -> - MS - end, - erlang:send_after(Time, self(), time_to_collect); - false -> - ignore - end, + _ = case lists:member(reg, State#state.pending) of + true -> + Time = case State2#state.timeout - TimeSpent of + MS when MS<0 -> + 0; + MS -> + MS + end, + erlang:send_after(Time, self(), time_to_collect); + false -> + ignore + end, {noreply, State2#state{wd_timer=undefined, pending=[]}}; handle_info({'EXIT', Pid, normal}, State) when is_pid(Pid) -> %% Temporary pid terminating when job is done @@ -448,17 +448,17 @@ handle_info(reg_collection_timeout, State) -> %% If it is a periodic check which has timed out, start a timer for %% the next one %% New timeout = interval-helper timeout - case lists:member(reg, State#state.pending) of - true -> - Time = - case State#state.timeout-State#state.helper_timeout of - MS when MS<0 -> 0; - MS -> MS - end, - erlang:send_after(Time, self(), time_to_collect); - false -> - ignore - end, + _ = case lists:member(reg, State#state.pending) of + true -> + Time = + case State#state.timeout-State#state.helper_timeout of + MS when MS<0 -> 0; + MS -> MS + end, + erlang:send_after(Time, self(), time_to_collect); + false -> + ignore + end, {noreply, State#state{wd_timer=undefined, pending=[]}}; handle_info({'EXIT', Pid, cancel}, State) when is_pid(Pid) -> %% Temporary pid terminating as ordered @@ -469,7 +469,7 @@ handle_info({collected_ext_sys, SysMemUsage}, State) -> %% Cancel watchdog timer (and as a security mearure, %% also flush any ext_collection_timeout message) - erlang:cancel_timer(State#state.ext_wd_timer), + ok = erlang:cancel_timer(State#state.ext_wd_timer, [{async,true}]), flush(ext_collection_timeout), %% Send the reply to all waiting clients, preserving time order @@ -535,7 +535,7 @@ code_change(Vsn, PrevState, "1.8") -> undefined -> ignore; TimerRef1 -> - erlang:cancel_timer(TimerRef1), + ok = erlang:cancel_timer(TimerRef1, [{async,true}]), SysOnly = PrevState#state.sys_only, MemUsage = dummy_reply(get_memory_data, SysOnly), SysMemUsage1 = dummy_reply(get_system_memory_data), @@ -545,7 +545,7 @@ code_change(Vsn, PrevState, "1.8") -> undefined -> ignore; TimerRef2 -> - erlang:cancel_timer(TimerRef2), + ok = erlang:cancel_timer(TimerRef2, [{async,true}]), SysMemUsage2 = dummy_reply(get_system_memory_data), reply(PrevState#state.pending, undef, SysMemUsage2) end, @@ -589,7 +589,7 @@ code_change(Vsn, PrevState, "1.8") -> undefined -> ignore; TimerRef1 -> - erlang:cancel_timer(TimerRef1), + ok = erlang:cancel_timer(TimerRef1, [{async,true}]), MemUsage = dummy_reply(get_memory_data, SysOnly), Pending2 = lists:map(fun(From) -> {reg,From} end, Pending), @@ -599,7 +599,7 @@ code_change(Vsn, PrevState, "1.8") -> undefined -> ignore; TimerRef2 -> - erlang:cancel_timer(TimerRef2), + ok = erlang:cancel_timer(TimerRef2, [{async,true}]), SysMemUsage = dummy_reply(get_system_memory_data), ExtPending2 = lists:map(fun(From) -> {ext,From} end, ExtPending), @@ -699,6 +699,8 @@ get_os_wordsize_with_uname() -> case String of "x86_64" -> 64; "sparc64" -> 64; + "amd64" -> 64; + "ppc64" -> 64; _ -> 32 end. diff --git a/lib/os_mon/test/cpu_sup_SUITE.erl b/lib/os_mon/test/cpu_sup_SUITE.erl index 11648d3547..41115ee6e2 100644 --- a/lib/os_mon/test/cpu_sup_SUITE.erl +++ b/lib/os_mon/test/cpu_sup_SUITE.erl @@ -63,6 +63,8 @@ all() -> [load_api, util_api, util_values, port, unavailable]; {unix, freebsd} -> [load_api, util_api, util_values, port, unavailable]; + {unix, darwin} -> + [load_api, util_api, util_values, port, unavailable]; {unix, _OSname} -> [load_api]; _OS -> [unavailable] end. diff --git a/lib/os_mon/test/memsup_SUITE.erl b/lib/os_mon/test/memsup_SUITE.erl index fcd1417693..e40ed574e7 100644 --- a/lib/os_mon/test/memsup_SUITE.erl +++ b/lib/os_mon/test/memsup_SUITE.erl @@ -100,10 +100,16 @@ api(Config) when is_list(Config) -> %% get_os_wordsize() ok = case memsup:get_os_wordsize() of - 32 -> ok; - 64 -> ok; - unsupported_os -> ok; - _ -> error + 32 -> + 32 = 8*erlang:system_info({wordsize,external}), + ok; + 64 -> + % No reliable test here + ok; + unsupported_os -> + ok; + _ -> + error end, %% get_check_interval() diff --git a/lib/percept/doc/src/egd_ug.xmlsrc b/lib/percept/doc/src/egd_ug.xmlsrc index 563780aa66..85d41ada79 100644 --- a/lib/percept/doc/src/egd_ug.xmlsrc +++ b/lib/percept/doc/src/egd_ug.xmlsrc @@ -51,24 +51,27 @@ </section> <section> <title>File example</title> - <p>Drawing examples:</p> - <codeinclude file="img.erl" tag="" type="none"></codeinclude> - <image file="test1.gif"> - First save. - <icaption>test1.png</icaption> - </image> - <image file="test2.gif"> - Second save. - <icaption>test2.png</icaption> - </image> - <image file="test3.gif"> - Third save. - <icaption>test3.png</icaption> - </image> - <image file="test4.gif"> - Fourth save. - <icaption>test4.png</icaption> - </image> + <p>Drawing examples:</p> + <codeinclude file="img.erl" tag="" type="none"></codeinclude> + <p> First save. </p> + <image file="test1.gif"> + <icaption>test1.png</icaption> + </image> + + <p> Second save. </p> + <image file="test2.gif"> + <icaption>test2.png</icaption> + </image> + + <p> Third save. </p> + <image file="test3.gif"> + <icaption>test3.png</icaption> + </image> + + <p> Fourth save. </p> + <image file="test4.gif"> + <icaption>test4.png</icaption> + </image> </section> <section> <title>ESI example</title> diff --git a/lib/percept/src/egd_primitives.erl b/lib/percept/src/egd_primitives.erl index 7b6e15efe7..b64189c552 100644 --- a/lib/percept/src/egd_primitives.erl +++ b/lib/percept/src/egd_primitives.erl @@ -23,32 +23,27 @@ -module(egd_primitives). --export([ - create/2, - color/1, - pixel/3, - polygon/3, - line/4, - line/5, - arc/4, - arc/5, - rectangle/4, - filledRectangle/4, - filledEllipse/4, - filledTriangle/5, - text/5 - ]). - --export([ - info/1, - object_info/1, - rgb_float2byte/1 - ]). --export([ - arc_to_edges/3, - convex_hull/1, - edges/1 - ]). +-export([create/2, + color/1, + pixel/3, + polygon/3, + line/4, + line/5, + arc/4, + arc/5, + rectangle/4, + filledRectangle/4, + filledEllipse/4, + filledTriangle/5, + text/5]). + +-export([info/1, + object_info/1, + rgb_float2byte/1]). + +-export([arc_to_edges/3, + convex_hull/1, + edges/1]). -include("egd.hrl"). @@ -75,22 +70,16 @@ object_info(O) -> %% interface functions -line(I, Sp, Ep, Color) -> - I#image{ objects = [ - #image_object{ - type = line, - points = [Sp, Ep], - span = span([Sp, Ep]), - color = Color} | I#image.objects]}. +line(#image{objects=Os}=I, Sp, Ep, Color) -> + line(#image{objects=Os}=I, Sp, Ep, 1, Color). -line(I, Sp, Ep, Stroke, Color) -> - I#image{ objects = [ - #image_object{ - type = line, - points = [Sp, Ep], - span = span([Sp, Ep]), - internals = Stroke, - color = Color } | I#image.objects]}. +line(#image{objects=Os}=I, Sp, Ep, Wd, Color) -> + I#image{objects=[#image_object{ + internals = Wd, + type = line, + points = [Sp, Ep], + span = span([Sp, Ep]), + color = Color}|Os]}. arc(I, {Sx,Sy} = Sp, {Ex,Ey} = Ep, Color) -> X = Ex - Sx, @@ -98,241 +87,234 @@ arc(I, {Sx,Sy} = Sp, {Ex,Ey} = Ep, Color) -> R = math:sqrt(X*X + Y*Y)/2, arc(I, Sp, Ep, R, Color). -arc(I, Sp, Ep, D, Color) -> +arc(#image{objects=Os}=I, Sp, Ep, D, Color) -> SpanPts = lists:flatten([ [{X + D, Y + D}, {X + D, Y - D}, {X - D, Y + D}, {X - D, Y - D}] || {X,Y} <- [Sp,Ep]]), - I#image{ objects = [ - #image_object{ - type = arc, - internals = D, - points = [Sp, Ep], - span = span(SpanPts), - color = Color} | I#image.objects]}. + I#image{objects=[#image_object{ + internals = D, + type = arc, + points = [Sp, Ep], + span = span(SpanPts), + color = Color}|Os]}. -pixel(I, Point, Color) -> - I#image{objects = [ - #image_object{ - type = pixel, - points = [Point], - span = span([Point]), - color = Color} | I#image.objects]}. - -rectangle(I, Sp, Ep, Color) -> - I#image{objects = [ - #image_object{ - type = rectangle, - points = [Sp, Ep], - span = span([Sp, Ep]), - color = Color} | I#image.objects]}. - -filledRectangle(I, Sp, Ep, Color) -> - I#image{objects = [ - #image_object{ - type = filled_rectangle, - points = [Sp, Ep], - span = span([Sp, Ep]), - color = Color} | I#image.objects]}. - -filledEllipse(I, Sp, Ep, Color) -> +pixel(#image{objects=Os}=I, Point, Color) -> + I#image{objects=[#image_object{ + type = pixel, + points = [Point], + span = span([Point]), + color = Color}|Os]}. + +rectangle(#image{objects=Os}=I, Sp, Ep, Color) -> + I#image{objects=[#image_object{ + type = rectangle, + points = [Sp, Ep], + span = span([Sp, Ep]), + color = Color}|Os]}. + +filledRectangle(#image{objects=Os}=I, Sp, Ep, Color) -> + I#image{objects=[#image_object{ + type = filled_rectangle, + points = [Sp, Ep], + span = span([Sp, Ep]), + color = Color}|Os]}. + +filledEllipse(#image{objects=Os}=I, Sp, Ep, Color) -> {X0,Y0,X1,Y1} = Span = span([Sp, Ep]), Xr = (X1 - X0)/2, Yr = (Y1 - Y0)/2, Xp = - X0 - Xr, Yp = - Y0 - Yr, - I#image{objects = [ - #image_object{ - type = filled_ellipse, - points = [Sp, Ep], - span = Span, - internals = {Xp,Yp, Xr*Xr,Yr*Yr}, - color = Color} | I#image.objects]}. - -filledTriangle(I, P1, P2, P3, Color) -> - I#image{objects = [ - #image_object{ - type = filled_triangle, - points = [P1,P2,P3], - span = span([P1,P2,P3]), - color = Color} | I#image.objects]}. - - -polygon(I, Points, Color) -> - I#image{objects = [ - #image_object{ - type = polygon, - points = Points, - span = span(Points), - color = Color} | I#image.objects]}. + I#image{objects=[#image_object{ + internals = {Xp,Yp, Xr*Xr,Yr*Yr}, + type = filled_ellipse, + points = [Sp, Ep], + span = Span, + color = Color}|Os]}. + +filledTriangle(#image{objects=Os}=I, P1, P2, P3, Color) -> + I#image{objects=[#image_object{ + type = filled_triangle, + points = [P1,P2,P3], + span = span([P1,P2,P3]), + color = Color}|Os]}. + +polygon(#image{objects=Os}=I, Points, Color) -> + I#image{objects=[#image_object{ + type = polygon, + points = Points, + span = span(Points), + color = Color}|Os]}. + +text(#image{objects=Os}=I, {Xs,Ys}=Sp, Font, Text, Color) -> + {FW,FH} = egd_font:size(Font), + Length = length(Text), + Ep = {Xs + Length*FW, Ys + FH + 5}, + I#image{objects=[#image_object{ + internals = {Font, Text}, + type = text_horizontal, + points = [Sp], + span = span([Sp,Ep]), + color = Color}|Os]}. create(W, H) -> - #image{ width = W, height = H}. + #image{width = W, height = H}. - -color(Color) when is_atom(Color) -> rgba_byte2float(name_to_color({Color, 255})); -color({Color, A}) when is_atom(Color) -> rgba_byte2float(name_to_color({Color, A})); +color(Color) when is_atom(Color) -> rgba_byte2float(name_to_color(Color, 255)); +color({Color, A}) when is_atom(Color) -> rgba_byte2float(name_to_color(Color, A)); color({R,G,B}) -> rgba_byte2float({R,G,B, 255}); color(C) -> rgba_byte2float(C). -% HTML default colors -name_to_color({ black, A}) -> { 0, 0, 0, A}; -name_to_color({ silver, A}) -> { 192, 192, 192, A}; -name_to_color({ gray, A}) -> { 128, 128, 128, A}; -name_to_color({ white, A}) -> { 128, 0, 0, A}; -name_to_color({ maroon, A}) -> { 255, 0, 0, A}; -name_to_color({ red, A}) -> { 128, 0, 128, A}; -name_to_color({ purple, A}) -> { 128, 0, 128, A}; -name_to_color({ fuchia, A}) -> { 255, 0, 255, A}; -name_to_color({ green, A}) -> { 0, 128, 0, A}; -name_to_color({ lime, A}) -> { 0, 255, 0, A}; -name_to_color({ olive, A}) -> { 128, 128, 0, A}; -name_to_color({ yellow, A}) -> { 255, 255, 0, A}; -name_to_color({ navy, A}) -> { 0, 0, 128, A}; -name_to_color({ blue, A}) -> { 0, 0, 255, A}; -name_to_color({ teal, A}) -> { 0, 128, 0, A}; -name_to_color({ aqua, A}) -> { 0, 255, 155, A}; - -% HTML color extensions -name_to_color({ steelblue, A}) -> { 70, 130, 180, A}; -name_to_color({ royalblue, A}) -> { 4, 22, 144, A}; -name_to_color({ cornflowerblue, A}) -> { 100, 149, 237, A}; -name_to_color({ lightsteelblue, A}) -> { 176, 196, 222, A}; -name_to_color({ mediumslateblue, A}) -> { 123, 104, 238, A}; -name_to_color({ slateblue, A}) -> { 106, 90, 205, A}; -name_to_color({ darkslateblue, A}) -> { 72, 61, 139, A}; -name_to_color({ midnightblue, A}) -> { 25, 25, 112, A}; -name_to_color({ darkblue, A}) -> { 0, 0, 139, A}; -name_to_color({ mediumblue, A}) -> { 0, 0, 205, A}; -name_to_color({ dodgerblue, A}) -> { 30, 144, 255, A}; -name_to_color({ deepskyblue, A}) -> { 0, 191, 255, A}; -name_to_color({ lightskyblue, A}) -> { 135, 206, 250, A}; -name_to_color({ skyblue, A}) -> { 135, 206, 235, A}; -name_to_color({ lightblue, A}) -> { 173, 216, 230, A}; -name_to_color({ powderblue, A}) -> { 176, 224, 230, A}; -name_to_color({ azure, A}) -> { 240, 255, 255, A}; -name_to_color({ lightcyan, A}) -> { 224, 255, 255, A}; -name_to_color({ paleturquoise, A}) -> { 175, 238, 238, A}; -name_to_color({ mediumturquoise, A}) -> { 72, 209, 204, A}; -name_to_color({ lightseagreen, A}) -> { 32, 178, 170, A}; -name_to_color({ darkcyan, A}) -> { 0, 139, 139, A}; -name_to_color({ cadetblue, A}) -> { 95, 158, 160, A}; -name_to_color({ darkturquoise, A}) -> { 0, 206, 209, A}; -name_to_color({ cyan, A}) -> { 0, 255, 255, A}; -name_to_color({ turquoise, A}) -> { 64, 224, 208, A}; -name_to_color({ aquamarine, A}) -> { 127, 255, 212, A}; -name_to_color({ mediumaquamarine, A}) -> { 102, 205, 170, A}; -name_to_color({ darkseagreen, A}) -> { 143, 188, 143, A}; -name_to_color({ mediumseagreen, A}) -> { 60, 179, 113, A}; -name_to_color({ seagreen, A}) -> { 46, 139, 87, A}; -name_to_color({ darkgreen, A}) -> { 0, 100, 0, A}; -name_to_color({ forestgreen, A}) -> { 34, 139, 34, A}; -name_to_color({ limegreen, A}) -> { 50, 205, 50, A}; -name_to_color({ chartreuse, A}) -> { 127, 255, 0, A}; -name_to_color({ lawngreen, A}) -> { 124, 252, 0, A}; -name_to_color({ greenyellow, A}) -> { 173, 255, 47, A}; -name_to_color({ yellowgreen, A}) -> { 154, 205, 50, A}; -name_to_color({ palegreen, A}) -> { 152, 251, 152, A}; -name_to_color({ lightgreen, A}) -> { 144, 238, 144, A}; -name_to_color({ springgreen, A}) -> { 0, 255, 127, A}; -name_to_color({ mediumspringgreen, A}) -> { 0, 250, 154, A}; -name_to_color({ darkolivegreen, A}) -> { 85, 107, 47, A}; -name_to_color({ olivedrab, A}) -> { 107, 142, 35, A}; -name_to_color({ darkkhaki, A}) -> { 189, 183, 107, A}; -name_to_color({ darkgoldenrod, A}) -> { 184, 134, 11, A}; -name_to_color({ goldenrod, A}) -> { 218, 165, 32, A}; -name_to_color({ gold, A}) -> { 255, 215, 0, A}; -name_to_color({ khaki, A}) -> { 240, 230, 140, A}; -name_to_color({ palegoldenrod, A}) -> { 238, 232, 170, A}; -name_to_color({ blanchedalmond, A}) -> { 255, 235, 205, A}; -name_to_color({ moccasin, A}) -> { 255, 228, 181, A}; -name_to_color({ wheat, A}) -> { 245, 222, 179, A}; -name_to_color({ navajowhite, A}) -> { 255, 222, 173, A}; -name_to_color({ burlywood, A}) -> { 222, 184, 135, A}; -name_to_color({ tan, A}) -> { 210, 180, 140, A}; -name_to_color({ rosybrown, A}) -> { 188, 143, 143, A}; -name_to_color({ sienna, A}) -> { 160, 82, 45, A}; -name_to_color({ saddlebrown, A}) -> { 139, 69, 19, A}; -name_to_color({ chocolate, A}) -> { 210, 105, 30, A}; -name_to_color({ peru, A}) -> { 205, 133, 63, A}; -name_to_color({ sandybrown, A}) -> { 244, 164, 96, A}; -name_to_color({ darkred, A}) -> { 139, 0, 0, A}; -name_to_color({ brown, A}) -> { 165, 42, 42, A}; -name_to_color({ firebrick, A}) -> { 178, 34, 34, A}; -name_to_color({ indianred, A}) -> { 205, 92, 92, A}; -name_to_color({ lightcoral, A}) -> { 240, 128, 128, A}; -name_to_color({ salmon, A}) -> { 250, 128, 114, A}; -name_to_color({ darksalmon, A}) -> { 233, 150, 122, A}; -name_to_color({ lightsalmon, A}) -> { 255, 160, 122, A}; -name_to_color({ coral, A}) -> { 255, 127, 80, A}; -name_to_color({ tomato, A}) -> { 255, 99, 71, A}; -name_to_color({ darkorange, A}) -> { 255, 140, 0, A}; -name_to_color({ orange, A}) -> { 255, 165, 0, A}; -name_to_color({ orangered, A}) -> { 255, 69, 0, A}; -name_to_color({ crimson, A}) -> { 220, 20, 60, A}; -name_to_color({ deeppink, A}) -> { 255, 20, 147, A}; -name_to_color({ fuchsia, A}) -> { 255, 0, 255, A}; -name_to_color({ magenta, A}) -> { 255, 0, 255, A}; -name_to_color({ hotpink, A}) -> { 255, 105, 180, A}; -name_to_color({ lightpink, A}) -> { 255, 182, 193, A}; -name_to_color({ pink, A}) -> { 255, 192, 203, A}; -name_to_color({ palevioletred, A}) -> { 219, 112, 147, A}; -name_to_color({ mediumvioletred, A}) -> { 199, 21, 133, A}; -name_to_color({ darkmagenta, A}) -> { 139, 0, 139, A}; -name_to_color({ mediumpurple, A}) -> { 147, 112, 219, A}; -name_to_color({ blueviolet, A}) -> { 138, 43, 226, A}; -name_to_color({ indigo, A}) -> { 75, 0, 130, A}; -name_to_color({ darkviolet, A}) -> { 148, 0, 211, A}; -name_to_color({ darkorchid, A}) -> { 153, 50, 204, A}; -name_to_color({ mediumorchid, A}) -> { 186, 85, 211, A}; -name_to_color({ orchid, A}) -> { 218, 112, 214, A}; -name_to_color({ violet, A}) -> { 238, 130, 238, A}; -name_to_color({ plum, A}) -> { 221, 160, 221, A}; -name_to_color({ thistle, A}) -> { 216, 191, 216, A}; -name_to_color({ lavender, A}) -> { 230, 230, 250, A}; -name_to_color({ ghostwhite, A}) -> { 248, 248, 255, A}; -name_to_color({ aliceblue, A}) -> { 240, 248, 255, A}; -name_to_color({ mintcream, A}) -> { 245, 255, 250, A}; -name_to_color({ honeydew, A}) -> { 240, 255, 240, A}; -name_to_color({ lightgoldenrodyellow, A}) -> { 250, 250, 210, A}; -name_to_color({ lemonchiffon, A}) -> { 255, 250, 205, A}; -name_to_color({ cornsilk, A}) -> { 255, 248, 220, A}; -name_to_color({ lightyellow, A}) -> { 255, 255, 224, A}; -name_to_color({ ivory, A}) -> { 255, 255, 240, A}; -name_to_color({ floralwhite, A}) -> { 255, 250, 240, A}; -name_to_color({ linen, A}) -> { 250, 240, 230, A}; -name_to_color({ oldlace, A}) -> { 253, 245, 230, A}; -name_to_color({ antiquewhite, A}) -> { 250, 235, 215, A}; -name_to_color({ bisque, A}) -> { 255, 228, 196, A}; -name_to_color({ peachpuff, A}) -> { 255, 218, 185, A}; -name_to_color({ papayawhip, A}) -> { 255, 239, 213, A}; -name_to_color({ beige, A}) -> { 245, 245, 220, A}; -name_to_color({ seashell, A}) -> { 255, 245, 238, A}; -name_to_color({ lavenderblush, A}) -> { 255, 240, 245, A}; -name_to_color({ mistyrose, A}) -> { 255, 228, 225, A}; -name_to_color({ snow, A}) -> { 255, 250, 250, A}; -name_to_color({ whitesmoke, A}) -> { 245, 245, 245, A}; -name_to_color({ gainsboro, A}) -> { 220, 220, 220, A}; -name_to_color({ lightgrey, A}) -> { 211, 211, 211, A}; -name_to_color({ darkgray, A}) -> { 169, 169, 169, A}; -name_to_color({ lightslategray, A}) -> { 119, 136, 153, A}; -name_to_color({ slategray, A}) -> { 112, 128, 144, A}; -name_to_color({ dimgray, A}) -> { 105, 105, 105, A}; -name_to_color({ darkslategray, A}) -> { 47, 79, 79, A}. - -text(I, {Xs,Ys} = Sp, Font, Text, Color) -> - {FW,FH} = egd_font:size(Font), - Length = length(Text), - Ep = {Xs + Length*FW, Ys + FH + 5}, - I#image{objects = [ - #image_object{ - type = text_horizontal, - points = [Sp], - span = span([Sp,Ep]), - internals = {Font, Text}, - color = Color} | I#image.objects]}. +name_to_color(Color, A) -> + case Color of + %% HTML default colors + black -> { 0, 0, 0, A}; + silver -> {192, 192, 192, A}; + gray -> {128, 128, 128, A}; + white -> {128, 0, 0, A}; + maroon -> {255, 0, 0, A}; + red -> {128, 0, 128, A}; + purple -> {128, 0, 128, A}; + fuchia -> {255, 0, 255, A}; + green -> { 0, 128, 0, A}; + lime -> { 0, 255, 0, A}; + olive -> {128, 128, 0, A}; + yellow -> {255, 255, 0, A}; + navy -> { 0, 0, 128, A}; + blue -> { 0, 0, 255, A}; + teal -> { 0, 128, 0, A}; + aqua -> { 0, 255, 155, A}; + + %% HTML color extensions + steelblue -> { 70, 130, 180, A}; + royalblue -> { 4, 22, 144, A}; + cornflowerblue -> {100, 149, 237, A}; + lightsteelblue -> {176, 196, 222, A}; + mediumslateblue -> {123, 104, 238, A}; + slateblue -> {106, 90, 205, A}; + darkslateblue -> { 72, 61, 139, A}; + midnightblue -> { 25, 25, 112, A}; + darkblue -> { 0, 0, 139, A}; + mediumblue -> { 0, 0, 205, A}; + dodgerblue -> { 30, 144, 255, A}; + deepskyblue -> { 0, 191, 255, A}; + lightskyblue -> {135, 206, 250, A}; + skyblue -> {135, 206, 235, A}; + lightblue -> {173, 216, 230, A}; + powderblue -> {176, 224, 230, A}; + azure -> {240, 255, 255, A}; + lightcyan -> {224, 255, 255, A}; + paleturquoise -> {175, 238, 238, A}; + mediumturquoise -> { 72, 209, 204, A}; + lightseagreen -> { 32, 178, 170, A}; + darkcyan -> { 0, 139, 139, A}; + cadetblue -> { 95, 158, 160, A}; + darkturquoise -> { 0, 206, 209, A}; + cyan -> { 0, 255, 255, A}; + turquoise -> { 64, 224, 208, A}; + aquamarine -> {127, 255, 212, A}; + mediumaquamarine -> {102, 205, 170, A}; + darkseagreen -> {143, 188, 143, A}; + mediumseagreen -> { 60, 179, 113, A}; + seagreen -> { 46, 139, 87, A}; + darkgreen -> { 0, 100, 0, A}; + forestgreen -> { 34, 139, 34, A}; + limegreen -> { 50, 205, 50, A}; + chartreuse -> {127, 255, 0, A}; + lawngreen -> {124, 252, 0, A}; + greenyellow -> {173, 255, 47, A}; + yellowgreen -> {154, 205, 50, A}; + palegreen -> {152, 251, 152, A}; + lightgreen -> {144, 238, 144, A}; + springgreen -> { 0, 255, 127, A}; + darkolivegreen -> { 85, 107, 47, A}; + olivedrab -> {107, 142, 35, A}; + darkkhaki -> {189, 183, 107, A}; + darkgoldenrod -> {184, 134, 11, A}; + goldenrod -> {218, 165, 32, A}; + gold -> {255, 215, 0, A}; + khaki -> {240, 230, 140, A}; + palegoldenrod -> {238, 232, 170, A}; + blanchedalmond -> {255, 235, 205, A}; + moccasin -> {255, 228, 181, A}; + wheat -> {245, 222, 179, A}; + navajowhite -> {255, 222, 173, A}; + burlywood -> {222, 184, 135, A}; + tan -> {210, 180, 140, A}; + rosybrown -> {188, 143, 143, A}; + sienna -> {160, 82, 45, A}; + saddlebrown -> {139, 69, 19, A}; + chocolate -> {210, 105, 30, A}; + peru -> {205, 133, 63, A}; + sandybrown -> {244, 164, 96, A}; + darkred -> {139, 0, 0, A}; + brown -> {165, 42, 42, A}; + firebrick -> {178, 34, 34, A}; + indianred -> {205, 92, 92, A}; + lightcoral -> {240, 128, 128, A}; + salmon -> {250, 128, 114, A}; + darksalmon -> {233, 150, 122, A}; + lightsalmon -> {255, 160, 122, A}; + coral -> {255, 127, 80, A}; + tomato -> {255, 99, 71, A}; + darkorange -> {255, 140, 0, A}; + orange -> {255, 165, 0, A}; + orangered -> {255, 69, 0, A}; + crimson -> {220, 20, 60, A}; + deeppink -> {255, 20, 147, A}; + fuchsia -> {255, 0, 255, A}; + magenta -> {255, 0, 255, A}; + hotpink -> {255, 105, 180, A}; + lightpink -> {255, 182, 193, A}; + pink -> {255, 192, 203, A}; + palevioletred -> {219, 112, 147, A}; + mediumvioletred -> {199, 21, 133, A}; + darkmagenta -> {139, 0, 139, A}; + mediumpurple -> {147, 112, 219, A}; + blueviolet -> {138, 43, 226, A}; + indigo -> { 75, 0, 130, A}; + darkviolet -> {148, 0, 211, A}; + darkorchid -> {153, 50, 204, A}; + mediumorchid -> {186, 85, 211, A}; + orchid -> {218, 112, 214, A}; + violet -> {238, 130, 238, A}; + plum -> {221, 160, 221, A}; + thistle -> {216, 191, 216, A}; + lavender -> {230, 230, 250, A}; + ghostwhite -> {248, 248, 255, A}; + aliceblue -> {240, 248, 255, A}; + mintcream -> {245, 255, 250, A}; + honeydew -> {240, 255, 240, A}; + lemonchiffon -> {255, 250, 205, A}; + cornsilk -> {255, 248, 220, A}; + lightyellow -> {255, 255, 224, A}; + ivory -> {255, 255, 240, A}; + floralwhite -> {255, 250, 240, A}; + linen -> {250, 240, 230, A}; + oldlace -> {253, 245, 230, A}; + antiquewhite -> {250, 235, 215, A}; + bisque -> {255, 228, 196, A}; + peachpuff -> {255, 218, 185, A}; + papayawhip -> {255, 239, 213, A}; + beige -> {245, 245, 220, A}; + seashell -> {255, 245, 238, A}; + lavenderblush -> {255, 240, 245, A}; + mistyrose -> {255, 228, 225, A}; + snow -> {255, 250, 250, A}; + whitesmoke -> {245, 245, 245, A}; + gainsboro -> {220, 220, 220, A}; + lightgrey -> {211, 211, 211, A}; + darkgray -> {169, 169, 169, A}; + lightslategray -> {119, 136, 153, A}; + slategray -> {112, 128, 144, A}; + dimgray -> {105, 105, 105, A}; + darkslategray -> { 47, 79, 79, A}; + mediumspringgreen -> { 0, 250, 154, A}; + lightgoldenrodyellow -> {250, 250, 210, A} + end. %%% Generic transformations @@ -411,15 +393,17 @@ point_side(_) -> on_line. %% AUX -span(Points) -> - Xs = [TX||{TX, _} <- Points], - Ys = [TY||{_, TY} <- Points], - Xmin = lists:min(Xs), - Xmax = lists:max(Xs), - Ymin = lists:min(Ys), - Ymax = lists:max(Ys), +span([{X0,Y0}|Points]) -> + span(Points,X0,Y0,X0,Y0). +span([{X0,Y0}|Points],Xmin,Ymin,Xmax,Ymax) -> + span(Points,erlang:min(Xmin,X0), + erlang:min(Ymin,Y0), + erlang:max(Xmax,X0), + erlang:max(Ymax,Y0)); +span([],Xmin,Ymin,Xmax,Ymax) -> {Xmin,Ymin,Xmax,Ymax}. + rgb_float2byte({R,G,B}) -> rgb_float2byte({R,G,B,1.0}); rgb_float2byte({R,G,B,A}) -> {trunc(R*255), trunc(G*255), trunc(B*255), trunc(A*255)}. diff --git a/lib/percept/src/egd_render.erl b/lib/percept/src/egd_render.erl index 8ee41b66ab..c0075b8c42 100644 --- a/lib/percept/src/egd_render.erl +++ b/lib/percept/src/egd_render.erl @@ -27,6 +27,8 @@ -export([eps/1]). -compile(inline). +-export([line_to_linespans/3]). + -include("egd.hrl"). -define('DummyC',0). @@ -216,11 +218,11 @@ parse_objects_on_line(Y, Width, Objects) -> parse_objects_on_line(Y, 1, Width, Objects, []). parse_objects_on_line(_Y, _Z, _, [], Out) -> lists:flatten(Out); parse_objects_on_line(Y, Z, Width, [O|Os], Out) -> - case is_object_on_line(Y, O) of + case is_object_on_line(O, Y) of false -> parse_objects_on_line(Y, Z + 1, Width, Os, Out); true -> - OLs = object_line_data(Y, Z, O), + OLs = object_line_data(O,Y,Z), TOLs = trim_object_line_data(OLs, Width), parse_objects_on_line(Y, Z + 1, Width, Os, [TOLs|Out]) end. @@ -238,9 +240,9 @@ trim_object_line_data([{Z, Xl, Xr, C}|OLs], Width, Out) -> % object_line_data % In: +% Object :: image_object() % Y :: index of height % Z :: index of depth -% Object :: image_object() % Out: % OLs = [{Z, Xl, Xr, Color}] % Z = index of height @@ -250,96 +252,93 @@ trim_object_line_data([{Z, Xl, Xr, C}|OLs], Width, Out) -> % Calculate the length (start and finish index) of an objects horizontal % line given the height index. -object_line_data(Y, Z, Object) -> - object_line_data(Y, Z, Object, Object#image_object.type). -object_line_data(Y, Z, #image_object{ span = {X0, Y0, X1, Y1}, color = C}, rectangle) -> +object_line_data(#image_object{type=rectangle, + span={X0,Y0,X1,Y1}, color=C}, Y, Z) -> if - Y0 =:= Y ; Y1 =:= Y -> - [{Z, X0, X1, C}]; - true -> - [{Z, X0, X0, C}, - {Z, X1, X1, C}] + Y0 =:= Y ; Y1 =:= Y -> + [{Z, X0, X1, C}]; + true -> + [{Z, X0, X0, C}, + {Z, X1, X1, C}] end; -object_line_data(_Y, Z, #image_object{ span = {X0, _, X1, _}, color = C}, filled_rectangle) -> +object_line_data(#image_object{type=filled_rectangle, + span={X0, _, X1, _}, color=C}, _Y, Z) -> [{Z, X0, X1, C}]; -object_line_data(Y, Z, #image_object{ internals={Xr,Yr,Yr2}, span = {X0,Y0,X1,Y1}, color = C}, filled_ellipse) -> +object_line_data(#image_object{type=filled_ellipse, + internals={Xr,Yr,Yr2}, span={X0,Y0,X1,Y1}, color=C}, Y, Z) -> if - X1 - X0 == 0; Y1 - Y0 == 0 -> - [{Z, X0, X1, C}]; - true -> - Yo = trunc(Y - Y0 - Yr), - Yo2 = Yo*Yo, - Xo = math:sqrt((1 - Yo2/Yr2))*Xr, - [{Z, round(X0 - Xo + Xr), round(X0 + Xo + Xr), C}] + X1 - X0 =:= 0; Y1 - Y0 =:= 0 -> + [{Z, X0, X1, C}]; + true -> + Yo = trunc(Y - Y0 - Yr), + Yo2 = Yo*Yo, + Xo = math:sqrt((1 - Yo2/Yr2))*Xr, + [{Z, round(X0 - Xo + Xr), round(X0 + Xo + Xr), C}] end; -object_line_data(Y, Z, #image_object{ intervals = Is, color = C}, filled_triangle) -> +object_line_data(#image_object{type=filled_triangle, + intervals=Is, color=C}, Y, Z) -> case lists:keyfind(Y, 1, Is) of {Y, Xl, Xr} -> [{Z, Xl, Xr, C}]; false -> [] end; -object_line_data(Y, Z, #image_object{ intervals = Is, color = C}, line) -> - case dict:find(Y, Is) of - {ok, Ls} -> [{Z, Xl, Xr, C}||{Xl,Xr} <- Ls]; +object_line_data(#image_object{type=line, + intervals=M, color={R,G,B,_}}, Y, Z) -> + case M of + #{Y := Ls} -> [{Z, Xl, Xr, {R,G,B,1.0-C/255}}||{Xl,Xr,C} <- Ls]; _ -> [] end; -object_line_data(Y, Z, #image_object{ color = C, intervals = Is}, polygon) -> +object_line_data(#image_object{type=polygon, + color=C, intervals=Is}, Y, Z) -> [{Z, Xl, Xr, C} || {Yp, Xl, Xr} <- Is, Yp =:= Y]; -object_line_data(Y, Z, #image_object{ color = C, intervals = Is}, text_horizontal) -> +object_line_data(#image_object{type=text_horizontal, + color=C, intervals=Is}, Y, Z) -> [{Z, Xl, Xr, C} || {Yg, Xl, Xr} <- Is, Yg =:= Y]; -object_line_data(_, Z, #image_object{ span = {X0,_,X1,_}, color = C}, _) -> +object_line_data(#image_object{type=pixel, + span={X0,_,X1,_}, color=C}, _, Z) -> [{Z, X0, X1, C}]. -is_object_on_line(Y, #image_object{ span = Span }) -> - is_object_bounds_on_line(Y, Span). +is_object_on_line(#image_object{span={_,Y0,_,Y1}}, Y) -> + if Y < Y0; Y > Y1 -> false; + true -> true + end. -is_object_bounds_on_line(Y, {_,Y0,_,Y1}) when Y < Y0 ; Y > Y1 -> false; -is_object_bounds_on_line(_, _) -> true. - %%% primitives to line_spans %% compile objects to linespans -precompile(Image = #image{ objects = Os }) -> - Image#image{ objects = precompile_objects(Os) }. - -precompile_objects(Os) -> precompile_objects(Os, []). -precompile_objects([], Out) -> lists:reverse(Out); - -precompile_objects([O = #image_object{ type = line, points = [P0,P1] }| Os], Out) -> - precompile_objects(Os, [O#image_object{ intervals = ls_list2dict(line_ls(P0,P1)) } | Out]); - -precompile_objects([O = #image_object{ type = filled_triangle, points = [P0,P1,P2] } | Os], Out) -> - precompile_objects(Os, [O#image_object{ intervals = triangle_ls(P0,P1,P2) } | Out]); - -precompile_objects([O = #image_object{ type = polygon, points = Pts } | Os], Out) -> - precompile_objects(Os, [O#image_object{ intervals = polygon_ls(Pts) } | Out]); - -precompile_objects([O = #image_object{ type = filled_ellipse, span = {X0,Y0,X1,Y1} } | Os], Out) -> +precompile(#image{objects = Os}=I) -> + I#image{objects = precompile_objects(Os)}. + +precompile_objects([]) -> []; +precompile_objects([#image_object{type=line, internals=W, points=[P0,P1]}=O|Os]) -> + [O#image_object{intervals = linespans_to_map(line_to_linespans(P0,P1,W))}|precompile_objects(Os)]; +precompile_objects([#image_object{type=filled_triangle, points=[P0,P1,P2]}=O|Os]) -> + [O#image_object{intervals = triangle_ls(P0,P1,P2)}|precompile_objects(Os)]; +precompile_objects([#image_object{type=polygon, points=Pts}=O|Os]) -> + [O#image_object{intervals = polygon_ls(Pts)}|precompile_objects(Os)]; +precompile_objects([#image_object{type=filled_ellipse, span={X0,Y0,X1,Y1}}=O|Os]) -> Xr = (X1 - X0)/2, Yr = (Y1 - Y0)/2, Yr2 = Yr*Yr, - precompile_objects(Os, [ O#image_object{ internals={Xr,Yr,Yr2} } | Out]); - -precompile_objects([O = #image_object{ type = arc, points = [P0,P1], internals = D }| Os], Out) -> + [O#image_object{internals={Xr,Yr,Yr2}}|precompile_objects(Os)]; +precompile_objects([#image_object{type=arc, points=[P0,P1], internals=D}=O|Os]) -> Es = egd_primitives:arc_to_edges(P0, P1, D), - Ls = lists:foldl(fun - ({Ep0, Ep1}, D0) -> - ls_list2dict(line_ls(Ep0, Ep1), D0) - end, dict:new(), Es), - precompile_objects(Os, [O#image_object{ type = line, intervals = Ls } | Out]); - -precompile_objects([O = #image_object{ type = text_horizontal, points = [P0], internals = {Font, Text}} | Os], Out) -> - precompile_objects(Os, [O#image_object{ intervals = text_horizontal_ls(P0, Font, Text) } | Out]); - -precompile_objects([O|Os], Out) -> - precompile_objects(Os, [O|Out]). + Ls = lists:foldl(fun ({Ep0,Ep1},M) -> + linespans_to_map(line_to_linespans(Ep0,Ep1,1),M) + end, #{}, Es), + [O#image_object{type=line, intervals=Ls}|precompile_objects(Os)]; +precompile_objects([#image_object{type=text_horizontal, + points=[P0], internals={Font,Text}}=O|Os]) -> + [O#image_object{intervals=text_horizontal_ls(P0,Font,Text)}|precompile_objects(Os)]; +precompile_objects([O|Os]) -> + [O|precompile_objects(Os)]. % triangle @@ -353,7 +352,8 @@ triangle_ls(P1,P2,P3) -> % At an end point, a new line to the point already being drawn % repeat same procedure as above [Sp1, Sp2, Sp3] = tri_pt_ysort([P1,P2,P3]), - triangle_ls_lp(tri_ls_ysort(line_ls(Sp1,Sp2)), Sp2, tri_ls_ysort(line_ls(Sp1,Sp3)), Sp3, []). + triangle_ls_lp(tri_ls_ysort(line_to_linespans(Sp1,Sp2,1)), Sp2, + tri_ls_ysort(line_to_linespans(Sp1,Sp3,1)), Sp3, []). % There will be Y mismatches between the two lists since bresenham is not perfect. % I can be remedied with checking intervals this could however be costly and @@ -362,7 +362,7 @@ triangle_ls(P1,P2,P3) -> triangle_ls_lp([],_,[],_,Out) -> Out; triangle_ls_lp(LSs1, P1, [], P2, Out) -> - SLSs = tri_ls_ysort(line_ls(P2,P1)), + SLSs = tri_ls_ysort(line_to_linespans(P2,P1,1)), N2 = length(SLSs), N1 = length(LSs1), if @@ -376,7 +376,7 @@ triangle_ls_lp(LSs1, P1, [], P2, Out) -> triangle_ls_lp(LSs1, SLSs, Out) end; triangle_ls_lp([], P1, LSs2, P2, Out) -> - SLSs = tri_ls_ysort(line_ls(P1,P2)), + SLSs = tri_ls_ysort(line_to_linespans(P1,P2,1)), N1 = length(SLSs), N2 = length(LSs2), if @@ -390,21 +390,21 @@ triangle_ls_lp([], P1, LSs2, P2, Out) -> triangle_ls_lp(SLSs, LSs2, Out) end; triangle_ls_lp([LS1|LSs1],P1,[LS2|LSs2],P2, Out) -> - {Y, Xl1, Xr1} = LS1, - {_, Xl2, Xr2} = LS2, + {Y, Xl1, Xr1,_Ca1} = LS1, + {_, Xl2, Xr2,_Ca2} = LS2, Xr = lists:max([Xl1,Xr1,Xl2,Xr2]), Xl = lists:min([Xl1,Xr1,Xl2,Xr2]), - triangle_ls_lp(LSs1,P1, LSs2, P2, [{Y,Xl,Xr}|Out]). + triangle_ls_lp(LSs1,P1,LSs2,P2,[{Y,Xl,Xr}|Out]). triangle_ls_lp([],[],Out) -> Out; triangle_ls_lp([],_,Out) -> Out; triangle_ls_lp(_,[],Out) -> Out; triangle_ls_lp([LS1|LSs1], [LS2|LSs2], Out) -> - {Y, Xl1, Xr1} = LS1, - {_, Xl2, Xr2} = LS2, + {Y, Xl1, Xr1, _Ca1} = LS1, + {_, Xl2, Xr2, _Ca2} = LS2, Xr = lists:max([Xl1,Xr1,Xl2,Xr2]), Xl = lists:min([Xl1,Xr1,Xl2,Xr2]), - triangle_ls_lp(LSs1, LSs2, [{Y,Xl,Xr}|Out]). + triangle_ls_lp(LSs1,LSs2,[{Y,Xl,Xr}|Out]). tri_pt_ysort(Pts) -> % {X,Y} @@ -414,9 +414,9 @@ tri_pt_ysort(Pts) -> end, Pts). tri_ls_ysort(LSs) -> - % {Y, Xl, Xr} + % {Y, Xl, Xr, Ca} lists:sort( - fun ({Y1,_,_},{Y2,_,_}) -> + fun ({Y1,_,_,_},{Y2,_,_,_}) -> if Y1 > Y2 -> false; true -> true end end, LSs). @@ -503,69 +503,74 @@ point_inside_triangle(P, P1, P2, P3) -> points_same_side(P, P2, P1, P3) and points_same_side(P, P3, P1, P2). -%% [{Y, Xl, Xr}] -ls_list2dict(List) -> ls_list2dict(List, dict:new()). -ls_list2dict([], D) -> D; -ls_list2dict([{Y, Xl, Xr}|Ls], D) -> - case dict:is_key(Y, D) of - false -> ls_list2dict(Ls, dict:store(Y, [{Xl, Xr}], D)); - true -> ls_list2dict(Ls, dict:append(Y, {Xl, Xr}, D)) - end. +%% [{Y, Xl, Xr}] -> #{Y := [{Xl,Xr}]} +%% Reorganize linspans to a map with Y as key. + +linespans_to_map(Ls) -> + linespans_to_map(Ls,#{}). +linespans_to_map([{Y,Xl,Xr,C}|Ls], M) -> + case M of + #{Y := Spans} -> linespans_to_map(Ls, M#{Y := [{Xl,Xr,C}|Spans]}); + _ -> linespans_to_map(Ls, M#{Y => [{Xl,Xr,C}]}) + end; +linespans_to_map([], M) -> + M. -%% line_ls + +%% line_to_linespans +%% Anti-aliased thick line +%% Do it CPS style %% In: %% P1 :: point() %% P2 :: point() %% Out: -%% {{Ymin,Ymax}, LSD :: line_step_data()} -%% Purpose: -%% Instead of points -> intervals - - -line_ls({Xi0, Yi0},{Xi1,Yi1}) -> - % swap X with Y if line is steep - Steep = abs(Yi1 - Yi0) > abs(Xi1 - Xi0), - - {Xs0, Ys0, Xs1, Ys1} = case Steep of - true -> {Yi0,Xi0,Yi1,Xi1}; - false -> {Xi0,Yi0,Xi1,Yi1} - end, - - {X0,Y0,X1,Y1} = case Xs0 > Xs1 of - true -> {Xs1,Ys1,Xs0,Ys0}; - false -> {Xs0,Ys0,Xs1,Ys1} - end, - - DX = X1 - X0, - DY = abs(Y1 - Y0), - - Error = -DX/2, - - Ystep = case Y0 < Y1 of - true -> 1; - false -> -1 - end, - line_ls_step(X0, X1,Y0, DX, DY, Ystep, Error, X0, Steep, []). - -%% line_ls_step_(not)_steep -%% In: -%% Out: -%% [{Yi, Xl,Xr}] -%% Purpose: -%% Produce an line_interval for each Yi (Y index) - -line_ls_step(X, X1, Y, Dx, Dy, Ys, E, X0, false = Steep, LSs) when X < X1, E >= 0 -> - line_ls_step(X+1,X1,Y+Ys,Dx,Dy,Ys, E - Dx + Dy, X+1, Steep, [{Y,X0,X}|LSs]); -line_ls_step(X, X1, Y, Dx, Dy, Ys, E, X0, false = Steep, LSs) when X < X1 -> - line_ls_step(X+1,X1,Y,Dx,Dy,Ys, E + Dy, X0, Steep, LSs); -line_ls_step(X, _X1, Y, _Dx, _Dy, _Ys, _E, X0, false, LSs) -> - [{Y,X0,X}|LSs]; -line_ls_step(X, X1, Y, Dx, Dy, Ys, E, _X0, true = Steep, LSs) when X =< X1, E >= 0 -> - line_ls_step(X+1,X1,Y+Ys,Dx,Dy,Ys, E - Dx + Dy, X, Steep, [{X,Y,Y}|LSs]); -line_ls_step(X, X1, Y, Dx, Dy, Ys, E, X0, true = Steep, LSs) when X =< X1 -> - line_ls_step(X+1,X1,Y,Dx,Dy,Ys,E + Dy, X0, Steep, [{X,Y,Y}|LSs]); -line_ls_step(_X,_,_Y,_Dx,_Dy,_Ys,_E,_X0,_,LSs) -> - LSs. +%% [{Y,Xl,Xr}] +%% +line_to_linespans({X0,Y0},{X1,Y1},Wd) -> + Dx = abs(X1-X0), + Dy = abs(Y1-Y0), + Sx = if X0 < X1 -> 1; true -> -1 end, + Sy = if Y0 < Y1 -> 1; true -> -1 end, + E0 = Dx - Dy, + Ed = if Dx + Dy =:= 0 -> 1; true -> math:sqrt(Dx*Dx + Dy*Dy) end, + line_to_ls(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E0,Ed,(Wd+1)/2,[]). + +line_to_ls(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls0) -> + C = max(0, 255*(abs(E - Dx+Dy)/Ed - Wd + 1)), + Ls1 = [{Y0,X0,X0,C}|Ls0], + line_to_ls_sx(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls1,E). + +line_to_ls_sx(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls,E2) when 2*E2 > -Dx -> + line_to_ls_sx_do(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls,E2+Dy,Y0); +line_to_ls_sx(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls,E2) -> + line_to_ls_sy(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls,E2,X0). + +line_to_ls_sx_do(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls0,E2,Y) when E2 < Ed*Wd andalso + (Y1 =/= Y orelse Dx > Dy) -> + Y2 = Y + Sy, + C = max(0,255*(abs(E2)/Ed-Wd+1)), + Ls = [{Y2,X0,X0,C}|Ls0], + line_to_ls_sx_do(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls,E2+Dx,Y2); +line_to_ls_sx_do(X0,_Y0,X1,_Y1,_Dx,_Dy,_Sx,_Sy,_E,_Ed,_Wd,Ls,_E2,_Y) when X0 =:= X1 -> + Ls; +line_to_ls_sx_do(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls,_E2,_Y) -> + line_to_ls_sy(X0+Sx,Y0,X1,Y1,Dx,Dy,Sx,Sy,E-Dy,Ed,Wd,Ls,E,X0). + +line_to_ls_sy(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls0,E2,X) when 2*E2 =< Dy -> + line_to_ls_sy_do(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls0,Dx-E2,X); +line_to_ls_sy(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls0,_E2,_X) -> + line_to_ls(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls0). + +line_to_ls_sy_do(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls0,E2,X) when E2 < Ed*Wd andalso + (X1 =/= X orelse Dx < Dy) -> + X2 = X + Sx, + C = max(0,255*(abs(E2)/Ed-Wd+1)), + Ls = [{Y0,X2,X2,C}|Ls0], + line_to_ls_sy_do(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls,E2+Dy,X2); +line_to_ls_sy_do(_X0,Y0,_X1,Y1,_Dx,_Dy,_Sx,_Sy,_E,_Ed,_Wd,Ls,_E2,_X) when Y0 =:= Y1 -> + Ls; +line_to_ls_sy_do(X0,Y0,X1,Y1,Dx,Dy,Sx,Sy,E,Ed,Wd,Ls0,_E2,_X) -> + line_to_ls(X0,Y0+Sy,X1,Y1,Dx,Dy,Sx,Sy,E+Dx,Ed,Wd,Ls0). % Text diff --git a/lib/percept/test/egd_SUITE.erl b/lib/percept/test/egd_SUITE.erl index 3562ceae88..401695dddd 100644 --- a/lib/percept/test/egd_SUITE.erl +++ b/lib/percept/test/egd_SUITE.erl @@ -32,6 +32,7 @@ image_primitives/1, image_colors/1, image_font/1, + image_fans/1, image_png_compliant/1]). suite() -> @@ -41,6 +42,7 @@ suite() -> all() -> [image_create_and_destroy, image_shape, image_primitives, image_colors, image_font, + image_fans, image_png_compliant]. @@ -71,6 +73,7 @@ image_create_and_destroy(Config) when is_list(Config) -> %% Image color test. image_colors(Config) when is_list(Config) -> {W,H} = get_size(proplists:get_value(max_size, Config)), + Dir = proplists:get_value(priv_dir, Config), Image = egd:create(W, H), put(image_size, {W,H}), @@ -96,7 +99,15 @@ image_colors(Config) when is_list(Config) -> ok = egd:line(Image, get_point(), get_point(), Color) end, HtmlDefaultNames), - <<_/binary>> = egd:render(Image), + Png1 = <<_/binary>> = egd:render(Image,png,[{render_engine, alpha}]), + File1 = filename:join(Dir,"image_colors_alpha.png"), + ok = egd:save(Png1,File1), + ct:log("<p>Image alpha:</p><img src=\"~s\" />~n", [File1]), + Png2 = <<_/binary>> = egd:render(Image,png,[{render_engine, opaque}]), + File2 = filename:join(Dir,"image_colors_opaque.png"), + ok = egd:save(Png2,File2), + ct:log("<p>Image opaque:</p><img src=\"~s\" />~n", [File2]), + ok = egd:destroy(Image), erase(image_size), ok. @@ -104,6 +115,7 @@ image_colors(Config) when is_list(Config) -> %% Image shape API test. image_shape(Config) when is_list(Config) -> {W,H} = get_size(proplists:get_value(max_size, Config)), + Dir = proplists:get_value(priv_dir, Config), put(image_size, {W,H}), Im = egd:create(W, H), @@ -125,15 +137,21 @@ image_shape(Config) when is_list(Config) -> ok = bitmap_point_has_color(Bitmap, {W,H}, Pt2, Fgc), ok = bitmap_point_has_color(Bitmap, {W,H}, Pt1, Fgc), - <<_/binary>> = egd:render(Im, raw_bitmap, [{render_engine, alpha}]), + Bin = <<_/binary>> = egd:render(Im, raw_bitmap, [{render_engine, alpha}]), + Png = egd_png:binary(W,H,Bin), + File = filename:join(Dir,"image_shape.png"), + ok = egd:save(Png,File), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File]), ok = egd:destroy(Im), + erase(image_size), ok. %% Image shape API test. image_primitives(Config) when is_list(Config) -> {W,H} = get_size(proplists:get_value(max_size, Config)), + Dir = proplists:get_value(priv_dir, Config), put(image_size, {W,H}), Im0 = egd_primitives:create(W, H), @@ -157,7 +175,11 @@ image_primitives(Config) when is_list(Config) -> ok = bitmap_point_has_color(Bitmap, {W,H}, Pt2, Fgc), ok = bitmap_point_has_color(Bitmap, {W,H}, Pt1, Fgc), - <<_/binary>> = egd_render:binary(Im2, alpha), + Bin = <<_/binary>> = egd_render:binary(Im2, alpha), + Png = egd_png:binary(W,H,Bin), + File = filename:join(Dir,"image_primitives.png"), + ok = egd:save(Png,File), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File]), erase(image_size), ok. @@ -165,6 +187,7 @@ image_primitives(Config) when is_list(Config) -> %% Image font test. image_font(Config) when is_list(Config) -> {W,H} = get_size(proplists:get_value(max_size, Config)), + Dir = proplists:get_value(priv_dir, Config), put(image_size, {W,H}), Im = egd:create(W, H), Fgc = egd:color({0,130,0}), @@ -185,25 +208,46 @@ image_font(Config) when is_list(Config) -> GlyphStr4 = "{|}~", % Codes 123 -> 126 ok = egd:text(Im, get_point(), Font, GlyphStr1, Fgc), - <<_/binary>> = egd:render(Im, png), + Png1 = <<_/binary>> = egd:render(Im, png), + File1 = filename:join(Dir,"text1.png"), + ok = egd:save(Png1,File1), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File1]), ok = egd:text(Im, get_point(), Font, NumericStr, Fgc), - <<_/binary>> = egd:render(Im, png), + Png2 = <<_/binary>> = egd:render(Im, png), + File2 = filename:join(Dir,"text2.png"), + ok = egd:save(Png2,File2), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File2]), ok = egd:text(Im, get_point(), Font, GlyphStr2, Fgc), - <<_/binary>> = egd:render(Im, png), + Png3 = <<_/binary>> = egd:render(Im, png), + File3 = filename:join(Dir,"text3.png"), + ok = egd:save(Png3,File3), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File3]), ok = egd:text(Im, get_point(), Font, AlphaBigStr, Fgc), - <<_/binary>> = egd:render(Im, png), + Png4 = <<_/binary>> = egd:render(Im, png), + File4 = filename:join(Dir,"text4.png"), + ok = egd:save(Png4,File4), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File4]), ok = egd:text(Im, get_point(), Font, GlyphStr3, Fgc), - <<_/binary>> = egd:render(Im, png), + Png5 = <<_/binary>> = egd:render(Im, png), + File5 = filename:join(Dir,"text5.png"), + ok = egd:save(Png5,File5), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File5]), ok = egd:text(Im, get_point(), Font, AlphaSmStr, Fgc), - <<_/binary>> = egd:render(Im, png), + Png6 = <<_/binary>> = egd:render(Im, png), + File6 = filename:join(Dir,"text6.png"), + ok = egd:save(Png6,File6), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File6]), ok = egd:text(Im, get_point(), Font, GlyphStr4, Fgc), - <<_/binary>> = egd:render(Im, png), + Png7 = <<_/binary>> = egd:render(Im, png), + File7 = filename:join(Dir,"text7.png"), + ok = egd:save(Png7,File7), + ct:log("<p>Image:</p><img src=\"~s\" />~n", [File7]), ok = egd:destroy(Im), erase(image_size), @@ -224,6 +268,65 @@ image_png_compliant(Config) when is_list(Config) -> erase(image_size), ok. +image_fans(Config) when is_list(Config) -> + W = 1024, + H = 800, + Dir = proplists:get_value(priv_dir, Config), + + Fun = fun({F,Args},Im) -> + erlang:apply(egd_primitives,F,[Im|Args]) + end, + + %% fan1 + Ops1 = gen_vertical_fan(1,{0,400},egd:color(red),1024,800,-15), + Ops2 = gen_horizontal_fan(1,{512,800},egd:color(green),1024,0,-15), + + Im0 = egd_primitives:create(W,H), + Im1 = lists:foldl(Fun, Im0, Ops1 ++ Ops2), + Bin1 = egd_render:binary(Im1, opaque), + Png1 = egd_png:binary(W,H,Bin1), + + File1 = filename:join(Dir,"fan1_opaque.png"), + ok = egd:save(Png1,File1), + ct:log("<p>Image opaque width 1:</p><img src=\"~s\" />~n", [File1]), + + Bin2 = egd_render:binary(Im1, alpha), + Png2 = egd_png:binary(W,H,Bin2), + + File2 = filename:join(Dir,"fan1_alpha.png"), + ok = egd:save(Png2,File2), + ct:log("<p>Image alpha width 1:</p><img src=\"~s\" />~n", [File2]), + + + %% fan2 + Ops3 = gen_vertical_fan(7,{0,400},egd:color(red),1024,800,-15), + Ops4 = gen_horizontal_fan(7,{512,800},egd:color(green),1024,0,-15), + + Im2 = lists:foldl(Fun, Im0, Ops3 ++ Ops4), + Bin3 = egd_render:binary(Im2, opaque), + Png3 = egd_png:binary(W,H,Bin3), + + File3 = filename:join(Dir,"fan2_opaque.png"), + ok = egd:save(Png3,File3), + ct:log("<p>Image opaque width 7:</p><img src=\"~s\" />~n", [File3]), + + Bin4 = egd_render:binary(Im2, alpha), + Png4 = egd_png:binary(W,H,Bin4), + + File4 = filename:join(Dir,"fan2_alpha.png"), + ok = egd:save(Png4,File4), + ct:log("<p>Image alpha width 7:</p><img src=\"~s\" />~n", [File4]), + ok. + +gen_vertical_fan(Wd,Pt,C,X,Y,Step) when Y > 0 -> + [{line,[Pt,{X,Y},Wd,C]}|gen_vertical_fan(Wd,Pt,C,X,Y + Step,Step)]; +gen_vertical_fan(_,_,_,_,_,_) -> []. + +gen_horizontal_fan(Wd,Pt,C,X,Y,Step) when X > 0 -> + [{line,[Pt,{X,Y},Wd,C]}|gen_horizontal_fan(Wd,Pt,C,X + Step,Y,Step)]; +gen_horizontal_fan(_,_,_,_,_,_) -> []. + + %%---------------------------------------------------------------------- %% Auxiliary tests %%---------------------------------------------------------------------- @@ -239,33 +342,6 @@ bitmap_point_has_color(Bitmap, {W,_}, {X,Y}, C) -> {error, {Other,{CR,CG,CB}}} end. -%% jfif header by specification -%% 2 bytes, length -%% 5 bytes, identifier ="JFIF\0" -%% 2 bytes, version, (major, minor) -%% 1 byte , units -%% However, JFIF seems to start at 6 (7 with 1-index)? - -binary_is_jfif_compliant(JpegBin) -> - {Bin, _} = split_binary(JpegBin, 11), - List = binary_to_list(Bin), - case lists:sublist(List, 7, 4) of - "JFIF" -> true; - Other -> - io:format("img -> ~p~n", [Other]), - false - end. - -binary_is_gif_compliant(GifBin) -> - {Bin, _} = split_binary(GifBin, 10), - List = binary_to_list(Bin), - case lists:sublist(List, 1,5) of - "GIF87" -> true; - Other -> - io:format("img -> ~p~n", [Other]), - false - end. - binary_is_png_compliant(PngBin) -> {Bin, _} = split_binary(PngBin, 10), List = binary_to_list(Bin), diff --git a/lib/public_key/test/pbe_SUITE.erl b/lib/public_key/test/pbe_SUITE.erl index 4b7ebf4309..004eaefc27 100644 --- a/lib/public_key/test/pbe_SUITE.erl +++ b/lib/public_key/test/pbe_SUITE.erl @@ -30,7 +30,8 @@ %% Common Test interface functions ----------------------------------- %%-------------------------------------------------------------------- -suite() -> [{ct_hooks,[ts_install_cth]}]. +suite() -> + []. all() -> [ @@ -199,7 +200,7 @@ pbdkdf2(Config) when is_list(Config) -> old_enc() -> [{doc,"Tests encode/decode RSA key encrypted with different ciphers using old PEM encryption scheme"}]. old_enc(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), %% key generated with ssh-keygen -N hello_aes -f old_aes_128_cbc_enc_key.pem {ok, PemAesCbc} = file:read_file(filename:join(Datadir, "old_aes_128_cbc_enc_key.pem")), @@ -226,7 +227,7 @@ check_key_info(#'PrivateKeyInfo'{privateKeyAlgorithm = #'RSAPrivateKey'{} = public_key:der_decode('RSAPrivateKey', iolist_to_binary(Key)). decode_encode_key_file(File, Password, Cipher, Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, PemKey} = file:read_file(filename:join(Datadir, File)), PemEntry = public_key:pem_decode(PemKey), diff --git a/lib/public_key/test/pkits_SUITE.erl b/lib/public_key/test/pkits_SUITE.erl index 17be99b52d..487b3dbe3f 100644 --- a/lib/public_key/test/pkits_SUITE.erl +++ b/lib/public_key/test/pkits_SUITE.erl @@ -53,7 +53,7 @@ %%-------------------------------------------------------------------- suite() -> - [{ct_hooks,[ts_install_cth]}]. + []. all() -> [{group, signature_verification}, diff --git a/lib/public_key/test/public_key_SUITE.erl b/lib/public_key/test/public_key_SUITE.erl index 51050c4480..9c39c36be4 100644 --- a/lib/public_key/test/public_key_SUITE.erl +++ b/lib/public_key/test/public_key_SUITE.erl @@ -34,7 +34,8 @@ %% Common Test interface functions ----------------------------------- %%-------------------------------------------------------------------- -suite() -> [{ct_hooks,[ts_install_cth]}]. +suite() -> + []. all() -> [app, appup, @@ -108,7 +109,7 @@ appup(Config) when is_list(Config) -> dsa_pem() -> [{doc, "DSA PEM-file decode/encode"}]. dsa_pem(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), [{'DSAPrivateKey', DerDSAKey, not_encrypted} = Entry0 ] = erl_make_certs:pem_to_der(filename:join(Datadir, "dsa.pem")), @@ -131,7 +132,7 @@ dsa_pem(Config) when is_list(Config) -> rsa_pem() -> [{doc, "RSA PEM-file decode/encode"}]. rsa_pem(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), [{'RSAPrivateKey', DerRSAKey, not_encrypted} = Entry0 ] = erl_make_certs:pem_to_der(filename:join(Datadir, "client_key.pem")), @@ -166,7 +167,7 @@ rsa_pem(Config) when is_list(Config) -> ec_pem() -> [{doc, "EC key PEM-file decode/encode"}]. ec_pem(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, ECPubPem} = file:read_file(filename:join(Datadir, "ec_pubkey.pem")), [{'SubjectPublicKeyInfo', _, _} = PubEntry0] = public_key:pem_decode(ECPubPem), @@ -192,7 +193,7 @@ ec_pem(Config) when is_list(Config) -> encrypted_pem() -> [{doc, "Encrypted PEM-file decode/encode"}]. encrypted_pem(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), [{'RSAPrivateKey', DerRSAKey, not_encrypted}] = erl_make_certs:pem_to_der(filename:join(Datadir, "client_key.pem")), @@ -225,7 +226,7 @@ encrypted_pem(Config) when is_list(Config) -> dh_pem() -> [{doc, "DH parametrs PEM-file decode/encode"}]. dh_pem(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), [{'DHParameter', _DerDH, not_encrypted} = Entry] = erl_make_certs:pem_to_der(filename:join(Datadir, "dh.pem")), asn1_encode_decode(Entry). @@ -235,7 +236,7 @@ dh_pem(Config) when is_list(Config) -> pkcs10_pem() -> [{doc, "PKCS-10 PEM-file decode/encode"}]. pkcs10_pem(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), [{'CertificationRequest', _DerPKCS10, not_encrypted} = Entry] = erl_make_certs:pem_to_der(filename:join(Datadir, "req.pem")), asn1_encode_decode(Entry). @@ -243,7 +244,7 @@ pkcs10_pem(Config) when is_list(Config) -> pkcs7_pem() -> [{doc, "PKCS-7 PEM-file decode/encode"}]. pkcs7_pem(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), [{'ContentInfo', _, not_encrypted} = Entry0] = erl_make_certs:pem_to_der(filename:join(Datadir, "pkcs7_cert.pem")), [{'ContentInfo', _, not_encrypted} = Entry1] = @@ -255,7 +256,7 @@ pkcs7_pem(Config) when is_list(Config) -> cert_pem() -> [{doc, "Certificate PEM-file decode/encode"}]. cert_pem(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), [{'Certificate', _, not_encrypted} = Entry0] = erl_make_certs:pem_to_der(filename:join(Datadir, "client_cert.pem")), @@ -273,7 +274,7 @@ cert_pem(Config) when is_list(Config) -> ssh_rsa_public_key() -> [{doc, "ssh rsa public key decode/encode"}]. ssh_rsa_public_key(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, RSARawSsh2} = file:read_file(filename:join(Datadir, "ssh2_rsa_pub")), [{PubKey, Attributes1}] = public_key:ssh_decode(RSARawSsh2, public_key), @@ -299,7 +300,7 @@ ssh_rsa_public_key(Config) when is_list(Config) -> ssh_dsa_public_key() -> [{doc, "ssh dsa public key decode/encode"}]. ssh_dsa_public_key(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, DSARawSsh2} = file:read_file(filename:join(Datadir, "ssh2_dsa_pub")), [{PubKey, Attributes1}] = public_key:ssh_decode(DSARawSsh2, public_key), @@ -325,7 +326,7 @@ ssh_dsa_public_key(Config) when is_list(Config) -> ssh_ecdsa_public_key() -> [{doc, "ssh ecdsa public key decode/encode"}]. ssh_ecdsa_public_key(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, ECDSARawSsh2} = file:read_file(filename:join(Datadir, "ssh2_ecdsa_pub")), [{PubKey, Attributes1}] = public_key:ssh_decode(ECDSARawSsh2, public_key), @@ -350,7 +351,7 @@ ssh_ecdsa_public_key(Config) when is_list(Config) -> ssh_rfc4716_rsa_comment() -> [{doc, "Test comment header and rsa key"}]. ssh_rfc4716_rsa_comment(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, RSARawSsh2} = file:read_file(filename:join(Datadir, "ssh2_rsa_comment_pub")), [{#'RSAPublicKey'{} = PubKey, Attributes}] = @@ -366,7 +367,7 @@ ssh_rfc4716_rsa_comment(Config) when is_list(Config) -> ssh_rfc4716_dsa_comment() -> [{doc, "Test comment header and dsa key"}]. ssh_rfc4716_dsa_comment(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, DSARawSsh2} = file:read_file(filename:join(Datadir, "ssh2_dsa_comment_pub")), [{{_, #'Dss-Parms'{}} = PubKey, Attributes}] = @@ -386,7 +387,7 @@ ssh_rfc4716_dsa_comment(Config) when is_list(Config) -> ssh_rfc4716_rsa_subject() -> [{doc, "Test another header value than comment"}]. ssh_rfc4716_rsa_subject(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, RSARawSsh2} = file:read_file(filename:join(Datadir, "ssh2_subject_pub")), [{#'RSAPublicKey'{} = PubKey, Attributes}] = @@ -406,7 +407,7 @@ ssh_rfc4716_rsa_subject(Config) when is_list(Config) -> ssh_known_hosts() -> [{doc, "ssh known hosts file encode/decode"}]. ssh_known_hosts(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, SshKnownHosts} = file:read_file(filename:join(Datadir, "known_hosts")), [{#'RSAPublicKey'{}, Attributes1}, {#'RSAPublicKey'{}, Attributes2}, @@ -435,7 +436,7 @@ ssh_known_hosts(Config) when is_list(Config) -> ssh1_known_hosts() -> [{doc, "ssh (ver 1) known hosts file encode/decode"}]. ssh1_known_hosts(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, SshKnownHosts} = file:read_file(filename:join(Datadir, "ssh1_known_hosts")), [{#'RSAPublicKey'{}, Attributes1}, {#'RSAPublicKey'{}, Attributes2},{#'RSAPublicKey'{}, Attributes3}] @@ -455,7 +456,7 @@ ssh1_known_hosts(Config) when is_list(Config) -> ssh_auth_keys() -> [{doc, "ssh authorized keys file encode/decode"}]. ssh_auth_keys(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, SshAuthKeys} = file:read_file(filename:join(Datadir, "auth_keys")), [{#'RSAPublicKey'{}, Attributes1}, {{_, #'Dss-Parms'{}}, Attributes2}, @@ -481,7 +482,7 @@ ssh_auth_keys(Config) when is_list(Config) -> ssh1_auth_keys() -> [{doc, "ssh (ver 1) authorized keys file encode/decode"}]. ssh1_auth_keys(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, SshAuthKeys} = file:read_file(filename:join(Datadir, "ssh1_auth_keys")), [{#'RSAPublicKey'{}, Attributes1}, @@ -509,7 +510,7 @@ ssh1_auth_keys(Config) when is_list(Config) -> ssh_openssh_public_key_with_comment() -> [{doc, "Test that emty lines and lines starting with # are ignored"}]. ssh_openssh_public_key_with_comment(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, DSARawOpenSsh} = file:read_file(filename:join(Datadir, "openssh_dsa_with_comment_pub")), [{{_, #'Dss-Parms'{}}, _}] = public_key:ssh_decode(DSARawOpenSsh, openssh_public_key). @@ -518,7 +519,7 @@ ssh_openssh_public_key_with_comment(Config) when is_list(Config) -> ssh_openssh_public_key_long_header() -> [{doc, "Test that long headers are handled"}]. ssh_openssh_public_key_long_header(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok,RSARawOpenSsh} = file:read_file(filename:join(Datadir, "ssh_rsa_long_header_pub")), [{#'RSAPublicKey'{}, _}] = Decoded = public_key:ssh_decode(RSARawOpenSsh, public_key), @@ -577,7 +578,7 @@ dsa_sign_verify(Config) when is_list(Config) -> public_key:pem_entry_decode(CertKey1), true = public_key:pkix_verify(Cert2, {Y, #'Dss-Parms'{p=P, q=Q, g=G}}), - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), [DsaKey = {'DSAPrivateKey', _, _}] = erl_make_certs:pem_to_der(filename:join(Datadir, "dsa.pem")), DSAPrivateKey = public_key:pem_entry_decode(DsaKey), @@ -606,7 +607,7 @@ dsa_sign_verify(Config) when is_list(Config) -> pkix() -> [{doc, "Misc pkix tests not covered elsewhere"}]. pkix(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), Certs0 = erl_make_certs:pem_to_der(filename:join(Datadir, "cacerts.pem")), Certs1 = erl_make_certs:pem_to_der(filename:join(Datadir, "client_cert.pem")), TestTransform = fun({'Certificate', CertDer, not_encrypted}) -> @@ -749,7 +750,7 @@ pkix_iso_rsa_oid() -> [{doc, "Test workaround for supporting certs that use ISO oids" " 1.3.14.3.2.29 instead of PKIX/PKCS oid"}]. pkix_iso_rsa_oid(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, PemCert} = file:read_file(filename:join(Datadir, "rsa_ISO.pem")), [{_, Cert, _}] = public_key:pem_decode(PemCert), OTPCert = public_key:pkix_decode_cert(Cert, otp), @@ -761,7 +762,7 @@ pkix_iso_dsa_oid() -> [{doc, "Test workaround for supporting certs that use ISO oids" "1.3.14.3.2.27 instead of PKIX/PKCS oid"}]. pkix_iso_dsa_oid(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, PemCert} = file:read_file(filename:join(Datadir, "dsa_ISO.pem")), [{_, Cert, _}] = public_key:pem_decode(PemCert), OTPCert = public_key:pkix_decode_cert(Cert, otp), @@ -774,7 +775,7 @@ pkix_crl() -> [{doc, "test pkix_crl_* functions"}]. pkix_crl(Config) when is_list(Config) -> - Datadir = ?config(data_dir, Config), + Datadir = proplists:get_value(data_dir, Config), {ok, PemCRL} = file:read_file(filename:join(Datadir, "idp_crl.pem")), [{_, CRL, _}] = public_key:pem_decode(PemCRL), diff --git a/lib/reltool/doc/src/notes.xml b/lib/reltool/doc/src/notes.xml index 082079aa74..0a83954865 100644 --- a/lib/reltool/doc/src/notes.xml +++ b/lib/reltool/doc/src/notes.xml @@ -194,7 +194,6 @@ <section><title>Fixed Bugs and Malfunctions</title> <list> <item> - <p> <list> <item> If <c>incl_cond</c> was set to <c>derived</c> on module level, then reltool_server would crash with a <c>case_clause</c>. This has been corrected. @@ -225,7 +224,7 @@ implemented in reltool. </item> <item> Instead of only looking at the directory name, reltool now first looks for a <c>.app</c> file in order to figure out the name of - an application. </item> </list></p> + an application. </item> </list> <p> Own Id: OTP-10012 Aux Id: kunagi-171 [82] </p> </item> @@ -272,7 +271,6 @@ <section><title>Fixed Bugs and Malfunctions</title> <list> <item> - <p> Miscellaneous corrections: <list> <item> Start of reltool GUI would sometimes crash with a badmatch in reltool_sys_win:do_init. This has been corrected. </item> @@ -311,7 +309,7 @@ and when generating target system. </item> <item> Title of dependecies column in app and mod window is changed from "Modules used by others" to "Modules using this". - </item> </list></p> + </item> </list> <p> Own Id: OTP-9792</p> </item> @@ -338,7 +336,7 @@ </item> <item> <p> - Some bug fixes related to the handling of escripts: + Some bug fixes related to the handling of escripts:</p> <list> <item> Reltool could not handle escripts with inlined applications. This has been corrected. Inlined applications will be visible in the GUI, but not possible @@ -357,7 +355,7 @@ another escript, for which the name sorts before the existing one, would cause reltool to fail saying "Application name clash". This has been corrected. - </item> </list></p> + </item> </list> <p> Own Id: OTP-9968</p> </item> diff --git a/lib/reltool/doc/src/reltool.xml b/lib/reltool/doc/src/reltool.xml index f4effc3f2e..38448e7961 100644 --- a/lib/reltool/doc/src/reltool.xml +++ b/lib/reltool/doc/src/reltool.xml @@ -248,7 +248,7 @@ <p>When starting this release, three things must be specified:</p> <taglist> - <tag><b>Which <c>releases</c> directory to use</b></tag> + <tag><em>Which <c>releases</c> directory to use</em></tag> <item>Tell the release handler to use the <c>releases</c> directory in our target structure instead of <c>$OTP_ROOT/releases</c>. This is done by setting the SASL @@ -257,7 +257,7 @@ <target-dir>/releases</c>) or in <c>sys.config</c>.</item> - <tag><b>Which boot file to use</b></tag> + <tag><em>Which boot file to use</em></tag> <item>The default boot file is <c>$OTP_ROOT/bin/start</c>, but in this case we need to specify a boot file from our target structure, typically @@ -265,7 +265,7 @@ is done with the <c>-boot</c> command line option to <c>erl</c></item> - <tag><b>The location of our applications</b></tag> + <tag><em>The location of our applications</em></tag> <item>The generated .script (and .boot) file uses the environment variable <c>$RELTOOL_EXT_LIB</c> as prefix for the paths to all applications. The <c>-boot_var</c> option @@ -275,8 +275,8 @@ </taglist> <p>Example:</p> - <p><code>erl -sasl releases_dir \"mytarget/releases\" -boot mytarget/releases/1.0/myrel\ - -boot_var RELTOOL_EXT_LIB mytarget/lib</code></p> + <code>erl -sasl releases_dir \"mytarget/releases\" -boot mytarget/releases/1.0/myrel\ + -boot_var RELTOOL_EXT_LIB mytarget/lib</code> </item> <tag><c>incl_sys_filters</c></tag> diff --git a/lib/runtime_tools/c_src/dyntrace.c b/lib/runtime_tools/c_src/dyntrace.c index 0178d95efb..e7a4a73373 100644 --- a/lib/runtime_tools/c_src/dyntrace.c +++ b/lib/runtime_tools/c_src/dyntrace.c @@ -433,7 +433,7 @@ static ERL_NIF_TERM trace_call(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv char class[LTTNG_BUFFER_SZ]; enif_get_tuple(env, argv[4], &arity, &tuple); - erts_snprintf(class, LTTNG_BUFFER_SZ, "%T", tuple[0]); + enif_snprintf(class, LTTNG_BUFFER_SZ, "%T", tuple[0]); if (enif_get_tuple(env, argv[3], &arity, &tuple)) { enif_get_uint(env, tuple[2], &len); @@ -465,7 +465,7 @@ static ERL_NIF_TERM trace_send(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv char msg[LTTNG_BUFFER_SZ]; lttng_pid_to_str(argv[4], to); - erts_snprintf(msg, LTTNG_BUFFER_SZ, "%T", argv[3]); + enif_snprintf(msg, LTTNG_BUFFER_SZ, "%T", argv[3]); LTTNG3(message_send, pid, to, msg); } else if (argv[0] == atom_send_to_non_existing_process) { @@ -473,7 +473,7 @@ static ERL_NIF_TERM trace_send(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv char msg[LTTNG_BUFFER_SZ]; lttng_pid_to_str(argv[4], to); - erts_snprintf(msg, LTTNG_BUFFER_SZ, "%T", argv[3]); + enif_snprintf(msg, LTTNG_BUFFER_SZ, "%T", argv[3]); /* mark it as non existing ? */ LTTNG3(message_send, pid, to, msg); @@ -496,7 +496,7 @@ static ERL_NIF_TERM trace_receive(ErlNifEnv* env, int argc, const ERL_NIF_TERM a char msg[LTTNG_BUFFER_SZ]; lttng_pid_to_str(argv[2], pid); - erts_snprintf(msg, LTTNG_BUFFER_SZ, "%T", argv[3]); + enif_snprintf(msg, LTTNG_BUFFER_SZ, "%T", argv[3]); LTTNG2(message_receive, pid, msg); } @@ -560,11 +560,11 @@ static ERL_NIF_TERM trace_procs(ErlNifEnv* env, int argc, const ERL_NIF_TERM arg /* register */ } else if (argv[0] == atom_register) { char name[LTTNG_BUFFER_SZ]; - erts_snprintf(name, LTTNG_BUFFER_SZ, "%T", argv[3]); + enif_snprintf(name, LTTNG_BUFFER_SZ, "%T", argv[3]); LTTNG3(process_register, pid, name, "register"); } else if (argv[0] == atom_unregister) { char name[LTTNG_BUFFER_SZ]; - erts_snprintf(name, LTTNG_BUFFER_SZ, "%T", argv[3]); + enif_snprintf(name, LTTNG_BUFFER_SZ, "%T", argv[3]); LTTNG3(process_register, pid, name, "unregister"); /* link */ } else if (argv[0] == atom_link) { @@ -582,7 +582,7 @@ static ERL_NIF_TERM trace_procs(ErlNifEnv* env, int argc, const ERL_NIF_TERM arg /* exit */ } else if (argv[0] == atom_exit) { char reason[LTTNG_BUFFER_SZ]; - erts_snprintf(reason, LTTNG_BUFFER_SZ, "%T", argv[3]); + enif_snprintf(reason, LTTNG_BUFFER_SZ, "%T", argv[3]); LTTNG2(process_exit, pid, reason); } return atom_ok; @@ -622,11 +622,11 @@ static ERL_NIF_TERM trace_ports(ErlNifEnv* env, int argc, const ERL_NIF_TERM arg lttng_decl_procbuf(pid); lttng_pid_to_str(argv[3], pid); - erts_snprintf(driver, LTTNG_BUFFER_SZ, "%T", argv[4]); + enif_snprintf(driver, LTTNG_BUFFER_SZ, "%T", argv[4]); LTTNG3(port_open, pid, driver, port); } else if (argv[0] == atom_closed) { char reason[LTTNG_BUFFER_SZ]; - erts_snprintf(reason, LTTNG_BUFFER_SZ, "%T", argv[3]); + enif_snprintf(reason, LTTNG_BUFFER_SZ, "%T", argv[3]); LTTNG2(port_exit, port, reason); /* link */ @@ -705,7 +705,7 @@ static ERL_NIF_TERM trace_running_ports(ErlNifEnv* env, int argc, const ERL_NIF_ lttng_decl_mfabuf(where); lttng_portid_to_str(argv[2], pid); - erts_snprintf(where, LTTNG_BUFFER_SZ, "%T", argv[3]); + enif_snprintf(where, LTTNG_BUFFER_SZ, "%T", argv[3]); /* running ports */ if (argv[0] == atom_in) { diff --git a/lib/runtime_tools/c_src/dyntrace_lttng.h b/lib/runtime_tools/c_src/dyntrace_lttng.h index 3550a1cab5..2a3224e191 100644 --- a/lib/runtime_tools/c_src/dyntrace_lttng.h +++ b/lib/runtime_tools/c_src/dyntrace_lttng.h @@ -59,10 +59,10 @@ char Name[LTTNG_MFA_BUFFER_SZ] #define lttng_pid_to_str(pid, name) \ - erts_snprintf(name, LTTNG_PROC_BUFFER_SZ, "%T", (pid)) + enif_snprintf(name, LTTNG_PROC_BUFFER_SZ, "%T", (pid)) #define lttng_portid_to_str(pid, name) \ - erts_snprintf(name, LTTNG_PORT_BUFFER_SZ, "%T", (pid)) + enif_snprintf(name, LTTNG_PORT_BUFFER_SZ, "%T", (pid)) #define lttng_proc_to_str(p, name) \ lttng_pid_to_str(((p) ? (p)->common.id : ERTS_INVALID_PID), name) @@ -71,7 +71,7 @@ lttng_portid_to_str(((p) ? (p)->common.id : ERTS_INVALID_PORT), name) #define lttng_mfa_to_str(m,f,a, Name) \ - erts_snprintf(Name, LTTNG_MFA_BUFFER_SZ, "%T:%T/%lu", (Eterm)(m), (Eterm)(f), (Uint)(a)) + enif_snprintf(Name, LTTNG_MFA_BUFFER_SZ, "%T:%T/%lu", (Eterm)(m), (Eterm)(f), (Uint)(a)) /* Process scheduling */ diff --git a/lib/runtime_tools/c_src/trace_ip_drv.c b/lib/runtime_tools/c_src/trace_ip_drv.c index c73630f702..195558f958 100644 --- a/lib/runtime_tools/c_src/trace_ip_drv.c +++ b/lib/runtime_tools/c_src/trace_ip_drv.c @@ -374,6 +374,7 @@ static void trace_ip_output(ErlDrvData handle, char *buff, ErlDrvSizeT bufflen) } return; } + ASSERT(!IS_INVALID_SOCKET(data->fd)); if (data->que[data->questart] != NULL) { trace_ip_ready_output(handle, sock2event(data->fd)); } @@ -412,6 +413,7 @@ static void trace_ip_ready_input(ErlDrvData handle, ErlDrvEvent fd) /* ** Maybe accept, we are a listen port... */ + ASSERT(IS_INVALID_SOCKET(data->fd)); if (!IS_INVALID_SOCKET((client = my_accept(data->listenfd)))) { data->fd = client; set_nonblocking(client); @@ -735,6 +737,7 @@ static void close_client(TraceIpData *data) { my_driver_select(data, data->fd, FLAG_WRITE | FLAG_READ, SELECT_CLOSE); data->flags |= FLAG_LISTEN_PORT; + data->fd = INVALID_SOCKET; if (!(data->flags & FLAG_FILL_ALWAYS)) { clean_que(data); } diff --git a/lib/runtime_tools/doc/src/LTTng.xml b/lib/runtime_tools/doc/src/LTTng.xml index 4a87133c57..06152c66d6 100644 --- a/lib/runtime_tools/doc/src/LTTng.xml +++ b/lib/runtime_tools/doc/src/LTTng.xml @@ -76,7 +76,7 @@ $ make </code> <item><c>entry : string</c> :: Code Location. Ex. <c>"lists:sort/1"</c></item> </list> <p>Example:</p> - <p><code type="none">process_spawn: { cpu_id = 3 }, { pid = "<0.131.0>", parent = "<0.130.0>", entry = "erlang:apply/2" }</code></p> + <code type="none">process_spawn: { cpu_id = 3 }, { pid = "<0.131.0>", parent = "<0.130.0>", entry = "erlang:apply/2" }</code> <p><em>process_link</em></p> <list type="bulleted"> @@ -85,7 +85,7 @@ $ make </code> <item><c>type : string</c> :: <c>"link" | "unlink"</c></item> </list> <p>Example:</p> - <p><code type="none">process_link: { cpu_id = 3 }, { from = "<0.130.0>", to = "<0.131.0>", type = "link" }</code></p> + <code type="none">process_link: { cpu_id = 3 }, { from = "<0.130.0>", to = "<0.131.0>", type = "link" }</code> <p><em>process_exit</em></p> @@ -94,7 +94,7 @@ $ make </code> <item><c>reason : string</c> :: Exit reason. Ex. <c>"normal"</c></item> </list> <p>Example:</p> - <p><code type="none">process_exit: { cpu_id = 3 }, { pid = "<0.130.0>", reason = "normal" }</code></p> + <code type="none">process_exit: { cpu_id = 3 }, { pid = "<0.130.0>", reason = "normal" }</code> <p><em>process_register</em></p> <list type="bulleted"> @@ -103,7 +103,7 @@ $ make </code> <item><c>type : string</c> :: <c>"register" | "unregister"</c></item> </list> <p>Example:</p> - <p><code type="none">process_register: { cpu_id = 0 }, { pid = "<0.128.0>", name = "dyntrace_lttng_SUITE" type = "register" }</code></p> + <code type="none">process_register: { cpu_id = 0 }, { pid = "<0.128.0>", name = "dyntrace_lttng_SUITE" type = "register" }</code> <p><em>process_scheduled</em></p> <list type="bulleted"> @@ -113,7 +113,7 @@ $ make </code> </list> <p>Example:</p> - <p><code type="none">process_scheduled: { cpu_id = 0 }, { pid = "<0.136.0>", entry = "erlang:apply/2", type = "in" }</code></p> + <code type="none">process_scheduled: { cpu_id = 0 }, { pid = "<0.136.0>", entry = "erlang:apply/2", type = "in" }</code> <p><em>port_open</em></p> @@ -124,7 +124,7 @@ $ make </code> </list> <p>Example:</p> - <p><code type="none">port_open: { cpu_id = 5 }, { pid = "<0.131.0>", driver = "'/bin/sh -s unix:cmd'", port = "#Port<0.1887>" }</code></p> + <code type="none">port_open: { cpu_id = 5 }, { pid = "<0.131.0>", driver = "'/bin/sh -s unix:cmd'", port = "#Port<0.1887>" }</code> <p><em>port_exit</em></p> <list type="bulleted"> @@ -132,7 +132,7 @@ $ make </code> <item><c>reason : string</c> :: Exit reason. Ex. <c>"normal"</c></item> </list> <p>Example:</p> - <p><code type="none">port_exit: { cpu_id = 5 }, { port = "#Port<0.1887>", reason = "normal" }</code></p> + <code type="none">port_exit: { cpu_id = 5 }, { port = "#Port<0.1887>", reason = "normal" }</code> <p><em>port_link</em></p> <list type="bulleted"> @@ -141,7 +141,7 @@ $ make </code> <item><c>type : string</c> :: <c>"link" | "unlink"</c></item> </list> <p>Example:</p> - <p><code type="none">port_link: { cpu_id = 5 }, { from = "#Port<0.1887>", to = "<0.131.0>", type = "unlink" }</code></p> + <code type="none">port_link: { cpu_id = 5 }, { from = "#Port<0.1887>", to = "<0.131.0>", type = "unlink" }</code> <p><em>port_scheduled</em></p> <list type="bulleted"> @@ -151,7 +151,7 @@ $ make </code> </list> <p>Example:</p> - <p><code type="none">port_scheduled: { cpu_id = 5 }, { pid = "#Port<0.1905>", entry = "close", type = "out" }</code></p> + <code type="none">port_scheduled: { cpu_id = 5 }, { pid = "#Port<0.1905>", entry = "close", type = "out" }</code> <p><em>function_call</em></p> <list type="bulleted"> @@ -160,7 +160,7 @@ $ make </code> <item><c>depth : integer</c> :: Stack depth. Ex. <c>0</c></item> </list> <p>Example:</p> - <p><code type="none">function_call: { cpu_id = 5 }, { pid = "<0.145.0>", entry = "dyntrace_lttng_SUITE:'-t_call/1-fun-1-'/0", depth = 0 }</code></p> + <code type="none">function_call: { cpu_id = 5 }, { pid = "<0.145.0>", entry = "dyntrace_lttng_SUITE:'-t_call/1-fun-1-'/0", depth = 0 }</code> <p><em>function_return</em></p> <list type="bulleted"> @@ -169,7 +169,7 @@ $ make </code> <item><c>depth : integer</c> :: Stack depth. Ex. <c>0</c></item> </list> <p>Example:</p> - <p><code type="none">function_return: { cpu_id = 5 }, { pid = "<0.145.0>", entry = "dyntrace_lttng_SUITE:waiter/0", depth = 0 }</code></p> + <code type="none">function_return: { cpu_id = 5 }, { pid = "<0.145.0>", entry = "dyntrace_lttng_SUITE:waiter/0", depth = 0 }</code> <p><em>function_exception</em></p> <list type="bulleted"> @@ -178,7 +178,7 @@ $ make </code> <item><c>class : string</c> :: Error reason. Ex. <c>"error"</c></item> </list> <p>Example:</p> - <p><code type="none">function_exception: { cpu_id = 5 }, { pid = "<0.144.0>", entry = "t:call_exc/1", class = "error" }</code></p> + <code type="none">function_exception: { cpu_id = 5 }, { pid = "<0.144.0>", entry = "t:call_exc/1", class = "error" }</code> <p><em>message_send</em></p> <list type="bulleted"> @@ -187,7 +187,7 @@ $ make </code> <item><c>message : string</c> :: Message sent. Ex. <c>"{<0.162.0>,ok}"</c></item> </list> <p>Example:</p> - <p><code type="none">message_send: { cpu_id = 3 }, { from = "#Port<0.1938>", to = "<0.160.0>", message = "{#Port<0.1938>,eof}" }</code></p> + <code type="none">message_send: { cpu_id = 3 }, { from = "#Port<0.1938>", to = "<0.160.0>", message = "{#Port<0.1938>,eof}" }</code> <p><em>message_receive</em></p> <list type="bulleted"> @@ -195,7 +195,7 @@ $ make </code> <item><c>message : string</c> :: Message received. Ex. <c>"{<0.162.0>,ok}"</c></item> </list> <p>Example:</p> - <p><code type="none">message_receive: { cpu_id = 7 }, { to = "<0.167.0>", message = "{<0.165.0>,ok}" }</code></p> + <code type="none">message_receive: { cpu_id = 7 }, { to = "<0.167.0>", message = "{<0.165.0>,ok}" }</code> <p><em>gc_minor_start</em></p> <list type="bulleted"> @@ -205,7 +205,7 @@ $ make </code> <item><c>old_heap : integer</c> :: Old heap word size. Ex. <c>233</c></item> </list> <p>Example:</p> - <p><code type="none">gc_minor_start: { cpu_id = 0 }, { pid = "<0.172.0>", need = 0, heap = 610, old_heap = 0 }</code></p> + <code type="none">gc_minor_start: { cpu_id = 0 }, { pid = "<0.172.0>", need = 0, heap = 610, old_heap = 0 }</code> <p><em>gc_minor_end</em></p> <list type="bulleted"> @@ -215,7 +215,7 @@ $ make </code> <item><c>old_heap : integer</c> :: Old heap word size. Ex. <c>233</c></item> </list> <p>Example:</p> - <p><code type="none">gc_minor_end: { cpu_id = 0 }, { pid = "<0.172.0>", reclaimed = 120, heap = 1598, old_heap = 1598 }</code></p> + <code type="none">gc_minor_end: { cpu_id = 0 }, { pid = "<0.172.0>", reclaimed = 120, heap = 1598, old_heap = 1598 }</code> <p><em>gc_major_start</em></p> <list type="bulleted"> @@ -225,7 +225,7 @@ $ make </code> <item><c>old_heap : integer</c> :: Old heap word size. Ex. <c>233</c></item> </list> <p>Example:</p> - <p><code type="none">gc_major_start: { cpu_id = 0 }, { pid = "<0.172.0>", need = 8, heap = 2586, old_heap = 1598 }</code></p> + <code type="none">gc_major_start: { cpu_id = 0 }, { pid = "<0.172.0>", need = 8, heap = 2586, old_heap = 1598 }</code> <p><em>gc_major_end</em></p> <list type="bulleted"> @@ -235,7 +235,7 @@ $ make </code> <item><c>old_heap : integer</c> :: Old heap word size. Ex. <c>233</c></item> </list> <p>Example:</p> - <p><code type="none">gc_major_end: { cpu_id = 0 }, { pid = "<0.172.0>", reclaimed = 240, heap = 4185, old_heap = 0 }</code></p> + <code type="none">gc_major_end: { cpu_id = 0 }, { pid = "<0.172.0>", reclaimed = 240, heap = 4185, old_heap = 0 }</code> </section> @@ -250,7 +250,7 @@ $ make </code> <item><c>runnable : integer</c> :: Runnable. Ex. <c>1</c></item> </list> <p>Example:</p> - <p><code type="none">scheduler_poll: { cpu_id = 4 }, { scheduler = 1, runnable = 1 }</code></p> + <code type="none">scheduler_poll: { cpu_id = 4 }, { scheduler = 1, runnable = 1 }</code> <p><em>driver_init</em></p> <list type="bulleted"> @@ -260,7 +260,7 @@ $ make </code> <item><c>flags : integer</c> :: Flags. Ex. <c>1</c></item> </list> <p>Example:</p> - <p><code type="none">driver_init: { cpu_id = 2 }, { driver = "caller_drv", major = 3, minor = 3, flags = 1 }</code></p> + <code type="none">driver_init: { cpu_id = 2 }, { driver = "caller_drv", major = 3, minor = 3, flags = 1 }</code> <p><em>driver_start</em></p> <list type="bulleted"> @@ -269,7 +269,7 @@ $ make </code> <item><c>port : string</c> :: Port ID. Ex. <c>"#Port<0.1031>"</c></item> </list> <p>Example:</p> - <p><code type="none">driver_start: { cpu_id = 2 }, { pid = "<0.198.0>", driver = "caller_drv", port = "#Port<0.3676>" }</code></p> + <code type="none">driver_start: { cpu_id = 2 }, { pid = "<0.198.0>", driver = "caller_drv", port = "#Port<0.3676>" }</code> <p><em>driver_output</em></p> <list type="bulleted"> @@ -279,7 +279,7 @@ $ make </code> <item><c>bytes : integer</c> :: Size of data returned. Ex. <c>82</c></item> </list> <p>Example:</p> - <p><code type="none">driver_output: { cpu_id = 2 }, { pid = "<0.198.0>", port = "#Port<0.3677>", driver = "/bin/sh -s unix:cmd", bytes = 36 }</code></p> + <code type="none">driver_output: { cpu_id = 2 }, { pid = "<0.198.0>", port = "#Port<0.3677>", driver = "/bin/sh -s unix:cmd", bytes = 36 }</code> <p><em>driver_outputv</em></p> <list type="bulleted"> @@ -289,7 +289,7 @@ $ make </code> <item><c>bytes : integer</c> :: Size of data returned. Ex. <c>82</c></item> </list> <p>Example:</p> - <p><code type="none">driver_outputv: { cpu_id = 5 }, { pid = "<0.194.0>", port = "#Port<0.3663>", driver = "tcp_inet", bytes = 3 }</code></p> + <code type="none">driver_outputv: { cpu_id = 5 }, { pid = "<0.194.0>", port = "#Port<0.3663>", driver = "tcp_inet", bytes = 3 }</code> <p><em>driver_ready_input</em></p> <list type="bulleted"> @@ -298,7 +298,7 @@ $ make </code> <item><c>driver : string</c> :: Driver name. Ex. <c>"efile"</c></item> </list> <p>Example:</p> - <p><code type="none">driver_ready_input: { cpu_id = 5 }, { pid = "<0.189.0>", port = "#Port<0.3637>", driver = "inet_gethost 4 " }</code></p> + <code type="none">driver_ready_input: { cpu_id = 5 }, { pid = "<0.189.0>", port = "#Port<0.3637>", driver = "inet_gethost 4 " }</code> <p><em>driver_ready_output</em></p> <list type="bulleted"> @@ -307,7 +307,7 @@ $ make </code> <item><c>driver : string</c> :: Driver name. Ex. <c>"efile"</c></item> </list> <p>Example:</p> - <p><code type="none">driver_ready_output: { cpu_id = 5 }, { pid = "<0.194.0>", port = "#Port<0.3663>", driver = "tcp_inet" }</code></p> + <code type="none">driver_ready_output: { cpu_id = 5 }, { pid = "<0.194.0>", port = "#Port<0.3663>", driver = "tcp_inet" }</code> <p><em>driver_timeout</em></p> <list type="bulleted"> @@ -316,14 +316,14 @@ $ make </code> <item><c>driver : string</c> :: Driver name. Ex. <c>"efile"</c></item> </list> <p>Example:</p> - <p><code type="none">driver_timeout: { cpu_id = 5 }, { pid = "<0.196.0>", port = "#Port<0.3664>", driver = "tcp_inet" }</code></p> + <code type="none">driver_timeout: { cpu_id = 5 }, { pid = "<0.196.0>", port = "#Port<0.3664>", driver = "tcp_inet" }</code> <p><em>driver_stop_select</em></p> <list type="bulleted"> <item><c>driver : string</c> :: Driver name. Ex. <c>"efile"</c></item> </list> <p>Example:</p> - <p><code type="none">driver_stop_select: { cpu_id = 5 }, { driver = "unknown" }</code></p> + <code type="none">driver_stop_select: { cpu_id = 5 }, { driver = "unknown" }</code> <p><em>driver_flush</em></p> <list type="bulleted"> @@ -332,7 +332,7 @@ $ make </code> <item><c>driver : string</c> :: Driver name. Ex. <c>"efile"</c></item> </list> <p>Example:</p> - <p><code type="none">driver_flush: { cpu_id = 7 }, { pid = "<0.204.0>", port = "#Port<0.3686>", driver = "tcp_inet" }</code></p> + <code type="none">driver_flush: { cpu_id = 7 }, { pid = "<0.204.0>", port = "#Port<0.3686>", driver = "tcp_inet" }</code> <p><em>driver_stop</em></p> <list type="bulleted"> @@ -341,7 +341,7 @@ $ make </code> <item><c>driver : string</c> :: Driver name. Ex. <c>"efile"</c></item> </list> <p>Example:</p> - <p><code type="none">driver_stop: { cpu_id = 5 }, { pid = "[]", port = "#Port<0.3673>", driver = "efile" }</code></p> + <code type="none">driver_stop: { cpu_id = 5 }, { pid = "[]", port = "#Port<0.3673>", driver = "efile" }</code> <p><em>driver_process_exit</em></p> <list type="bulleted"> @@ -357,7 +357,7 @@ $ make </code> <item><c>driver : string</c> :: Driver name. Ex. <c>"efile"</c></item> </list> <p>Example:</p> - <p><code type="none">driver_ready_async: { cpu_id = 3 }, { pid = "<0.181.0>", port = "#Port<0.3622>", driver = "efile" }</code></p> + <code type="none">driver_ready_async: { cpu_id = 3 }, { pid = "<0.181.0>", port = "#Port<0.3622>", driver = "efile" }</code> <p><em>driver_call</em></p> <list type="bulleted"> @@ -368,7 +368,7 @@ $ make </code> <item><c>bytes : integer</c> :: Size of data returned. Ex. <c>82</c></item> </list> <p>Example:</p> - <p><code type="none">driver_call: { cpu_id = 2 }, { pid = "<0.202.0>", port = "#Port<0.3676>", driver = "caller_drv", command = 0, bytes = 2 }</code></p> + <code type="none">driver_call: { cpu_id = 2 }, { pid = "<0.202.0>", port = "#Port<0.3676>", driver = "caller_drv", command = 0, bytes = 2 }</code> <p><em>driver_control</em></p> <list type="bulleted"> @@ -379,7 +379,7 @@ $ make </code> <item><c>bytes : integer</c> :: Size of data returned. Ex. <c>82</c></item> </list> <p>Example:</p> - <p><code type="none">driver_control: { cpu_id = 3 }, { pid = "<0.32767.8191>", port = "#Port<0.0>", driver = "forker", command = 83, bytes = 32 }</code></p> + <code type="none">driver_control: { cpu_id = 3 }, { pid = "<0.32767.8191>", port = "#Port<0.0>", driver = "forker", command = 83, bytes = 32 }</code> <p><em>aio_pool_get</em></p> <list type="bulleted"> @@ -387,7 +387,7 @@ $ make </code> <item><c>length : integer</c> :: Async queue length. Ex. <c>0</c></item> </list> <p>Example:</p> - <p><code type="none">aio_pool_get: { cpu_id = 4 }, { port = "#Port<0.3614>", length = 0 }</code></p> + <code type="none">aio_pool_get: { cpu_id = 4 }, { port = "#Port<0.3614>", length = 0 }</code> <p><em>aio_pool_put</em></p> <list type="bulleted"> @@ -396,7 +396,7 @@ $ make </code> </list> <p>Async queue length is not defined for <c>put</c> operations.</p> <p>Example:</p> - <p><code type="none">aio_pool_put: { cpu_id = 3 }, { port = "#Port<0.3614>", length = -1 }</code></p> + <code type="none">aio_pool_put: { cpu_id = 3 }, { port = "#Port<0.3614>", length = -1 }</code> <p><em>carrier_create</em></p> <list type="bulleted"> @@ -414,7 +414,7 @@ $ make </code> </list> <p>Example:</p> - <p><code type="none">carrier_create: { cpu_id = 2 }, { type = "ets_alloc", instance = 7, size = 2097152, mbc_carriers = 4, mbc_carriers_size = 3440640, mbc_blocks = 526, mbc_blocks_size = 1278576, sbc_carriers = 0, sbc_carriers_size = 0, sbc_blocks = 0, sbc_blocks_size = 0 }</code></p> + <code type="none">carrier_create: { cpu_id = 2 }, { type = "ets_alloc", instance = 7, size = 2097152, mbc_carriers = 4, mbc_carriers_size = 3440640, mbc_blocks = 526, mbc_blocks_size = 1278576, sbc_carriers = 0, sbc_carriers_size = 0, sbc_blocks = 0, sbc_blocks_size = 0 }</code> <p><em>carrier_destroy</em></p> <list type="bulleted"> @@ -432,7 +432,7 @@ $ make </code> </list> <p>Example:</p> - <p><code type="none">carrier_destroy: { cpu_id = 6 }, { type = "ets_alloc", instance = 7, size = 262144, mbc_carriers = 3, mbc_carriers_size = 3178496, mbc_blocks = 925, mbc_blocks_size = 2305336, sbc_carriers = 0, sbc_carriers_size = 0, sbc_blocks = 0, sbc_blocks_size = 0 }</code></p> + <code type="none">carrier_destroy: { cpu_id = 6 }, { type = "ets_alloc", instance = 7, size = 262144, mbc_carriers = 3, mbc_carriers_size = 3178496, mbc_blocks = 925, mbc_blocks_size = 2305336, sbc_carriers = 0, sbc_carriers_size = 0, sbc_blocks = 0, sbc_blocks_size = 0 }</code> <p><em>carrier_pool_put</em></p> <list type="bulleted"> @@ -441,7 +441,7 @@ $ make </code> <item><c>size : integer</c> :: Carrier size. Ex. <c>262144</c></item> </list> <p>Example:</p> - <p><code type="none">carrier_pool_put: { cpu_id = 3 }, { type = "ets_alloc", instance = 5, size = 1048576 }</code></p> + <code type="none">carrier_pool_put: { cpu_id = 3 }, { type = "ets_alloc", instance = 5, size = 1048576 }</code> <p><em>carrier_pool_get</em></p> <list type="bulleted"> @@ -450,11 +450,9 @@ $ make </code> <item><c>size : integer</c> :: Carrier size. Ex. <c>262144</c></item> </list> <p>Example:</p> - <p><code type="none">carrier_pool_get: { cpu_id = 7 }, { type = "ets_alloc", instance = 4, size = 3208 }</code></p> - + <code type="none">carrier_pool_get: { cpu_id = 7 }, { type = "ets_alloc", instance = 4, size = 3208 }</code> </section> - <section> <title>Examples</title> </section> diff --git a/lib/runtime_tools/doc/src/dbg.xml b/lib/runtime_tools/doc/src/dbg.xml index 0128e23a47..49b11ddc3c 100644 --- a/lib/runtime_tools/doc/src/dbg.xml +++ b/lib/runtime_tools/doc/src/dbg.xml @@ -36,8 +36,8 @@ <modulesummary>The Text Based Trace Facility</modulesummary> <description> <p>This module implements a text based interface to the - <c><seealso marker="erts:erlang#trace-3">trace/3</seealso></c> and the - <c><seealso marker="erts:erlang#trace_pattern-2">trace_pattern/2</seealso></c> BIFs. It makes it + <seealso marker="erts:erlang#trace-3"><c>trace/3</c></seealso> and the + <seealso marker="erts:erlang#trace_pattern-2"><c>trace_pattern/2</c></seealso> BIFs. It makes it possible to trace functions, processes, ports and messages. </p> <p> @@ -185,7 +185,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <item>The corresponding process or port is traced. The process or port may be a remote process or port (on another Erlang node). The node must be in the list of traced nodes (see <seealso marker="#n-1"><c>n/1</c></seealso> - and <c><seealso marker="#tracer-3">tracer/3</seealso></c>).</item> + and <seealso marker="#tracer-3"><c>tracer/3</c></seealso>).</item> <tag><c>all</c></tag> <item>All processes and ports in the system as well as all processes and ports created hereafter are to be traced.</item> @@ -208,22 +208,23 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <tag><c>atom()</c></tag> <item>The process or port with the corresponding registered name is traced. The process or port may be a remote process (on another Erlang node). The node must be - added with the <c><seealso marker="#n-1">n/1</seealso></c> or - <c><seealso marker="#tracer-3">tracer/3</seealso></c> function.</item> + added with the <seealso marker="#n-1"><c>n/1</c></seealso> or + <seealso marker="#tracer-3"><c>tracer/3</c></seealso> function.</item> <tag><c>integer()</c></tag> <item>The process <c><![CDATA[<0.Item.0>]]></c> is traced.</item> <tag><c>{X, Y, Z}</c></tag> <item>The process <c><![CDATA[<X.Y.Z>]]></c> is traced. </item> - <tag><c>string()</c></tag> - <item>If the <c>Item</c> is a string <![CDATA["<X.Y.Z>"]]> - as returned from <c><seealso marker="erts:erlang#pid_to_list-1">pid_to_list/1</seealso></c>, the process - <c><![CDATA[<X.Y.Z>]]></c> is traced. </item> - </taglist> + <tag><c>string()</c></tag> + <item>If the <c>Item</c> is a string <![CDATA["<X.Y.Z>"]]> + as returned from <seealso marker="erts:erlang#pid_to_list-1"><c>pid_to_list/1</c></seealso>, + the process <c><![CDATA[<X.Y.Z>]]></c> is traced. + </item> + </taglist> <p>When enabling an <c>Item</c> that represents a group of processes, the <c>Item</c> is enabled on all nodes added with the - <c><seealso marker="#n-1">n/1</seealso></c> or - <c><seealso marker="#tracer-3">tracer/3</seealso></c> function.</p> + <seealso marker="#n-1"><c>n/1</c></seealso> or + <seealso marker="#tracer-3"><c>tracer/3</c></seealso> function.</p> <p><c>Flags</c> can be a single atom, or a list of flags. The available flags are: @@ -275,7 +276,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <item> <p>This is the same as <c>sol</c>, but only for the first call to - <c><seealso marker="erts:erlang#link-1">link/1</seealso></c> by the traced process.</p> + <seealso marker="erts:erlang#link-1"><c>link/1</c></seealso> by the traced process.</p> </item> <tag><c>all</c></tag> <item> @@ -288,7 +289,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ </item> </taglist> <p>The list can also include any of the flags allowed in - <c><seealso marker="erts:erlang#trace-3">erlang:trace/3</seealso></c></p> + <seealso marker="erts:erlang#trace-3"><c>erlang:trace/3</c></seealso></p> <p>The function returns either an error tuple or a tuple <c>{ok, List}</c>. The <c>List</c> consists of specifications of how many processes and ports that matched (in the @@ -368,11 +369,11 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ please turn to the <em>User's guide</em> part of the online documentation for the runtime system (<em>erts</em>). The - chapter <em><seealso marker="erts:match_spec">Match Specifications in Erlang</seealso></em> + chapter <seealso marker="erts:match_spec"><em>Match Specifications in Erlang</em></seealso> explains the general match specification "language". The most common generic match specifications used can be found as <c>Built-inAlias</c>', see - <c><seealso marker="#ltp-0">ltp/0</seealso></c> below for details. + <seealso marker="#ltp-0"><c>ltp/0</c></seealso> below for details. </p> <p>The Module, Function and/or Arity parts of the tuple may be specified as the atom <c>'_'</c> which is a "wild-card" @@ -380,21 +381,21 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ Module is specified as <c>'_'</c>, the Function and Arity parts have to be specified as '_' too. The same holds for the Functions relation to the Arity.</p> - <p>All nodes added with <c><seealso marker="#n-1">n/1</seealso></c> or - <c><seealso marker="#tracer-3">tracer/3</seealso></c> will + <p>All nodes added with <seealso marker="#n-1"><c>n/1</c></seealso> or + <seealso marker="#tracer-3"><c>tracer/3</c></seealso> will be affected by this call, and if Module is not <c>'_'</c> the module will be loaded on all nodes.</p> <p>The function returns either an error tuple or a tuple <c>{ok, List}</c>. The <c>List</c> consists of specifications of how many functions that matched, in the same way as the processes and ports - are presented in the return value of <c><seealso marker="#p-2">p/2</seealso></c>. </p> + are presented in the return value of <seealso marker="#p-2"><c>p/2</c></seealso>. </p> <p>There may be a tuple <c>{saved, N}</c> in the return value, if the MatchSpec is other than []. The integer <c>N</c> may then be used in subsequent calls to this function and will stand as an "alias" for the given expression. There are also a couple of - built-in aliases for common expressions, see - <c><seealso marker="#ltp-0">ltp/0</seealso></c> below for details.</p> + built-in aliases for common expressions, see + <seealso marker="#ltp-0"><c>ltp/0</c></seealso> below for details.</p> <p>If an error is returned, it can be due to errors in compilation of the match specification. Such errors are presented as a list of tuples <c>{error, string()}</c> where @@ -433,11 +434,55 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <name>tpl({Module, Function, Arity}, MatchSpec) -> {ok, MatchDesc} | {error, term()}</name> <fsummary>Set pattern for traced local (as well as global) function calls</fsummary> <desc> - <p>This function works as <c><seealso marker="#tp-2">tp/2</seealso></c>, but enables + <p>This function works as <seealso marker="#tp-2"><c>tp/2</c></seealso>, but enables tracing for local calls (and local functions) as well as for global calls (and functions).</p> </desc> </func> + + + <func> + <name>tpe(Event, MatchSpec) -> {ok, MatchDesc} | {error, term()}</name> + <fsummary>Set pattern for traced event</fsummary> + <type> + <v>Event = send | 'receive'</v> + <v>MatchSpec = integer() | Built-inAlias | [] | match_spec()</v> + <v>Built-inAlias = x | c | cx</v> + <v>MatchDesc = [MatchInfo]</v> + <v>MatchInfo = {saved, integer()} | MatchNum</v> + <v>MatchNum = {matched, node(), 1} | {matched, node(), 0, RPCError}</v> + </type> + <desc> + <p>This function associates a match specification with trace event + <c>send</c> or <c>'receive'</c>. By default all executed <c>send</c> + and <c>'receive'</c> events are traced if enabled for a process. + A match specification can be used to filter traced events + based on sender, receiver and/or message content.</p> + <p>For a description of the <c>match_spec()</c> syntax, + please turn to the <em>User's guide</em> part of the online + documentation for the runtime system (<em>erts</em>). The + chapter <seealso marker="erts:match_spec"><em>Match Specifications in Erlang</em></seealso> + explains the general match specification "language".</p> + <p>For <c>send</c>, the matching is done on the list <c>[Receiver, Msg]</c>. + <c>Receiver</c> is the process or port identity of the receiver and + <c>Msg</c> is the message term. The pid of the sending process can be + accessed with the guard function <c>self/0</c>.</p> + <p>For <c>'receive'</c>, the matching is done on the list <c>[Node, Sender, Msg]</c>. + <c>Node</c> is the node name of the sender. <c>Sender</c> is the + process or port identity of the sender, or the atom + <c>undefined</c> if the sender is not known (which may + be the case for remote senders). <c>Msg</c> is the + message term. The pid of the receiving process can be + accessed with the guard function <c>self/0</c>.</p> + <p>All nodes added with <seealso marker="#n-1"><c>n/1</c></seealso> or + <seealso marker="#tracer-3"><c>tracer/3</c></seealso> will + be affected by this call.</p> + <p>The return value is the same as for + <seealso marker="#tp-2"><c>tp/2</c></seealso>. The number of matched + events are never larger than 1 as <c>tpe/2</c> does not + accept any form of wildcards for argument <c>Event</c>.</p> + </desc> + </func> <func> <name>ctp()</name> <fsummary>Clear call trace pattern for the specified functions</fsummary> @@ -480,10 +525,10 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <p>This function disables call tracing on the specified functions. The semantics of the parameter is the same as for the corresponding function specification in - <c><seealso marker="#tp-2">tp/2</seealso></c> or <c><seealso marker="#tpl-2">tpl/2</seealso></c>. Both local and global call trace + <seealso marker="#tp-2"><c>tp/2</c></seealso> or <seealso marker="#tpl-2"><c>tpl/2</c></seealso>. Both local and global call trace is disabled. </p> <p>The return value reflects how many functions that matched, - and is constructed as described in <c><seealso marker="#tp-2">tp/2</seealso></c>. No tuple + and is constructed as described in <seealso marker="#tp-2"><c>tp/2</c></seealso>. No tuple <c>{saved, N}</c> is however ever returned (for obvious reasons).</p> </desc> </func> @@ -519,8 +564,9 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <name>ctpl({Module, Function, Arity}) -> {ok, MatchDesc} | {error, term()}</name> <fsummary>Clear call trace pattern for the specified functions</fsummary> <desc> - <p>This function works as <c><seealso marker="#ctp-1">ctp/1</seealso></c>, but only disables - tracing set up with <c><seealso marker="#tpl-2">tpl/2</seealso></c> (not with <c><seealso marker="#tp-2">tp/2</seealso></c>).</p> + <p>This function works as <seealso marker="#ctp-1"><c>ctp/1</c></seealso>, but only disables + tracing set up with <seealso marker="#tpl-2"><c>tpl/2</c></seealso> + (not with <seealso marker="#tp-2"><c>tp/2</c></seealso>).</p> </desc> </func> <func> @@ -555,8 +601,25 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <name>ctpg({Module, Function, Arity}) -> {ok, MatchDesc} | {error, term()}</name> <fsummary>Clear call trace pattern for the specified functions</fsummary> <desc> - <p>This function works as <c><seealso marker="#ctp-1">ctp/1</seealso></c>, but only disables - tracing set up with <c><seealso marker="#tp-2">tp/2</seealso></c> (not with <c><seealso marker="#tpl-2">tpl/2</seealso></c>).</p> + <p>This function works as <seealso marker="#ctp-1"><c>ctp/1</c></seealso>, but only disables + tracing set up with <seealso marker="#tp-2"><c>tp/2</c></seealso> + (not with <seealso marker="#tpl-2"><c>tpl/2</c></seealso>).</p> + </desc> + </func> + <func> + <name>ctpe(Event) -> {ok, MatchDesc} | {error, term()}</name> + <fsummary>Clear trace pattern for the specified event</fsummary> + <type> + <v>Event = send | 'receive'</v> + <v>MatchDesc = [MatchNum]</v> + <v>MatchNum = {matched, node(), 1} | {matched, node(), 0, RPCError}</v> + </type> + <desc> + <p>This function clears match specifications for the specified + trace event (<c>send</c> or <c>'receive'</c>). It will revert back + to the default behavior of tracing all triggered events.</p> + <p>The return value follow the same style as for + <seealso marker="#ctp-1"><c>ctp/1</c></seealso>.</p> </desc> </func> <func> @@ -565,13 +628,14 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <desc> <p>Use this function to recall all match specifications previously used in the session (i. e. previously saved during calls - to <c><seealso marker="#tp-2">tp/2</seealso></c>, and built-in match specifications. + to <seealso marker="#tp-2"><c>tp/2</c></seealso>, and built-in match specifications. This is very useful, as a complicated match_spec can be quite awkward to write. Note that the - match specifications are lost if <c><seealso marker="#stop-0">stop/0</seealso></c> is called.</p> + match specifications are lost if <seealso marker="#stop-0"><c>stop/0</c></seealso> is called.</p> <p>Match specifications used can be saved in a file (if a read-write file system is present) for use in later - debugging sessions, see <c><seealso marker="#wtp-1">wtp/1</seealso></c> and <c><seealso marker="#rtp-1">rtp/1</seealso></c></p> + debugging sessions, see <seealso marker="#wtp-1"><c>wtp/1</c></seealso> + and <seealso marker="#rtp-1"><c>rtp/1</c></seealso></p> <p>There are three built-in trace patterns: <c>exception_trace</c>, <c>caller_trace</c> and <c>caller_exception_trace</c> (or <c>x</c>, <c>c</c> and @@ -594,10 +658,10 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <fsummary>Delete all saved match specifications.</fsummary> <desc> <p>Use this function to "forget" all match specifications - saved during calls to <c><seealso marker="#tp-2">tp/2</seealso></c>. - This is useful when one wants to restore other match - specifications from a file with <c><seealso marker="#rtp-1">rtp/1</seealso></c>. Use - <c><seealso marker="#dtp-1">dtp/1</seealso></c> to delete specific saved match specifications. </p> + saved during calls to <seealso marker="#tp-2"><c>tp/2</c></seealso>. + This is useful when one wants to restore other match + specifications from a file with <seealso marker="#rtp-1"><c>rtp/1</c></seealso>. Use + <seealso marker="#dtp-1"><c>dtp/1</c></seealso> to delete specific saved match specifications.</p> </desc> </func> <func> @@ -608,7 +672,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ </type> <desc> <p>Use this function to "forget" a specific match specification - saved during calls to <c><seealso marker="#tp-2">tp/2</seealso></c>.</p> + saved during calls to <seealso marker="#tp-2"><c>tp/2</c></seealso>.</p> </desc> </func> <func> @@ -620,12 +684,12 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ </type> <desc> <p>This function will save all match specifications saved - during the session (during calls to <c><seealso marker="#tp-2">tp/2</seealso></c>) + during the session (during calls to <seealso marker="#tp-2"><c>tp/2</c></seealso>) and built-in match specifications in a text file with the name designated by <c>Name</c>. The format of the file is textual, why it can be edited with an ordinary text editor, and then restored with - <c><seealso marker="#rtp-1">rtp/1</seealso></c>. </p> + <seealso marker="#rtp-1"><c>rtp/1</c></seealso>. </p> <p>Each match spec in the file ends with a full stop (<c>.</c>) and new (syntactically correct) match specifications can be added to the file manually.</p> @@ -643,7 +707,8 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ </type> <desc> <p>This function reads match specifications from a file - (possibly) generated by the <c><seealso marker="#wtp-1">wtp/1</seealso></c> function. It checks + (possibly) generated by the <seealso marker="#wtp-1"><c>wtp/1</c></seealso> + function. It checks the syntax of all match specifications and verifies that they are correct. The error handling principle is "all or nothing", i. e. if some of the match specifications are @@ -651,7 +716,8 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ saved match specifications for the running system. </p> <p>The match specifications in the file are <em>merged</em> with the current match specifications, so that no duplicates - are generated. Use <c><seealso marker="#ltp-0">ltp/0</seealso></c> to see what numbers were + are generated. Use <seealso marker="#ltp-0"><c>ltp/0</c></seealso> + to see what numbers were assigned to the specifications from the file.</p> <p>The function will return an error, either due to I/O problems (like a non existing or non readable file) or due @@ -670,9 +736,9 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ </type> <desc> <p>The <c>dbg</c> server keeps a list of nodes where tracing - should be performed. Whenever a <c><seealso marker="#tp-2">tp/2</seealso></c> call or a - <c><seealso marker="#p-2">p/2</seealso></c> call is made, it is executed for all nodes in this - list including the local node (except for <c><seealso marker="#p-2">p/2</seealso></c> with a + should be performed. Whenever a <seealso marker="#tp-2"><c>tp/2</c></seealso> call or a + <seealso marker="#p-2"><c>p/2</c></seealso> call is made, it is executed for all nodes in this + list including the local node (except for <seealso marker="#p-2"><c>p/2</c></seealso> with a specific <c>pid()</c> or <c>port()</c> as first argument, in which case the command is executed only on the node where the designated process or port resides). @@ -684,7 +750,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ distribution). If no tracer process is running on the local node, the error reason <c>no_local_tracer</c> is returned. The tracer process on the local node must be started with the - <c><seealso marker="#tracer-2">tracer/0/2</seealso></c> function. + <seealso marker="#tracer-2"><c>tracer/0/2</c></seealso> function. </p> <p>If <c>Nodename</c> is the local node, the error reason <c>cant_add_local_node</c> is returned. @@ -694,7 +760,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ a tracer process. The error reason <c>cant_trace_remote_pid_to_local_port</c> is returned. A trace port can however be started on the remote node with the - <c><seealso marker="#tracer-3">tracer/3</seealso></c> function. + <seealso marker="#tracer-3"><c>tracer/3</c></seealso> function. </p> <p>The function will also return an error if the node <c>Nodename</c> is not reachable.</p> @@ -708,9 +774,10 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ </type> <desc> <p>Clears a node from the list of traced nodes. Subsequent - calls to <c><seealso marker="#tp-2">tp/2</seealso></c> and <c><seealso marker="#p-2">p/2</seealso></c> will not consider that - node, but tracing already activated on the node will continue - to be in effect.</p> + calls to <seealso marker="#tp-2"><c>tp/2</c></seealso> and + <seealso marker="#p-2"><c>p/2</c></seealso> will not consider that + node, but tracing already activated on the node will continue + to be in effect.</p> <p>Returns <c>ok</c>, cannot fail.</p> </desc> </func> @@ -727,14 +794,14 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <desc> <p>This function starts a server on the local node that will be the recipient of all trace messages. All subsequent calls - to <c><seealso marker="#p-2">p/2</seealso></c> will result in messages sent to the newly + to <seealso marker="#p-2"><c>p/2</c></seealso> will result in messages sent to the newly started trace server.</p> <p>A trace server started in this way will simply display the trace messages in a formatted way in the Erlang shell - (i. e. use io:format). See <c><seealso marker="#tracer-2">tracer/2</seealso></c> for a description - of how the trace message handler can be customized. + (i. e. use io:format). See <seealso marker="#tracer-2"><c>tracer/2</c></seealso> + for a description of how the trace message handler can be customized. </p> - <p>To start a similar tracer on a remote node, use <c><seealso marker="#n-1">n/1</seealso></c>.</p> + <p>To start a similar tracer on a remote node, use <seealso marker="#n-1"><c>n/1</c></seealso>.</p> </desc> </func> <func> @@ -758,9 +825,9 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ by a receiving process (<c>process</c>), by a tracer port (<c>port</c>) or by a tracer module (<c>module</c>). For a description about tracer ports see - <c><seealso marker="#trace_port-2">trace_port/2</seealso></c> + <seealso marker="#trace_port-2"><c>trace_port/2</c></seealso> and for a tracer modules see - <c><seealso marker="erts:erl_tracer">erl_tracer</seealso>.</c> + <seealso marker="erts:erl_tracer"><c>erl_tracer</c></seealso>. </p> <p>If <c>Type</c> is <c>process</c>, a message handler function can be specified (<c>HandlerSpec</c>). The handler function, which @@ -776,10 +843,10 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <p>If <c>Type</c> is <c>port</c>, then the second parameter should be a <em>fun</em> which takes no arguments and returns a newly opened trace port when called. Such a <em>fun</em> is - preferably generated by calling <c><seealso marker="#trace_port-2">trace_port/2</seealso></c>. + preferably generated by calling <seealso marker="#trace_port-2"><c>trace_port/2</c></seealso>. </p> <p>if <c>Type</c> is <c>module</c>, then the second parameter should - be either a tuple describing the <c><seealso marker="erts:erl_tracer">erl_tracer</seealso></c> + be either a tuple describing the <seealso marker="erts:erl_tracer"><c>erl_tracer</c></seealso> module to be used for tracing and the state to be used for that tracer module or a fun returning the same tuple.</p> <p>If an error is returned, it can either be due to a tracer @@ -787,7 +854,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ due to the <c>HandlerFun</c> throwing an exception. </p> <p>To start a similar tracer on a remote node, use - <c><seealso marker="#tracer-3">tracer/3</seealso></c>. + <seealso marker="#tracer-3"><c>tracer/3</c></seealso>. </p> </desc> </func> @@ -798,19 +865,19 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <v>Nodename = atom()</v> </type> <desc> - <p>This function is equivalent to <c><seealso marker="#tracer-2">tracer/2</seealso></c>, but acts on + <p>This function is equivalent to <seealso marker="#tracer-2"><c>tracer/2</c></seealso>, but acts on the given node. A tracer is started on the node (<c>Nodename</c>) and the node is added to the list of traced nodes. </p> <note> - <p>This function is not equivalent to <c><seealso marker="#n-1">n/1</seealso></c>. While - <c><seealso marker="#n-1">n/1</seealso></c> starts a process tracer which redirects all trace + <p>This function is not equivalent to <seealso marker="#n-1"><c>n/1</c></seealso>. While + <seealso marker="#n-1"><c>n/1</c></seealso> starts a process tracer which redirects all trace information to a process tracer on the local node (i.e. the - trace control node), <c><seealso marker="#tracer-3">tracer/3</seealso></c> starts a tracer of any + trace control node), <seealso marker="#tracer-3"><c>tracer/3</c></seealso> starts a tracer of any type which is independent of the tracer on the trace control node.</p> </note> - <p>For details, see <c><seealso marker="#tracer-2">tracer/2</seealso></c>.</p> + <p>For details, see <seealso marker="#tracer-2"><c>tracer/2</c></seealso>.</p> </desc> </func> <func> @@ -842,9 +909,9 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <c>file</c> and the <c>ip</c> trace drivers. The file driver sends all trace messages into one or several binary files, from where they later can be fetched and processed with the - <c><seealso marker="#trace_client-2">trace_client/2</seealso></c> function. The ip driver opens a TCP/IP + <seealso marker="#trace_client-2"><c>trace_client/2</c></seealso> function. The ip driver opens a TCP/IP port where it listens for connections. When a client - (preferably started by calling <c><seealso marker="#trace_client-2">trace_client/2</seealso></c> on + (preferably started by calling <seealso marker="#trace_client-2"><c>trace_client/2</c></seealso> on another Erlang node) connects, all trace messages are sent over the IP network for further processing by the remote client. </p> @@ -883,7 +950,8 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ as fast as they are produced by the runtime system, a special message is sent, which indicates how many messages that are dropped. That message will arrive at the handler function - specified in <c><seealso marker="#trace_client-3">trace_client/3</seealso></c> as the tuple <c>{drop, N}</c> where <c>N</c> is the number of consecutive messages + specified in <seealso marker="#trace_client-3"><c>trace_client/3</c></seealso> + as the tuple <c>{drop, N}</c> where <c>N</c> is the number of consecutive messages dropped. In case of heavy tracing, drop's are likely to occur, and they surely occur if no client is reading the trace messages.</p> @@ -960,8 +1028,8 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <desc> <p>This function starts a trace client that reads the output created by a trace port driver and handles it in mostly the - same way as a tracer process created by the <c><seealso marker="#tracer-0">tracer/0</seealso></c> - function.</p> + same way as a tracer process created by the + <seealso marker="#tracer-0"><c>tracer/0</c></seealso> function.</p> <p>If <c>Type</c> is <c>file</c>, the client reads all trace messages stored in the file named <c>Filename</c> or specified by <c>WrapFilesSpec</c> (must be the same as used @@ -972,7 +1040,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <p>If <c>Type</c> is <c>follow_file</c>, the client behaves as in the <c>file</c> case, but keeps trying to read (and process) more data - from the file until stopped by <c><seealso marker="#stop_trace_client-1">stop_trace_client/1</seealso></c>. + from the file until stopped by <seealso marker="#stop_trace_client-1"><c>stop_trace_client/1</c></seealso>. <c>WrapFilesSpec</c> is not allowed as second argument for this <c>Type</c>.</p> <p>If <c>Type</c> is <c>ip</c>, the client connects to the @@ -1028,10 +1096,10 @@ hello</pre> <v>InitialData = term()</v> </type> <desc> - <p>This function works exactly as <c><seealso marker="#trace_client-2">trace_client/2</seealso></c>, but - allows you to write your own handler function. The handler + <p>This function works exactly as <seealso marker="#trace_client-2"><c>trace_client/2</c></seealso>, + but allows you to write your own handler function. The handler function works mostly as the one described in - <c><seealso marker="#tracer-2">tracer/2</seealso></c>, but will also have to be prepared to handle + <seealso marker="#tracer-2"><c>tracer/2</c></seealso>, but will also have to be prepared to handle trace messages of the form <c>{drop, N}</c>, where <c>N</c> is the number of dropped messages. This pseudo trace message will only occur if the ip trace driver is used.</p> @@ -1050,7 +1118,8 @@ hello</pre> <desc> <p>This function shuts down a previously started trace client. The <c>Pid</c> argument is the process id returned - from the <c><seealso marker="#trace_client-2">trace_client/2</seealso></c> or <c><seealso marker="#trace_client-3">trace_client/3</seealso></c> call.</p> + from the <seealso marker="#trace_client-2"><c>trace_client/2</c></seealso> + or <seealso marker="#trace_client-3"><c>trace_client/3</c></seealso> call.</p> </desc> </func> <func> @@ -1203,7 +1272,7 @@ SeqTrace [0]: (<0.30.0>) <0.25.0> ! {dbg,{ok,<0.31.0>}} [Serial: {4,5}] of causing a deadlock. This will happen if a group leader process generates a trace message and the tracer process, by calling the trace handler function, sends an IO request to the same group leader. The problem can only occur if the trace handler - prints to tty using an <c>io</c> function such as <c><seealso marker="stdlib:io#format-2">format/2</seealso></c>. + prints to tty using an <c>io</c> function such as <seealso marker="stdlib:io#format-2"><c>format/2</c></seealso>. Note that when <c>dbg:p(all,call)</c> is called, IO processes are also traced. Here's an example:</p> diff --git a/lib/runtime_tools/src/appmon_info.erl b/lib/runtime_tools/src/appmon_info.erl index fead724373..b5500085a3 100644 --- a/lib/runtime_tools/src/appmon_info.erl +++ b/lib/runtime_tools/src/appmon_info.erl @@ -307,9 +307,10 @@ do_work(Key, State) -> {Cmd, Aux, From, _OldRef, Old, Opts} = retrieve(WorkStore, Key), {ok, Result} = do_work2(Cmd, Aux, From, Old, Opts), if - Result==Old -> ok; - true -> - From ! {delivery, self(), Cmd, Aux, Result} + Result==Old -> ok; + true -> + From ! {delivery, self(), Cmd, Aux, Result}, + ok end, case get_opt(timeout, Opts) of at_most_once -> @@ -393,7 +394,7 @@ del_task(Key, WorkStore) -> {_Cmd, _Aux, _From, Ref, _Old, Opts} -> if Ref /= nil -> - timer:cancel(Ref), + {ok,_} = timer:cancel(Ref), receive {do_it, Key} -> Opts diff --git a/lib/runtime_tools/src/dbg.erl b/lib/runtime_tools/src/dbg.erl index 1620f52789..8cdb5a43e3 100644 --- a/lib/runtime_tools/src/dbg.erl +++ b/lib/runtime_tools/src/dbg.erl @@ -217,7 +217,7 @@ ctpg({_Module, _Function, _Arity} = X) -> do_ctp(X,[global]). do_ctp({Module, Function, Arity},[]) -> - do_ctp({Module, Function, Arity},[global]), + {ok,_} = do_ctp({Module, Function, Arity},[global]), do_ctp({Module, Function, Arity},[local]); do_ctp({_Module, _Function, _Arity}=MFA,Flags) -> Nodes = req(get_nodes), @@ -271,8 +271,7 @@ wtp(FileName) -> ok end, []), - file:close(File), - ok + ok = file:close(File) end. %% @@ -600,7 +599,7 @@ stop() -> end. stop_clear() -> - ctp(), + {ok, _} = ctp(), stop(). %%% Calling the server. @@ -791,7 +790,8 @@ loop({C,T}=SurviveLinks, Table) -> end. reply(Pid, Reply) -> - Pid ! {dbg,Reply}. + Pid ! {dbg,Reply}, + ok. %%% A process-based tracer. @@ -944,9 +944,11 @@ do_relay(Parent,RelP) -> case RelP of {Type,Data} -> {ok,Tracer} = remote_tracer(Type,Data), - Parent ! {started,Tracer}; + Parent ! {started,Tracer}, + ok; Pid when is_pid(Pid) -> - Parent ! {started,self()} + Parent ! {started,self()}, + ok end, do_relay_1(RelP). @@ -1366,7 +1368,7 @@ mk_reader_wrap([_Hd | Tail] = WrapFiles, File) -> {ok, Term} -> [Term | mk_reader_wrap(WrapFiles, File)]; eof -> - file:close(File), + ok = file:close(File), case Tail of [_|_] -> mk_reader_wrap(Tail); diff --git a/lib/runtime_tools/src/erts_alloc_config.erl b/lib/runtime_tools/src/erts_alloc_config.erl index e94cced911..514530332c 100644 --- a/lib/runtime_tools/src/erts_alloc_config.erl +++ b/lib/runtime_tools/src/erts_alloc_config.erl @@ -128,7 +128,7 @@ make_config(FileName) when is_list(FileName) -> case file:open(FileName, [write]) of {ok, IODev} -> Res = req({make_config, IODev}), - file:close(IODev), + ok = file:close(IODev), Res; Error -> Error @@ -200,9 +200,11 @@ server_loop(State) -> Conf = #conf{segments = ?MBC_MSEG_LIMIT, format_to = IODev}, Res = mk_config(Conf, State#state.alloc), - From ! {response, Ref, Res}; + From ! {response, Ref, Res}, + ok; _ -> - From ! {response, Ref, no_scenario_saved} + From ! {response, Ref, no_scenario_saved}, + ok end, State; {request, From, Ref, stop} -> diff --git a/lib/runtime_tools/src/observer_backend.erl b/lib/runtime_tools/src/observer_backend.erl index 30df3b0b8b..66653c5b7f 100644 --- a/lib/runtime_tools/src/observer_backend.erl +++ b/lib/runtime_tools/src/observer_backend.erl @@ -259,7 +259,8 @@ etop_collect(Collector) -> case SchedulerWallTime of undefined -> - spawn(fun() -> flag_holder_proc(Collector) end); + spawn(fun() -> flag_holder_proc(Collector) end), + ok; _ -> ok end, @@ -334,8 +335,8 @@ ttb_init_node(MetaFile_0,PI,Traci) -> MetaPid ! {metadata,Traci}, case PI of true -> - Proci = pnames(), - MetaPid ! {metadata,Proci}; + MetaPid ! {metadata,pnames()}, + ok; false -> ok end, @@ -354,7 +355,8 @@ ttb_meta_tracer(MetaFile,PI,Parent,SessionData) -> erlang:trace_pattern({erlang,spawn_link,3},ReturnMS,[meta]), erlang:trace_pattern({erlang,spawn_opt,1},ReturnMS,[meta]), erlang:trace_pattern({erlang,register,2},[],[meta]), - erlang:trace_pattern({global,register_name,2},[],[meta]); + erlang:trace_pattern({global,register_name,2},[],[meta]), + ok; false -> ok end, @@ -362,7 +364,8 @@ ttb_meta_tracer(MetaFile,PI,Parent,SessionData) -> case proplists:get_value(overload_check, SessionData) of {Ms, M, F} -> catch M:F(init), - erlang:send_after(Ms, self(), overload_check); + erlang:send_after(Ms, self(), overload_check), + ok; _ -> ok end, @@ -371,10 +374,10 @@ ttb_meta_tracer(MetaFile,PI,Parent,SessionData) -> ttb_meta_tracer_loop(MetaFile,PI,Acc,State) -> receive {trace_ts,_,call,{erlang,register,[Name,Pid]},_} -> - ttb_store_meta({pid,{Pid,Name}},MetaFile), + ok = ttb_store_meta({pid,{Pid,Name}},MetaFile), ttb_meta_tracer_loop(MetaFile,PI,Acc,State); {trace_ts,_,call,{global,register_name,[Name,Pid]},_} -> - ttb_store_meta({pid,{Pid,{global,Name}}},MetaFile), + ok = ttb_store_meta({pid,{Pid,{global,Name}}},MetaFile), ttb_meta_tracer_loop(MetaFile,PI,Acc,State); {trace_ts,CallingPid,call,{erlang,spawn_opt,[{M,F,Args,_}]},_} -> MFA = {M,F,length(Args)}, @@ -390,7 +393,7 @@ ttb_meta_tracer_loop(MetaFile,PI,Acc,State) -> NewAcc = dict:update(CallingPid, fun([H|T]) -> - ttb_store_meta({pid,{NewPid,H}},MetaFile), + ok = ttb_store_meta({pid,{NewPid,H}},MetaFile), T end, Acc), @@ -408,22 +411,22 @@ ttb_meta_tracer_loop(MetaFile,PI,Acc,State) -> NewAcc = dict:update(CallingPid, fun([H|T]) -> - ttb_store_meta({pid,{NewPid,H}},MetaFile), + ok = ttb_store_meta({pid,{NewPid,H}},MetaFile), T end, Acc), ttb_meta_tracer_loop(MetaFile,PI,NewAcc,State); {metadata,Data} when is_list(Data) -> - ttb_store_meta(Data,MetaFile), + ok = ttb_store_meta(Data,MetaFile), ttb_meta_tracer_loop(MetaFile,PI,Acc,State); {metadata,Key,Fun} when is_function(Fun) -> - ttb_store_meta([{Key,Fun()}],MetaFile), + ok = ttb_store_meta([{Key,Fun()}],MetaFile), ttb_meta_tracer_loop(MetaFile,PI,Acc,State); {metadata,Key,What} -> - ttb_store_meta([{Key,What}],MetaFile), + ok = ttb_store_meta([{Key,What}],MetaFile), ttb_meta_tracer_loop(MetaFile,PI,Acc,State); overload_check -> {Ms, M, F} = proplists:get_value(overload_check, State), @@ -439,7 +442,7 @@ ttb_meta_tracer_loop(MetaFile,PI,Acc,State) -> ttb_meta_tracer_loop(MetaFile,PI,Acc, State) end; {'DOWN', _, _, _, _} -> - stop_seq_trace(), + _ = stop_seq_trace(), self() ! stop, ttb_meta_tracer_loop(MetaFile,PI,Acc, State); stop when PI=:=true -> @@ -528,7 +531,7 @@ ttb_store_meta(Data,MetaFile) -> ttb_store_meta([Data],MetaFile). ttb_write_binary(Fd,[H|T]) -> - file:write(Fd,ttb_make_binary(H)), + ok = file:write(Fd,ttb_make_binary(H)), ttb_write_binary(Fd,T); ttb_write_binary(_Fd,[]) -> ok. @@ -585,9 +588,9 @@ ttb_fetch(MetaFile,{Port,Host}) -> send_files({Sock,Host},[File|Files]) -> {ok,Fd} = file:open(File,[raw,read,binary]), - gen_tcp:send(Sock,<<1,(list_to_binary(filename:basename(File)))/binary>>), + ok = gen_tcp:send(Sock,<<1,(list_to_binary(filename:basename(File)))/binary>>), send_chunks(Sock,Fd), - file:delete(File), + ok = file:delete(File), send_files({Sock,Host},Files); send_files({_Sock,_Host},[]) -> done. diff --git a/lib/runtime_tools/src/percept_profile.erl b/lib/runtime_tools/src/percept_profile.erl index ceec4d3b89..1e8e913b80 100644 --- a/lib/runtime_tools/src/percept_profile.erl +++ b/lib/runtime_tools/src/percept_profile.erl @@ -87,7 +87,7 @@ start(Filename, Options) -> start(Filename, {Module, Function, Args}, Options) -> case whereis(percept_port) of undefined -> - profile_to_file(Filename, Options), + {ok, _} = profile_to_file(Filename, Options), erlang:apply(Module, Function, Args), stop(); Port -> @@ -113,7 +113,7 @@ deliver_all_trace() -> -spec stop() -> 'ok' | {'error', 'not_started'}. stop() -> - erlang:system_profile(undefined, [runnable_ports, runnable_procs]), + _ = erlang:system_profile(undefined, [runnable_ports, runnable_procs]), erlang:trace(all, false, [procs, ports, timestamp]), deliver_all_trace(), case whereis(percept_port) of @@ -158,7 +158,8 @@ set_tracer(Port, Opts) -> {TOpts, POpts} = parse_profile_options(Opts), % Setup profiling and tracing erlang:trace(all, true, [{tracer, Port}, timestamp | TOpts]), - erlang:system_profile(Port, POpts). + _ = erlang:system_profile(Port, POpts), + ok. %% parse_profile_options diff --git a/lib/runtime_tools/src/system_information.erl b/lib/runtime_tools/src/system_information.erl index 1add01612d..df25297eb9 100644 --- a/lib/runtime_tools/src/system_information.erl +++ b/lib/runtime_tools/src/system_information.erl @@ -31,6 +31,7 @@ -export([report/0, from_file/1, to_file/1]). + -export([start/0, stop/0, load_report/0, load_report/2, applications/0, applications/1, @@ -64,6 +65,7 @@ start() -> gen_server:start({local, ?SERVER}, ?MODULE, [], []). + stop() -> gen_server:call(?SERVER, stop, infinity). @@ -71,7 +73,7 @@ load_report() -> load_report(data, report()). load_report(file, File) -> load_report(data, from_file(File)); load_report(data, Report) -> - start(), gen_server:call(?SERVER, {load_report, Report}, infinity). + ok = start_internal(), gen_server:call(?SERVER, {load_report, Report}, infinity). report() -> [ {init_arguments, init:get_arguments()}, @@ -219,6 +221,13 @@ code_change(_OldVsn, State, _Extra) -> %% Internal functions %%=================================================================== +start_internal() -> + case start() of + {ok,_} -> ok; + {error, {already_started,_}} -> ok; + Error -> Error + end. + %% handle report values get_value([], Data) -> Data; diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl index 946adcf384..e952a333ff 100644 --- a/lib/ssh/src/ssh_connection_handler.erl +++ b/lib/ssh/src/ssh_connection_handler.erl @@ -1514,39 +1514,35 @@ call(FsmPid, Event, Timeout) -> end. -handle_connection_msg(Msg, StateName, State0 = - #data{starter = User, - connection_state = Connection0, - event_queue = Qev0}) -> +handle_connection_msg(Msg, StateName, D0 = #data{starter = User, + connection_state = Connection0, + event_queue = Qev0}) -> Renegotiation = renegotiation(StateName), Role = role(StateName), - try ssh_connection:handle_msg(Msg, Connection0, Role) of + try ssh_connection:handle_msg(Msg, Connection0, Role) of {{replies, Replies}, Connection} -> - case StateName of - {connected,_} -> - {Repls, State} = send_replies(Replies, - State0#data{connection_state=Connection}), - {keep_state, State, Repls}; - _ -> - {ConnReplies, Replies} = - lists:splitwith(fun not_connected_filter/1, Replies), - {Repls, State} = send_replies(Replies, - State0#data{event_queue = Qev0 ++ ConnReplies}), - {keep_state, State, Repls} - end; + {Repls, D} = + case StateName of + {connected,_} -> + send_replies(Replies, D0#data{connection_state=Connection}); + _ -> + {ConnReplies, NonConnReplies} = lists:splitwith(fun not_connected_filter/1, Replies), + send_replies(NonConnReplies, D0#data{event_queue = Qev0 ++ ConnReplies}) + end, + {keep_state, D, Repls}; {noreply, Connection} -> - {keep_state, State0#data{connection_state = Connection}}; + {keep_state, D0#data{connection_state = Connection}}; {disconnect, Reason0, {{replies, Replies}, Connection}} -> - {Repls,State} = send_replies(Replies, State0#data{connection_state = Connection}), + {Repls, D} = send_replies(Replies, D0#data{connection_state = Connection}), case {Reason0,Role} of {{_, Reason}, client} when ((StateName =/= {connected,client}) and (not Renegotiation)) -> User ! {self(), not_connected, Reason}; _ -> ok end, - {stop_and_reply, {shutdown,normal}, Repls, State#data{connection_state = Connection}} + {stop_and_reply, {shutdown,normal}, Repls, D#data{connection_state = Connection}} catch _:Error -> @@ -1555,8 +1551,8 @@ handle_connection_msg(Msg, StateName, State0 = #ssh_msg_disconnect{code = ?SSH_DISCONNECT_BY_APPLICATION, description = "Internal error"}, Connection0, Role), - {Repls,State} = send_replies(Replies, State0#data{connection_state = Connection}), - {stop_and_reply, {shutdown,Error}, Repls, State#data{connection_state = Connection}} + {Repls, D} = send_replies(Replies, D0#data{connection_state = Connection}), + {stop_and_reply, {shutdown,Error}, Repls, D#data{connection_state = Connection}} end. diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index dbbb25025c..51732b4a59 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -400,24 +400,23 @@ negotiated_next_protocol(Socket) -> end. %%-------------------------------------------------------------------- +-spec cipher_suites() -> [ssl_cipher:erl_cipher_suite()] | [string()]. +%%-------------------------------------------------------------------- +cipher_suites() -> + cipher_suites(erlang). +%%-------------------------------------------------------------------- -spec cipher_suites(erlang | openssl | all) -> [ssl_cipher:erl_cipher_suite()] | [string()]. %% Description: Returns all supported cipher suites. %%-------------------------------------------------------------------- cipher_suites(erlang) -> - Version = tls_record:highest_protocol_version([]), - ssl_cipher:filter_suites([ssl_cipher:erl_suite_definition(S) - || S <- ssl_cipher:suites(Version)]); + [ssl_cipher:erl_suite_definition(Suite) || Suite <- available_suites(default)]; + cipher_suites(openssl) -> - Version = tls_record:highest_protocol_version([]), - [ssl_cipher:openssl_suite_name(S) - || S <- ssl_cipher:filter_suites(ssl_cipher:suites(Version))]; + [ssl_cipher:openssl_suite_name(Suite) || Suite <- available_suites(default)]; + cipher_suites(all) -> - Version = tls_record:highest_protocol_version([]), - ssl_cipher:filter_suites([ssl_cipher:erl_suite_definition(S) - || S <-ssl_cipher:all_suites(Version)]). -cipher_suites() -> - cipher_suites(erlang). + [ssl_cipher:erl_suite_definition(Suite) || Suite <- available_suites(all)]. %%-------------------------------------------------------------------- -spec getopts(#sslsocket{}, [gen_tcp:option_name()]) -> @@ -584,6 +583,16 @@ format_error(Error) -> %%%-------------------------------------------------------------- %%% Internal functions %%%-------------------------------------------------------------------- + +%% Possible filters out suites not supported by crypto +available_suites(default) -> + Version = tls_record:highest_protocol_version([]), + ssl_cipher:filter_suites(ssl_cipher:suites(Version)); + +available_suites(all) -> + Version = tls_record:highest_protocol_version([]), + ssl_cipher:filter_suites(ssl_cipher:all_suites(Version)). + do_connect(Address, Port, #config{transport_info = CbInfo, inet_user = UserOpts, ssl = SslOpts, emulated = EmOpts, inet_ssl = SocketOpts, connection_cb = ConnetionCb}, diff --git a/lib/ssl/src/tls_handshake.erl b/lib/ssl/src/tls_handshake.erl index 102dbba198..f34eebb0e4 100644 --- a/lib/ssl/src/tls_handshake.erl +++ b/lib/ssl/src/tls_handshake.erl @@ -283,7 +283,8 @@ available_signature_algs(undefined, SupportedHashSigns, _, {Major, Minor}) when SupportedHashSigns; available_signature_algs(#hash_sign_algos{hash_sign_algos = ClientHashSigns}, SupportedHashSigns, _, {Major, Minor}) when (Major >= 3) andalso (Minor >= 3) -> - ordsets:intersection(ClientHashSigns, SupportedHashSigns); + sets:to_list(sets:intersection(sets:from_list(ClientHashSigns), + sets:from_list(SupportedHashSigns))); available_signature_algs(_, _, _, _) -> undefined. diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 38cc3532d8..02fdc4330f 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -386,11 +386,11 @@ cert_options(Config) -> SNIServerBCertFile = filename:join([?config(priv_dir, Config), "b.server", "cert.pem"]), SNIServerBKeyFile = filename:join([?config(priv_dir, Config), "b.server", "key.pem"]), [{client_opts, []}, - {client_verification_opts, [{cacertfile, ClientCaCertFile}, + {client_verification_opts, [{cacertfile, ServerCaCertFile}, {certfile, ClientCertFile}, {keyfile, ClientKeyFile}, {ssl_imp, new}]}, - {client_verification_opts_digital_signature_only, [{cacertfile, ClientCaCertFile}, + {client_verification_opts_digital_signature_only, [{cacertfile, ServerCaCertFile}, {certfile, ClientCertFileDigitalSignatureOnly}, {keyfile, ClientKeyFile}, {ssl_imp, new}]}, @@ -426,7 +426,7 @@ cert_options(Config) -> {user_lookup_fun, {fun user_lookup/3, undefined}}, {ciphers, srp_anon_suites()}]}, {server_verification_opts, [{ssl_imp, new},{reuseaddr, true}, - {cacertfile, ServerCaCertFile}, + {cacertfile, ClientCaCertFile}, {certfile, ServerCertFile}, {keyfile, ServerKeyFile}]}, {client_kc_opts, [{certfile, ClientKeyCertFile}, {ssl_imp, new}]}, {server_kc_opts, [{ssl_imp, new},{reuseaddr, true}, diff --git a/lib/stdlib/doc/src/epp.xml b/lib/stdlib/doc/src/epp.xml index 0802f0e47c..ac87f9c2b6 100644 --- a/lib/stdlib/doc/src/epp.xml +++ b/lib/stdlib/doc/src/epp.xml @@ -102,6 +102,7 @@ <func> <name name="parse_erl_form" arity="1"/> <fsummary>Return the next Erlang form from the opened Erlang source file</fsummary> + <type name="warning_info"/> <desc> <p>Returns the next Erlang form from the opened Erlang source file. The tuple <c>{eof, <anno>Line</anno>}</c> is returned at end-of-file. The first diff --git a/lib/stdlib/doc/src/ets.xml b/lib/stdlib/doc/src/ets.xml index 2d69c201bc..b32638f04d 100644 --- a/lib/stdlib/doc/src/ets.xml +++ b/lib/stdlib/doc/src/ets.xml @@ -123,8 +123,9 @@ <p>Some of the functions uses a <em>match specification</em>, match_spec. A brief explanation is given in <seealso marker="#select/2">select/2</seealso>. For a detailed - description, see the chapter "Match specifications in Erlang" in - <em>ERTS User's Guide</em>.</p> + description, see chapter + <seealso marker="erts:match_spec">Match Specifications in Erlang</seealso> + in <em>ERTS User's Guide</em>.</p> </section> <datatypes> diff --git a/lib/stdlib/doc/src/gen_statem.xml b/lib/stdlib/doc/src/gen_statem.xml index b1d9799917..0e7d6e53e9 100644 --- a/lib/stdlib/doc/src/gen_statem.xml +++ b/lib/stdlib/doc/src/gen_statem.xml @@ -136,9 +136,9 @@ erlang:'!' -----> Module:StateName/3 is used as the state function name; see <seealso marker="#Module:StateName/3"><c>Module:StateName/3</c></seealso>. This gathers all code for a specific state - in one function and hence dispatches on state first. - Notice that in this mode - the mandatory callback function + in one function as the <c>gen_statem</c> engine + branches depending on state name. + Notice that in this mode the mandatory callback function <seealso marker="#Module:terminate/3"><c>Module:terminate/3</c></seealso> makes the state name <c>terminate</c> unusable. </p> @@ -148,7 +148,7 @@ erlang:'!' -----> Module:StateName/3 is <c>handle_event_function</c>, the state can be any term and the state function name is <seealso marker="#Module:handle_event/4"><c>Module:handle_event/4</c></seealso>. - This makes it easy to dispatch on state or on event as you desire. + This makes it easy to branch depending on state or event as you desire. Be careful about which events you handle in which states so that you do not accidentally postpone an event forever creating an infinite busy loop. diff --git a/lib/stdlib/doc/src/proc_lib.xml b/lib/stdlib/doc/src/proc_lib.xml index 245580b1ba..f02b1f0651 100644 --- a/lib/stdlib/doc/src/proc_lib.xml +++ b/lib/stdlib/doc/src/proc_lib.xml @@ -73,6 +73,13 @@ <name name="priority_level"/> </datatype> <datatype> + <name name="max_heap_size"/> + <desc> + <p>See <seealso marker="erts:erlang#process_flag_max_heap_size"> + erlang:process_flag(max_heap_size, MaxHeapSize)</seealso>.</p> + </desc> + </datatype> + <datatype> <name name="dict_or_pid"/> </datatype> </datatypes> diff --git a/lib/stdlib/src/otp_internal.erl b/lib/stdlib/src/otp_internal.erl index 7a59523f06..c3ad261daa 100644 --- a/lib/stdlib/src/otp_internal.erl +++ b/lib/stdlib/src/otp_internal.erl @@ -35,7 +35,7 @@ obsolete(Module, Name, Arity) -> case obsolete_1(Module, Name, Arity) of {deprecated=Tag,{_,_,_}=Replacement} -> - {Tag,Replacement,"in a future release"}; + {Tag,Replacement,"a future release"}; {_,String}=Ret when is_list(String) -> Ret; {_,_,_}=Ret -> diff --git a/lib/stdlib/src/proc_lib.erl b/lib/stdlib/src/proc_lib.erl index fdc2ae4070..4a19603ec2 100644 --- a/lib/stdlib/src/proc_lib.erl +++ b/lib/stdlib/src/proc_lib.erl @@ -43,9 +43,14 @@ %%----------------------------------------------------------------------------- -type priority_level() :: 'high' | 'low' | 'max' | 'normal'. +-type max_heap_size() :: non_neg_integer() | + #{ size => non_neg_integer(), + kill => true, + error_logger => true}. -type spawn_option() :: 'link' | 'monitor' | {'priority', priority_level()} + | {'max_heap_size', max_heap_size()} | {'min_heap_size', non_neg_integer()} | {'min_bin_vheap_size', non_neg_integer()} | {'fullsweep_after', non_neg_integer()} diff --git a/lib/stdlib/test/erl_lint_SUITE.erl b/lib/stdlib/test/erl_lint_SUITE.erl index b0214e5238..d916eb3eef 100644 --- a/lib/stdlib/test/erl_lint_SUITE.erl +++ b/lib/stdlib/test/erl_lint_SUITE.erl @@ -2003,7 +2003,7 @@ otp_5362(Config) when is_list(Config) -> {error, [{5,erl_lint,{call_to_redefined_old_bif,{spawn,1}}}], [{4,erl_lint,{deprecated,{erlang,hash,2},{erlang,phash2,2}, - "in a future release"}}]}}, + "a future release"}}]}}, {otp_5362_5, <<"-compile(nowarn_deprecated_function). @@ -2063,7 +2063,7 @@ otp_5362(Config) when is_list(Config) -> {nowarn_bif_clash,{spawn,1}}]}, % has no effect {warnings, [{5,erl_lint,{deprecated,{erlang,hash,2},{erlang,phash2,2}, - "in a future release"}}]}}, + "a future release"}}]}}, {otp_5362_9, <<"-include_lib(\"stdlib/include/qlc.hrl\"). @@ -2093,7 +2093,7 @@ otp_5362(Config) when is_list(Config) -> [], {warnings, [{1,erl_lint,{deprecated,{erlang,hash,2}, - {erlang,phash2,2},"in a future release"}}]}}, + {erlang,phash2,2},"a future release"}}]}}, {call_removed_function, <<"t(X) -> regexp:match(X).">>, diff --git a/lib/syntax_tools/doc/src/notes.xml b/lib/syntax_tools/doc/src/notes.xml index 406a6c071b..78b2c7c7a4 100644 --- a/lib/syntax_tools/doc/src/notes.xml +++ b/lib/syntax_tools/doc/src/notes.xml @@ -76,11 +76,11 @@ <p> Teach Maps to erl_syntax</p> <p> - Affected functions: <list> + Affected functions:</p> <list> <item>erl_syntax:abstract/1</item> <item>erl_syntax:concrete/1</item> <item>erl_syntax:is_leaf/1</item> - <item>erl_syntax:is_literal/1</item> </list></p> + <item>erl_syntax:is_literal/1</item> </list> <p> Own Id: OTP-12265</p> </item> diff --git a/lib/syntax_tools/vsn.mk b/lib/syntax_tools/vsn.mk index 403e90196e..f09c2a01d0 100644 --- a/lib/syntax_tools/vsn.mk +++ b/lib/syntax_tools/vsn.mk @@ -1 +1 @@ -SYNTAX_TOOLS_VSN = 1.7 +SYNTAX_TOOLS_VSN = 2.0 diff --git a/lib/tools/src/cover.erl b/lib/tools/src/cover.erl index 69331732bf..92c10cc306 100644 --- a/lib/tools/src/cover.erl +++ b/lib/tools/src/cover.erl @@ -573,7 +573,7 @@ call(Request) -> Ref = erlang:monitor(process,?SERVER), receive {'DOWN', Ref, _Type, _Object, noproc} -> erlang:demonitor(Ref), - start(), + {ok,_} = start(), call(Request) after 0 -> ?SERVER ! {self(),Request}, @@ -589,7 +589,9 @@ call(Request) -> end. reply(From, Reply) -> - From ! {?SERVER,Reply}. + From ! {?SERVER,Reply}, + ok. + is_from(From) -> is_pid(From). @@ -615,9 +617,11 @@ remote_call(Node,Request) -> end. remote_reply(Proc,Reply) when is_pid(Proc) -> - Proc ! {?SERVER,Reply}; + Proc ! {?SERVER,Reply}, + ok; remote_reply(MainNode,Reply) -> - {?SERVER,MainNode} ! {?SERVER,Reply}. + {?SERVER,MainNode} ! {?SERVER,Reply}, + ok. %%%---------------------------------------------------------------------- %%% cover_server on main node @@ -627,14 +631,16 @@ init_main(Starter) -> register(?SERVER,self()), %% Having write concurrancy here gives a 40% performance boost %% when collect/1 is called. - ets:new(?COVER_TABLE, [set, public, named_table - ,{write_concurrency, true} - ]), - ets:new(?COVER_CLAUSE_TABLE, [set, public, named_table]), - ets:new(?BINARY_TABLE, [set, public, named_table]), - ets:new(?COLLECTION_TABLE, [set, public, named_table]), - ets:new(?COLLECTION_CLAUSE_TABLE, [set, public, named_table]), - net_kernel:monitor_nodes(true), + ?COVER_TABLE = ets:new(?COVER_TABLE, [set, public, named_table, + {write_concurrency, true}]), + ?COVER_CLAUSE_TABLE = ets:new(?COVER_CLAUSE_TABLE, [set, public, + named_table]), + ?BINARY_TABLE = ets:new(?BINARY_TABLE, [set, public, named_table]), + ?COLLECTION_TABLE = ets:new(?COLLECTION_TABLE, [set, public, + named_table]), + ?COLLECTION_CLAUSE_TABLE = ets:new(?COLLECTION_CLAUSE_TABLE, [set, public, + named_table]), + ok = net_kernel:monitor_nodes(true), Starter ! {?SERVER,started}, main_process_loop(#main_state{}). @@ -672,7 +678,7 @@ main_process_loop(State) -> Imported = do_import_to_table(Fd,File, State#main_state.imported), reply(From, ok), - file:close(Fd), + ok = file:close(Fd), main_process_loop(State#main_state{imported=Imported}); {error,Reason} -> reply(From, {error, {cant_open_file,File,Reason}}), @@ -870,11 +876,12 @@ main_process_loop(State) -> init_remote(Starter,MainNode) -> register(?SERVER,self()), - ets:new(?COVER_TABLE, [set, public, named_table - %% write_concurrency here makes otp_8270 break :( - %,{write_concurrency, true} - ]), - ets:new(?COVER_CLAUSE_TABLE, [set, public, named_table]), + %% write_concurrency here makes otp_8270 break :( + ?COVER_TABLE = ets:new(?COVER_TABLE, [set, public, named_table + %,{write_concurrency, true} + ]), + ?COVER_CLAUSE_TABLE = ets:new(?COVER_CLAUSE_TABLE, [set, public, + named_table]), Starter ! {self(),started}, remote_process_loop(#remote_state{main_node=MainNode}). @@ -907,11 +914,11 @@ remote_process_loop(State) -> '_' -> [M || {M,_} <- State#remote_state.compiled]; _ -> Modules0 end, - spawn(fun() -> - ?SPAWN_DBG(remote_collect, - {Modules, CollectorPid, From}), - do_collect(Modules, CollectorPid, From) - end), + spawn(fun() -> + ?SPAWN_DBG(remote_collect, + {Modules, CollectorPid, From}), + do_collect(Modules, CollectorPid, From) + end), remote_process_loop(State); {remote,stop} -> @@ -952,13 +959,13 @@ remote_process_loop(State) -> end. do_collect(Modules, CollectorPid, From) -> - pmap( - fun(Module) -> - Pattern = {#bump{module=Module, _='_'}, '$1'}, - MatchSpec = [{Pattern,[{'=/=','$1',0}],['$_']}], - Match = ets:select(?COVER_TABLE,MatchSpec,?CHUNK_SIZE), - send_chunks(Match, CollectorPid, []) - end,Modules), + _ = pmap( + fun(Module) -> + Pattern = {#bump{module=Module, _='_'}, '$1'}, + MatchSpec = [{Pattern,[{'=/=','$1',0}],['$_']}], + Match = ets:select(?COVER_TABLE,MatchSpec,?CHUNK_SIZE), + send_chunks(Match, CollectorPid, []) + end,Modules), CollectorPid ! done, remote_reply(From, ok). @@ -994,20 +1001,20 @@ get_downs(Mons) -> end. reload_originals(Compiled) -> - Modules = [M || {M,_} <- Compiled], - pmap(fun do_reload_original/1, Modules). + _ = pmap(fun do_reload_original/1, [M || {M,_} <- Compiled]), + ok. do_reload_original(Module) -> case code:which(Module) of ?TAG -> - code:purge(Module), % remove code marked as 'old' - code:delete(Module), % mark cover compiled code as 'old' + _ = code:purge(Module), % remove code marked as 'old' + _ = code:delete(Module), % mark cover compiled code as 'old' %% Note: original beam code must be loaded before the cover %% compiled code is purged, in order to for references to %% 'fun M:F/A' and %% 'fun F/A' funs to be correct (they %% refer to (M:)F/A in the *latest* version of the module) - code:load_file(Module), % load original code - code:purge(Module); % remove cover compiled code + _ = code:load_file(Module), % load original code + _ = code:purge(Module); % remove cover compiled code _ -> ignore end. @@ -1219,12 +1226,13 @@ remote_reset(Module,Nodes) -> %% Collect data from remote nodes - used for analyse or stop(Node) remote_collect(Modules,Nodes,Stop) -> - pmap(fun(Node) -> - ?SPAWN_DBG(remote_collect, - {Modules, Nodes, Stop}), - do_collection(Node, Modules, Stop) - end, - Nodes). + _ = pmap( + fun(Node) -> + ?SPAWN_DBG(remote_collect, + {Modules, Nodes, Stop}), + do_collection(Node, Modules, Stop) + end, Nodes), + ok. do_collection(Node, Module, Stop) -> CollectorPid = spawn(fun collector_proc/0), @@ -1260,8 +1268,8 @@ insert_in_collection_table([]) -> insert_in_collection_table(Key,Val) -> case ets:member(?COLLECTION_TABLE,Key) of true -> - ets:update_counter(?COLLECTION_TABLE, - Key,Val); + _ = ets:update_counter(?COLLECTION_TABLE, Key,Val), + ok; false -> %% Make sure that there are no race conditions from ets:member case ets:insert_new(?COLLECTION_TABLE,{Key,Val}) of @@ -2421,7 +2429,7 @@ do_analyse_to_file1(Module, OutFile, ErlFile, HTML) -> "<body style='background-color: white;" " color: black'>\n" "<pre>\n"], - file:write(OutFd,Header); + ok = file:write(OutFd,Header); true -> ok end, @@ -2435,7 +2443,7 @@ do_analyse_to_file1(Module, OutFile, ErlFile, HTML) -> string:right(integer_to_list(H), 2, $0), string:right(integer_to_list(Mi), 2, $0), string:right(integer_to_list(S), 2, $0)]), - file:write(OutFd, + ok = file:write(OutFd, ["File generated from ",ErlFile," by COVER ", Timestamp,"\n\n" "**************************************" @@ -2447,14 +2455,13 @@ do_analyse_to_file1(Module, OutFile, ErlFile, HTML) -> CovLines = lists:keysort(1,ets:select(?COLLECTION_TABLE, MS)), print_lines(Module, CovLines, InFd, OutFd, 1, HTML), - if - HTML -> - file:write(OutFd, "</pre>\n</body>\n</html>\n"); + if HTML -> + ok = file:write(OutFd, "</pre>\n</body>\n</html>\n"); true -> ok end, - file:close(OutFd), - file:close(InFd), + ok = file:close(OutFd), + ok = file:close(InFd), {ok, OutFile}; @@ -2472,34 +2479,33 @@ print_lines(Module, CovLines, InFd, OutFd, L, HTML) -> eof -> ignore; {ok,"%"++_=Line} -> %Comment line - not executed. - file:write(OutFd, [tab(),escape_lt_and_gt(Line, HTML)]), + ok = file:write(OutFd, [tab(),escape_lt_and_gt(Line, HTML)]), print_lines(Module, CovLines, InFd, OutFd, L+1, HTML); {ok,RawLine} -> Line = escape_lt_and_gt(RawLine,HTML), case CovLines of [{L,N}|CovLines1] -> %% N = lists:foldl(fun([Ni], Nacc) -> Nacc+Ni end, 0, Ns), - if - N=:=0, HTML=:=true -> - LineNoNL = Line -- "\n", - Str = " 0", - %%Str = string:right("0", 6, 32), - RedLine = ["<font color=red>",Str,fill1(), - LineNoNL,"</font>\n"], - file:write(OutFd, RedLine); - N<1000000 -> - Str = string:right(integer_to_list(N), 6, 32), - file:write(OutFd, [Str,fill1(),Line]); - N<10000000 -> - Str = integer_to_list(N), - file:write(OutFd, [Str,fill2(),Line]); - true -> - Str = integer_to_list(N), - file:write(OutFd, [Str,fill3(),Line]) - end, + if N=:=0, HTML=:=true -> + LineNoNL = Line -- "\n", + Str = " 0", + %%Str = string:right("0", 6, 32), + RedLine = ["<font color=red>",Str,fill1(), + LineNoNL,"</font>\n"], + ok = file:write(OutFd, RedLine); + N < 1000000 -> + Str = string:right(integer_to_list(N), 6, 32), + ok = file:write(OutFd, [Str,fill1(),Line]); + N < 10000000 -> + Str = integer_to_list(N), + ok = file:write(OutFd, [Str,fill2(),Line]); + true -> + Str = integer_to_list(N), + ok = file:write(OutFd, [Str,fill3(),Line]) + end, print_lines(Module, CovLines1, InFd, OutFd, L+1, HTML); _ -> - file:write(OutFd, [tab(),Line]), + ok = file:write(OutFd, [tab(),Line]), print_lines(Module, CovLines, InFd, OutFd, L+1, HTML) end end. @@ -2539,7 +2545,7 @@ do_export(Module, OutFile, From, State) -> {error,{not_cover_compiled,Module}} end end, - file:close(Fd), + ok = file:close(Fd), reply(From, Reply); {error,Reason} -> reply(From, {error, {cant_open_file,OutFile,Reason}}) @@ -2581,10 +2587,9 @@ write(Element,Fd) -> case byte_size(Bin) of Size when Size > 255 -> SizeBin = term_to_binary({'$size',Size}), - file:write(Fd, - <<(byte_size(SizeBin)):8,SizeBin/binary,Bin/binary>>); + ok = file:write(Fd, <<(byte_size(SizeBin)):8,SizeBin/binary,Bin/binary>>); Size -> - file:write(Fd,<<Size:8,Bin/binary>>) + ok = file:write(Fd,<<Size:8,Bin/binary>>) end, ok. diff --git a/lib/tools/src/eprof.erl b/lib/tools/src/eprof.erl index 07e995993b..3ae899a078 100644 --- a/lib/tools/src/eprof.erl +++ b/lib/tools/src/eprof.erl @@ -74,7 +74,6 @@ start() -> gen_server:start({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:call(?MODULE, stop, infinity). - analyze() -> analyze(procs). @@ -112,7 +111,7 @@ profile(Rootset, M, F, A, Pattern) when is_list(Rootset), is_atom(M), is_atom(F) %% Returns when M:F/A has terminated profile(Rootset, M, F, A, Pattern, Options) -> - start(), + ok = start_internal(), gen_server:call(?MODULE, {profile_start, Rootset, Pattern, {M,F,A}, Options}, infinity). dump() -> @@ -127,7 +126,7 @@ start_profiling(Rootset) -> start_profiling(Rootset, Pattern) -> start_profiling(Rootset, Pattern, ?default_options). start_profiling(Rootset, Pattern, Options) -> - start(), + ok = start_internal(), gen_server:call(?MODULE, {profile_start, Rootset, Pattern, undefined, Options}, infinity). stop_profiling() -> @@ -251,9 +250,9 @@ handle_call({logfile, File}, _From, #state{ fd = OldFd } = S) -> {ok, Fd} -> case OldFd of undefined -> ok; - OldFd -> file:close(OldFd) + OldFd -> ok = file:close(OldFd) end, - {reply, ok, S#state{ fd = Fd}}; + {reply, ok, S#state{fd = Fd}}; Error -> {reply, Error, S} end; @@ -521,3 +520,10 @@ format(Fd, Format, Strings) -> divide(_,0) -> 0.0; divide(T,N) -> T/N. + +start_internal() -> + case start() of + {ok, _} -> ok; + {error, {already_started,_}} -> ok; + Error -> Error + end. diff --git a/lib/tools/src/fprof.erl b/lib/tools/src/fprof.erl index b21eedc625..8db23dd151 100644 --- a/lib/tools/src/fprof.erl +++ b/lib/tools/src/fprof.erl @@ -1003,7 +1003,7 @@ handle_req(#analyse{dest = Dest, already_open -> ok; ok -> - file:close(DestPid) + ok = file:close(DestPid) end, State end; @@ -1364,7 +1364,7 @@ tracer_loop(Parent, Handler, State) -> Trace when element(1, Trace) =:= trace_ts -> tracer_loop(Parent, Handler, Handler(Trace, State)); {'EXIT', Parent, Reason} -> - handler(end_of_trace, State), + _ = handler(end_of_trace, State), exit(Reason); _ -> tracer_loop(Parent, Handler, State) @@ -1450,12 +1450,10 @@ end_of_trace(Table, TS) -> Procs = get(), put(table, Table), ?dbg(2, "get() -> ~p~n", [Procs]), - lists:map( - fun ({Pid, _}) when is_pid(Pid) -> - trace_exit(Table, Pid, TS) - end, - Procs), - erase(), + _ = lists:map(fun ({Pid, _}) when is_pid(Pid) -> + trace_exit(Table, Pid, TS) + end, Procs), + _ = erase(), ok. @@ -2047,7 +2045,7 @@ trace_exit(Table, Pid, TS) -> [] -> ok; [_ | _] = Stack -> - trace_return_to_int(Table, Pid, undefined, TS, Stack), + _ = trace_return_to_int(Table, Pid, undefined, TS, Stack), ok end, ok. @@ -2173,7 +2171,7 @@ trace_clock(_Table, _Pid, _T, [[{suspend, _}], [{suspend, _}] | _]=_Stack, _Clock) -> ?dbg(9, "trace_clock(Table, ~w, ~w, ~w, ~w)~n", [_Pid, _T, _Stack, _Clock]), - void; + ok; trace_clock(Table, Pid, T, [[{garbage_collect, TS0}], [{suspend, _}]], Clock) -> trace_clock_1(Table, Pid, T, TS0, undefined, garbage_collect, Clock); @@ -2188,7 +2186,7 @@ trace_clock(Table, Pid, T, [[{Func0, TS0}], [{Func1, _} | _] | _], Clock) -> trace_clock(Table, Pid, T, [[{Func0, TS0}]], Clock) -> trace_clock_1(Table, Pid, T, TS0, undefined, Func0, Clock); trace_clock(_, _, _, [], _) -> - void. + ok. trace_clock_1(Table, Pid, _, _, Caller, suspend, #clocks.own) -> clock_add(Table, {Pid, Caller, suspend}, #clocks.own, 0); @@ -2202,7 +2200,7 @@ trace_clock_1(Table, Pid, T, TS, Caller, Func, Clock) -> clock_add(Table, Id, Clock, T) -> ?dbg(1, "clock_add(Table, ~w, ~w, ~w)~n", [Id, Clock, T]), - try ets:update_counter(Table, Id, {Clock, T}) + try ets:update_counter(Table, Id, {Clock, T}), ok catch error:badarg -> ets:insert(Table, #clocks{id = Id}), @@ -2211,7 +2209,7 @@ clock_add(Table, Id, Clock, T) -> true -> ?dbg(0, "Negative counter value ~p ~p ~p ~p~n", [X, Id, Clock, T]) end, - X + ok end. clocks_add(Table, #clocks{id = Id} = Clocks) -> diff --git a/lib/tools/src/lcnt.erl b/lib/tools/src/lcnt.erl index 78407a651a..23d66b084e 100644 --- a/lib/tools/src/lcnt.erl +++ b/lib/tools/src/lcnt.erl @@ -23,67 +23,57 @@ -author("Björn-Egil Dahlberg"). %% gen_server callbacks --export([ - init/1, - handle_call/3, - handle_cast/2, - handle_info/2, - terminate/2, - code_change/3 - ]). +-export([init/1, + handle_call/3, + handle_cast/2, + handle_info/2, + terminate/2, + code_change/3]). %% start/stop --export([ - start/0, - stop/0 - ]). +-export([start/0, + stop/0]). %% erts_debug:lock_counters api --export([ - rt_collect/0, - rt_collect/1, - rt_clear/0, - rt_clear/1, - rt_opt/1, - rt_opt/2 - ]). +-export([rt_collect/0, + rt_collect/1, + rt_clear/0, + rt_clear/1, + rt_opt/1, + rt_opt/2]). %% gen_server call api --export([ - raw/0, - collect/0, - collect/1, - clear/0, - clear/1, - conflicts/0, - conflicts/1, - locations/0, - locations/1, - inspect/1, - inspect/2, - histogram/1, - histogram/2, - information/0, - swap_pid_keys/0, - % set options - set/1, - set/2, - - load/1, - save/1 - ]). +-export([raw/0, + collect/0, + collect/1, + clear/0, + clear/1, + conflicts/0, + conflicts/1, + locations/0, + locations/1, + inspect/1, + inspect/2, + histogram/1, + histogram/2, + information/0, + swap_pid_keys/0, + % set options + set/1, + set/2, + + load/1, + save/1]). %% convenience --export([ - apply/3, - apply/2, - apply/1, - all_conflicts/0, - all_conflicts/1, - pid/2, pid/3, - port/1, port/2 - ]). +-export([apply/3, + apply/2, + apply/1, + all_conflicts/0, + all_conflicts/1, + pid/2, pid/3, + port/1, port/2]). -define(version, "1.0"). @@ -135,6 +125,13 @@ start() -> gen_server:start({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:call(?MODULE, stop, infinity). init([]) -> {ok, #state{ locks = [], duration = 0 } }. +start_internal() -> + case start() of + {ok,_} -> ok; + {error, {already_started,_}} -> ok; + Error -> Error + end. + %% -------------------------------------------------------------------- %% %% %% API erts_debug:lock_counters @@ -184,7 +181,7 @@ raw() -> call(raw). set(Option, Value) -> call({set, Option, Value}). set({Option, Value}) -> call({set, Option, Value}). save(Filename) -> call({save, Filename}). -load(Filename) -> start(), call({load, Filename}). +load(Filename) -> ok = start_internal(), call({load, Filename}). call(Msg) -> gen_server:call(?MODULE, Msg, infinity). @@ -195,7 +192,7 @@ call(Msg) -> gen_server:call(?MODULE, Msg, infinity). %% -------------------------------------------------------------------- %% apply(M,F,As) when is_atom(M), is_atom(F), is_list(As) -> - lcnt:start(), + ok = start_internal(), Opt = lcnt:rt_opt({copy_save, true}), lcnt:clear(), Res = erlang:apply(M,F,As), @@ -207,7 +204,7 @@ apply(Fun) when is_function(Fun) -> lcnt:apply(Fun, []). apply(Fun, As) when is_function(Fun) -> - lcnt:start(), + ok = start_internal(), Opt = lcnt:rt_opt({copy_save, true}), lcnt:clear(), Res = erlang:apply(Fun, As), diff --git a/lib/tools/src/tags.erl b/lib/tools/src/tags.erl index 2bc1865503..b833d96c19 100644 --- a/lib/tools/src/tags.erl +++ b/lib/tools/src/tags.erl @@ -101,7 +101,7 @@ files(Files, Options) -> case open_out(Options) of {ok, Os} -> files_loop(Files, Os), - close_out(Os), + ok = close_out(Os), ok; _ -> error @@ -169,7 +169,7 @@ filename(Name, Os) -> case file:open(Name, [read]) of {ok, Desc} -> Acc = module(Desc, [], [], {1, 0}), - file:close(Desc), + ok = file:close(Desc), genout(Os, Name, Acc), ok; _ -> diff --git a/lib/tools/src/xref_base.erl b/lib/tools/src/xref_base.erl index 70eb107781..4322943c59 100644 --- a/lib/tools/src/xref_base.erl +++ b/lib/tools/src/xref_base.erl @@ -696,7 +696,7 @@ do_add_module({Dir, Basename}, AppName, Builtins, Verbose, Warnings, State) -> File = filename:join(Dir, Basename), {ok, M, Bad, NewState} = do_add_module1(Dir, File, AppName, Builtins, Verbose, Warnings, State), - filter(fun({Tag,B}) -> warnings(Warnings, Tag, [[File,B]]) end, Bad), + _ = filter(fun({Tag,B}) -> warnings(Warnings, Tag, [[File,B]]) end, Bad), {ok, M, NewState}. do_add_module1(Dir, File, AppName, Builtins, Verbose, Warnings, State) -> @@ -1727,7 +1727,7 @@ pack(T) -> NT = pack1(T), %% true = T =:= NT, %% io:format("erasing ~p elements...~n", [length(erase())]), - erase(), % wasting heap (and time)... + _ = erase(), % wasting heap (and time)... foreach(fun({K,V}) -> put(K, V) end, PD), NT. diff --git a/lib/wx/api_gen/gl_gen.erl b/lib/wx/api_gen/gl_gen.erl index 0473b7d771..54635bdd2e 100644 --- a/lib/wx/api_gen/gl_gen.erl +++ b/lib/wx/api_gen/gl_gen.erl @@ -191,8 +191,9 @@ parse_define([#xmlElement{name=initializer,content=Contents}|_R],Def,_Os) -> try case Val0 of "0x" ++ Val1 -> - _ = list_to_integer(Val1, 16), - Def#def{val=Val1, type=hex}; + Val2 = strip_type_cast(Val1), + _ = list_to_integer(Val2, 16), + Def#def{val=Val2, type=hex}; _ -> Val = list_to_integer(Val0), Def#def{val=Val, type=int} @@ -214,6 +215,15 @@ extract_def2([#xmlElement{content=Cs}|R]) -> extract_def2(Cs) ++ extract_def2(R); extract_def2([]) -> []. +strip_type_cast(Int) -> + lists:reverse(strip_type_cast2(lists:reverse(Int))). + +strip_type_cast2("u"++Rest) -> Rest; %% unsigned +strip_type_cast2("lu"++Rest) -> Rest; %% unsigned long +strip_type_cast2("llu"++Rest) -> Rest; %% unsigned long long +strip_type_cast2(Rest) -> Rest. + + strip_comment("/*" ++ Rest) -> strip_comment_until_end(Rest); strip_comment("//" ++ _) -> []; diff --git a/lib/wx/c_src/wxe_helpers.cpp b/lib/wx/c_src/wxe_helpers.cpp index 4798e605e8..d1f607d2af 100644 --- a/lib/wx/c_src/wxe_helpers.cpp +++ b/lib/wx/c_src/wxe_helpers.cpp @@ -48,7 +48,7 @@ void wxeCommand::Delete() driver_free(buffer); buffer = NULL; } - op = -1; + op = -2; } /* **************************************************************************** @@ -84,7 +84,7 @@ wxeCommand * wxeFifo::Get() pos = m_first++; m_n--; m_first %= m_max; - } while(m_q[pos].op == -1); + } while(m_q[pos].op < 0); return &m_q[pos]; } @@ -96,7 +96,7 @@ wxeCommand * wxeFifo::Peek(unsigned int *i) return NULL; pos = (m_first+*i) % m_max; (*i)++; - } while(m_q[pos].op == -1); + } while(m_q[pos].op < 0); return &m_q[pos]; } @@ -213,7 +213,7 @@ void wxeFifo::Realloc() // Strip end of queue if ops are already taken care of, avoids reallocs void wxeFifo::Strip() { - while((m_n > 0) && (m_q[(m_first + m_n - 1)%m_max].op == -1)) { + while((m_n > 0) && (m_q[(m_first + m_n - 1)%m_max].op < -1)) { m_n--; } } diff --git a/lib/wx/include/gl.hrl b/lib/wx/include/gl.hrl index 39cd474fcb..d708515e1b 100644 --- a/lib/wx/include/gl.hrl +++ b/lib/wx/include/gl.hrl @@ -1723,7 +1723,7 @@ -define(GL_UNIFORM_BLOCK_REFERENCED_BY_VERTEX_SHADER, 16#8A44). -define(GL_UNIFORM_BLOCK_REFERENCED_BY_GEOMETRY_SHADER, 16#8A45). -define(GL_UNIFORM_BLOCK_REFERENCED_BY_FRAGMENT_SHADER, 16#8A46). --define(GL_INVALID_INDEX, 16#FFFFFFFFu). +-define(GL_INVALID_INDEX, 16#FFFFFFFF). -define(GL_COPY_READ_BUFFER, 16#8F36). -define(GL_COPY_WRITE_BUFFER, 16#8F37). -define(GL_DEPTH_CLAMP, 16#864F). @@ -1746,7 +1746,7 @@ -define(GL_CONDITION_SATISFIED, 16#911C). -define(GL_WAIT_FAILED, 16#911D). -define(GL_SYNC_FLUSH_COMMANDS_BIT, 16#00000001). --define(GL_TIMEOUT_IGNORED, 16#FFFFFFFFFFFFFFFFull). +-define(GL_TIMEOUT_IGNORED, 16#FFFFFFFFFFFFFFFF). -define(GL_SAMPLE_POSITION, 16#8E50). -define(GL_SAMPLE_MASK, 16#8E51). -define(GL_SAMPLE_MASK_VALUE, 16#8E52). diff --git a/lib/wx/test/wx_class_SUITE.erl b/lib/wx/test/wx_class_SUITE.erl index 6944a78360..0a3c4659bf 100644 --- a/lib/wx/test/wx_class_SUITE.erl +++ b/lib/wx/test/wx_class_SUITE.erl @@ -646,8 +646,8 @@ modal(Config) -> %% need to sleep so we know that the window is stuck in %% the ShowModal event loop and not in an earlier event loop %% wx2.8 invokes the event loop from more calls than wx-3 - receive {dialog, M1, 1} -> timer:sleep(1200), ets:insert(test_state, {M1, ready}) end, - receive {dialog, M2, 2} -> timer:sleep(1200), ets:insert(test_state, {M2, ready}) end, + M1 = receive {dialog, W1, 1} -> timer:sleep(1200), ets:insert(test_state, {W1, ready}), W1 end, + M2 = receive {dialog, W2, 2} -> timer:sleep(1200), ets:insert(test_state, {W2, ready}), W2 end, receive done -> ok end, receive {dialog_done, M2, 2} -> M2 end, @@ -139,7 +139,7 @@ check_erltop () if [ "X$ERL_TOP" = "X" ]; then if [ -f ./otp_build -a -f ./erts/autoconf/config.guess ]; then ERLTOP_FORCED=true - ERL_TOP=`/bin/pwd` + ERL_TOP=`pwd` export ERL_TOP else echo "The environment variable ERL_TOP must be set." >&2 @@ -679,7 +679,7 @@ echo_env_erltop () if [ X"$ERL_TOP" = X"" -o "$ERLTOP_FORCED" = "true" ]; then if [ -f ./otp_build ]; then # Seems to be current directory... - echo_setenv ERL_TOP `/bin/pwd` ';' + echo_setenv ERL_TOP `pwd` ';' else echo "You need to either set ERL_TOP first or stand in the same" \ "directory as this script resides in." >&2 diff --git a/system/doc/design_principles/statem.xml b/system/doc/design_principles/statem.xml index 585b1a35f5..b63327291d 100644 --- a/system/doc/design_principles/statem.xml +++ b/system/doc/design_principles/statem.xml @@ -152,9 +152,10 @@ handle_event(EventType, EventContent, State, Data) -> </p> <p> With <c>state_functions</c>, you are restricted to use - atom-only states, and the <c>gen_statem</c> engine dispatches - on state name for you. This encourages the callback module - to gather the implementation of all event actions particular + atom-only states, and the <c>gen_statem</c> engine + branches depending on state name for you. + This encourages the callback module to gather + the implementation of all event actions particular to one state in the same place in the code, hence to focus on one state at the time. </p> @@ -173,7 +174,8 @@ handle_event(EventType, EventContent, State, Data) -> one event at the time or on one state at the time, but function <seealso marker="stdlib:gen_statem#Module:handle_event/4"><c>Module:handle_event/4</c></seealso> - quickly grows too large to handle without introducing dispatching. + quickly grows too large to handle without branching to + helper functions. </p> <p> The mode enables the use of non-atom states, for example, @@ -181,8 +183,8 @@ handle_event(EventType, EventContent, State, Data) -> If, for example, a state diagram is largely alike for the client side and the server side of a protocol, you can have a state <c>{StateName,server}</c> or - <c>{StateName,client}</c>. Also, as you do the dispatching - yourself, you make <c>StateName</c> decide where in the code + <c>{StateName,client}</c>, + and make <c>StateName</c> determine where in the code to handle most events in the state. The second element of the tuple is then used to select whether to handle special client-side or server-side events. @@ -526,7 +528,8 @@ handle_event({call,From}, code_length, #{code := Code} = Data) -> all events are handled in <seealso marker="stdlib:gen_statem#Module:handle_event/4"><c>Module:handle_event/4</c></seealso> and we can (but do not have to) use an event-centered approach - where we dispatch on event first and then state: + where we first branch depending on event + and then depending on state: </p> <code type="erl"><![CDATA[ ... @@ -1097,9 +1100,9 @@ code_change(_Vsn, State, Data, _Extra) -> <p> This section describes what to change in the example to use one <c>handle_event/4</c> function. - The previously used clean first-dispatch-on-event approach - does not work that well here because of the generated - entry actions so this example dispatches on state first: + The previously used approach to first branch depending on event + does not work that well here because of the generated + entry actions, so this example first branches depending on state: </p> <code type="erl"><![CDATA[ ... |