aboutsummaryrefslogtreecommitdiffstats
path: root/erts
diff options
context:
space:
mode:
Diffstat (limited to 'erts')
-rw-r--r--erts/doc/src/driver_entry.xml23
-rw-r--r--erts/doc/src/erl_driver.xml57
-rw-r--r--erts/doc/src/erl_nif.xml97
-rw-r--r--erts/emulator/beam/erl_bits.c18
-rw-r--r--erts/emulator/beam/erl_db.c4
-rw-r--r--erts/emulator/beam/erl_port_task.c19
-rw-r--r--erts/emulator/beam/erl_process.c66
-rw-r--r--erts/emulator/beam/erl_process.h2
-rw-r--r--erts/emulator/hipe/hipe_amd64_bifs.m44
-rw-r--r--erts/emulator/hipe/hipe_bif_list.m42
-rw-r--r--erts/emulator/hipe/hipe_x86_bifs.m410
-rw-r--r--erts/emulator/test/bs_bit_binaries_SUITE.erl43
-rw-r--r--erts/emulator/test/bs_construct_SUITE.erl49
-rw-r--r--erts/emulator/valgrind/suppress.halfword56
-rw-r--r--erts/emulator/valgrind/suppress.patched.3.6.020
-rw-r--r--erts/emulator/valgrind/suppress.standard12
16 files changed, 386 insertions, 96 deletions
diff --git a/erts/doc/src/driver_entry.xml b/erts/doc/src/driver_entry.xml
index a2efdf3ebc..929c485c36 100644
--- a/erts/doc/src/driver_entry.xml
+++ b/erts/doc/src/driver_entry.xml
@@ -34,6 +34,29 @@
<lib>driver_entry</lib>
<libsummary>The driver-entry structure used by erlang drivers.</libsummary>
<description>
+ <marker id="WARNING"/>
+ <warning><p><em>Use this functionality with extreme care!</em></p>
+ <p>A driver callback is executed as a direct extension of the
+ native code of the VM. Execution is not made in a safe environment.
+ The VM can <em>not</em> provide the same services as provided when
+ executing Erlang code, such as preemptive scheduling or memory
+ protection. If the driver callback function doesn't behave well,
+ the whole VM will misbehave.</p>
+ <list>
+ <item><p>A driver callback that crash will crash the whole VM.</p></item>
+ <item><p>An erroneously implemented driver callback might cause
+ a VM internal state inconsistency which may cause a crash of the VM,
+ or miscellaneous misbehaviors of the VM at any point after the call
+ to the driver callback.</p></item>
+ <item><p>A driver callback that do
+ <seealso marker="erl_driver#lengthy_work">lengthy work</seealso>
+ before returning will degrade responsiveness of the VM,
+ and may cause miscellaneous strange behaviors. Such strange behaviors
+ include, but are not limited to, extreme memory usage, and bad load
+ balancing between schedulers. Strange behaviors that might occur due
+ to lengthy work may also vary between OTP releases.</p></item>
+ </list>
+ </warning>
<p>
As of erts version 5.9 (OTP release R15B) the driver interface
has been changed with larger types for the callbacks
diff --git a/erts/doc/src/erl_driver.xml b/erts/doc/src/erl_driver.xml
index 187c263b60..e16fd744c0 100644
--- a/erts/doc/src/erl_driver.xml
+++ b/erts/doc/src/erl_driver.xml
@@ -34,6 +34,32 @@
<lib>erl_driver</lib>
<libsummary>API functions for an Erlang driver</libsummary>
<description>
+ <p>An Erlang driver is a library containing a set of native driver
+ callback functions that the Erlang VM calls when certain
+ events occur. There may be multiple instances of a driver, each
+ instance is associated with an Erlang port.</p>
+ <marker id="WARNING"/>
+ <warning><p><em>Use this functionality with extreme care!</em></p>
+ <p>A driver callback is executed as a direct extension of the
+ native code of the VM. Execution is not made in a safe environment.
+ The VM can <em>not</em> provide the same services as provided when
+ executing Erlang code, such as preemptive scheduling or memory
+ protection. If the driver callback function doesn't behave well,
+ the whole VM will misbehave.</p>
+ <list>
+ <item><p>A driver callback that crash will crash the whole VM.</p></item>
+ <item><p>An erroneously implemented driver callback might cause
+ a VM internal state inconsistency which may cause a crash of the VM,
+ or miscellaneous misbehaviors of the VM at any point after the call
+ to the driver callback.</p></item>
+ <item><p>A driver callback that do <seealso marker="#lengthy_work">lengthy
+ work</seealso> before returning will degrade responsiveness of the VM,
+ and may cause miscellaneous strange behaviors. Such strange behaviors
+ include, but are not limited to, extreme memory usage, and bad load
+ balancing between schedulers. Strange behaviors that might occur due
+ to lengthy work may also vary between OTP releases.</p></item>
+ </list>
+ </warning>
<p>As of erts version 5.5.3 the driver interface has been extended
(see <seealso marker="driver_entry#extended_marker">extended marker</seealso>).
The extended interface introduce
@@ -53,16 +79,12 @@
<p>The driver calls back to the emulator, using the API
functions declared in <c>erl_driver.h</c>. They are used for
outputting data from the driver, using timers, etc.</p>
- <p>A driver is a library with a set of function that the emulator
- calls, in response to Erlang functions and message
- sending. There may be multiple instances of a driver, each
- instance is connected to an Erlang port. Every port has a port
- owner process. Communication with the port is normally done
- through the port owner process.</p>
- <p>Most of the functions take the <c>port</c> handle as an
- argument. This identifies the driver instance. Note that this
- port handle must be stored by the driver, it is not given when
- the driver is called from the emulator (see
+ <p>Each driver instance is associated with a port. Every port
+ has a port owner process. Communication with the port is normally
+ done through the port owner process. Most of the functions take
+ the <c>port</c> handle as an argument. This identifies the driver
+ instance. Note that this port handle must be stored by the driver,
+ it is not given when the driver is called from the emulator (see
<seealso marker="driver_entry#emulator">driver_entry</seealso>).</p>
<p>Some of the functions take a parameter of type
<c>ErlDrvBinary</c>, a driver binary. It should be both
@@ -129,6 +151,21 @@
are <em>only</em> thread safe when used in a runtime
system with SMP support.</p>
</note>
+ <p><marker id="lengthy_work"/>
+ As mentioned in the <seealso marker="#WARNING">warning</seealso> text at
+ the beginning of this document it is of vital importance that a driver callback
+ does return relatively fast. It is hard to give an exact maximum amount
+ of time that a driver callback is allowed to work, but as a rule of thumb
+ a well behaving driver callback should return before a millisecond has
+ passed. This can be achieved using different approaches.
+ If you have full control over the code that are to execute in the driver
+ callback, the best approach is to divide the work into multiple chunks of
+ work and trigger multiple calls to the
+ <seealso marker="driver_entry#timeout">timeout callback</seealso> using
+ zero timeouts. This might, however, not always be possible, e.g. when
+ calling third party libraries. In this case you typically want to dispatch
+ the work to another thread. Information about thread primitives can be
+ found below.</p>
</description>
<section>
diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml
index f484e9eaf7..f00f7b9f46 100644
--- a/erts/doc/src/erl_nif.xml
+++ b/erts/doc/src/erl_nif.xml
@@ -34,30 +34,6 @@
<lib>erl_nif</lib>
<libsummary>API functions for an Erlang NIF library</libsummary>
<description>
- <note><p>The NIF concept is officially supported from R14B. NIF source code
- written for earlier experimental versions might need adaption to run on R14B.</p>
- <p>No incompatible changes between <em>R14B</em> and R14A.</p>
- <p>Incompatible changes between <em>R14A</em> and R13B04:</p>
- <list>
- <item>Environment argument removed for <c>enif_alloc</c>,
- <c>enif_realloc</c>, <c>enif_free</c>, <c>enif_alloc_binary</c>,
- <c>enif_realloc_binary</c>, <c>enif_release_binary</c>,
- <c>enif_alloc_resource</c>, <c>enif_release_resource</c>,
- <c>enif_is_identical</c> and <c>enif_compare</c>.</item>
- <item>Character encoding argument added to <c>enif_get_atom</c>
- and <c>enif_make_existing_atom</c>.</item>
- <item>Module argument added to <c>enif_open_resource_type</c>
- while changing name spaces of resource types from global to module local.</item>
- </list>
- <p>Incompatible changes between <em>R13B04</em> and R13B03:</p>
- <list>
- <item>The function prototypes of the NIFs have changed to expect <c>argc</c> and <c>argv</c>
- arguments. The arity of a NIF is by that no longer limited to 3.</item>
- <item><c>enif_get_data</c> renamed as <c>enif_priv_data</c>.</item>
- <item><c>enif_make_string</c> got a third argument for character encoding.</item>
- </list>
- </note>
-
<p>A NIF library contains native implementation of some functions
of an Erlang module. The native implemented functions (NIFs) are
called like any other functions without any difference to the
@@ -67,6 +43,57 @@
is to throw an exception. But it can also be used as a fallback
implementation if the NIF library is not implemented for some
architecture.</p>
+ <marker id="WARNING"/>
+ <warning><p><em>Use this functionality with extreme care!</em></p>
+ <p>A native function is executed as a direct extension of the
+ native code of the VM. Execution is not made in a safe environment.
+ The VM can <em>not</em> provide the same services as provided when
+ executing Erlang code, such as preemptive scheduling or memory
+ protection. If the native function doesn't behave well, the whole
+ VM will misbehave.</p>
+ <list>
+ <item><p>A native function that crash will crash the whole VM.</p></item>
+ <item><p>An erroneously implemented native function might cause
+ a VM internal state inconsistency which may cause a crash of the VM,
+ or miscellaneous misbehaviors of the VM at any point after the call
+ to the native function.</p></item>
+ <item><p>A native function that do <seealso marker="#lengthy_work">lengthy
+ work</seealso> before returning will degrade responsiveness of the VM,
+ and may cause miscellaneous strange behaviors. Such strange behaviors
+ include, but are not limited to, extreme memory usage, and bad load
+ balancing between schedulers. Strange behaviors that might occur due
+ to lengthy work may also vary between OTP releases.</p></item>
+ </list>
+ </warning>
+
+ <p>The NIF concept is officially supported from R14B. NIF source code
+ written for earlier experimental versions might need adaption to run on R14B
+ or later versions:</p>
+ <list>
+ <item>No incompatible changes between <em>R14B</em> and R14A.</item>
+ <item>Incompatible changes between <em>R14A</em> and R13B04:
+ <list>
+ <item>Environment argument removed for <c>enif_alloc</c>,
+ <c>enif_realloc</c>, <c>enif_free</c>, <c>enif_alloc_binary</c>,
+ <c>enif_realloc_binary</c>, <c>enif_release_binary</c>,
+ <c>enif_alloc_resource</c>, <c>enif_release_resource</c>,
+ <c>enif_is_identical</c> and <c>enif_compare</c>.</item>
+ <item>Character encoding argument added to <c>enif_get_atom</c>
+ and <c>enif_make_existing_atom</c>.</item>
+ <item>Module argument added to <c>enif_open_resource_type</c>
+ while changing name spaces of resource types from global to module local.</item>
+ </list>
+ </item>
+ <item>Incompatible changes between <em>R13B04</em> and R13B03:
+ <list>
+ <item>The function prototypes of the NIFs have changed to expect <c>argc</c> and <c>argv</c>
+ arguments. The arity of a NIF is by that no longer limited to 3.</item>
+ <item><c>enif_get_data</c> renamed as <c>enif_priv_data</c>.</item>
+ <item><c>enif_make_string</c> got a third argument for character encoding.</item>
+ </list>
+ </item>
+ </list>
+
<p>A minimal example of a NIF library can look like this:</p>
<p/>
<code type="none">
@@ -136,7 +163,23 @@ ok
then retrieved by calling <seealso marker="#enif_priv_data">enif_priv_data</seealso>.</p>
<p>There is no way to explicitly unload a NIF library. A library will be
automatically unloaded when the module code that it belongs to is purged
- by the code server.</p>
+ by the code server.</p>
+
+ <p><marker id="lengthy_work"/>
+ As mentioned in the <seealso marker="#WARNING">warning</seealso> text at
+ the beginning of this document it is of vital importance that a native function
+ does return relatively fast. It is hard to give an exact maximum amount
+ of time that a native function is allowed to work, but as a rule of thumb
+ a well behaving native function should return to its caller before a
+ millisecond has passed. This can be achieved using different approaches.
+ If you have full control over the code that are to execute in the native
+ function, the best approach is to divide the work into multiple chunks of
+ work and call the native function multiple times. This might, however,
+ not always be possible, e.g. when calling third party libraries. In this
+ case you typically want to dispatch the work to another thread, return
+ from the native function, and wait for the result. The thread can send
+ the result back to the calling thread using message passing. Information
+ about thread primitives can be found below.</p>
</description>
<section>
<title>FUNCTIONALITY</title>
@@ -266,10 +309,6 @@ ok
mutable.</p>
<p>The library initialization callbacks <c>load</c>, <c>reload</c> and
<c>upgrade</c> are all thread-safe even for shared state data.</p>
- <p>Avoid doing lengthy work in NIF calls as that may degrade the
- responsiveness of the VM. NIFs are called directly by the same scheduler
- thread that executed the calling Erlang code. The calling scheduler will thus
- be blocked from doing any other work until the NIF returns.</p>
</item>
</taglist>
</section>
diff --git a/erts/emulator/beam/erl_bits.c b/erts/emulator/beam/erl_bits.c
index 6bc227eeda..3753b618e1 100644
--- a/erts/emulator/beam/erl_bits.c
+++ b/erts/emulator/beam/erl_bits.c
@@ -1247,6 +1247,12 @@ erts_bs_append(Process* c_p, Eterm* reg, Uint live, Eterm build_size_term,
*/
erts_bin_offset = 8*sb->size + sb->bitsize;
+ if (unit > 1) {
+ if ((unit == 8 && (erts_bin_offset & 7) != 0) ||
+ (erts_bin_offset % unit) != 0) {
+ goto badarg;
+ }
+ }
used_size_in_bits = erts_bin_offset + build_size_in_bits;
sb->is_writable = 0; /* Make sure that no one else can write. */
pb->size = NBYTES(used_size_in_bits);
@@ -1316,6 +1322,12 @@ erts_bs_append(Process* c_p, Eterm* reg, Uint live, Eterm build_size_term,
*/
ERTS_GET_BINARY_BYTES(bin, src_bytes, bitoffs, bitsize);
erts_bin_offset = 8*binary_size(bin) + bitsize;
+ if (unit > 1) {
+ if ((unit == 8 && (erts_bin_offset & 7) != 0) ||
+ (erts_bin_offset % unit) != 0) {
+ goto badarg;
+ }
+ }
used_size_in_bits = erts_bin_offset + build_size_in_bits;
used_size_in_bytes = NBYTES(used_size_in_bits);
bin_size = 2*used_size_in_bytes;
@@ -1363,12 +1375,6 @@ erts_bs_append(Process* c_p, Eterm* reg, Uint live, Eterm build_size_term,
/*
* Now copy the data into the binary.
*/
- if (unit > 1) {
- if ((unit == 8 && (erts_bin_offset & 7) != 0) ||
- (erts_bin_offset % unit) != 0) {
- return THE_NON_VALUE;
- }
- }
copy_binary_to_buffer(erts_current_bin, 0, src_bytes, bitoffs, erts_bin_offset);
return make_binary(sb);
diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c
index 51bdf53823..7409564167 100644
--- a/erts/emulator/beam/erl_db.c
+++ b/erts/emulator/beam/erl_db.c
@@ -250,7 +250,6 @@ free_dbtable(DbTable* tb)
#endif
ASSERT(is_immed(tb->common.heir_data));
erts_db_free(ERTS_ALC_T_DB_TABLE, tb, (void *) tb, sizeof(DbTable));
- ERTS_ETS_MISC_MEM_ADD(-sizeof(DbTable));
ERTS_SMP_MEMORY_BARRIER;
}
@@ -1443,7 +1442,6 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2)
erts_smp_atomic_init_nob(&init_tb.common.memory_size, 0);
tb = (DbTable*) erts_db_alloc(ERTS_ALC_T_DB_TABLE,
&init_tb, sizeof(DbTable));
- ERTS_ETS_MISC_MEM_ADD(sizeof(DbTable));
erts_smp_atomic_init_nob(&tb->common.memory_size,
erts_smp_atomic_read_nob(&init_tb.common.memory_size));
}
@@ -2888,7 +2886,6 @@ void init_db(void)
meta_pid_to_tab = (DbTable*) erts_db_alloc(ERTS_ALC_T_DB_TABLE,
&init_tb,
sizeof(DbTable));
- ERTS_ETS_MISC_MEM_ADD(sizeof(DbTable));
erts_smp_atomic_init_nob(&meta_pid_to_tab->common.memory_size,
erts_smp_atomic_read_nob(&init_tb.common.memory_size));
@@ -2920,7 +2917,6 @@ void init_db(void)
meta_pid_to_fixed_tab = (DbTable*) erts_db_alloc(ERTS_ALC_T_DB_TABLE,
&init_tb,
sizeof(DbTable));
- ERTS_ETS_MISC_MEM_ADD(sizeof(DbTable));
erts_smp_atomic_init_nob(&meta_pid_to_fixed_tab->common.memory_size,
erts_smp_atomic_read_nob(&init_tb.common.memory_size));
diff --git a/erts/emulator/beam/erl_port_task.c b/erts/emulator/beam/erl_port_task.c
index 0f1a0d441a..3dc7c14faf 100644
--- a/erts/emulator/beam/erl_port_task.c
+++ b/erts/emulator/beam/erl_port_task.c
@@ -573,13 +573,8 @@ erts_port_task_schedule(Eterm id,
ERTS_PT_CHK_PRES_PORTQ(runq, pp);
- if (!pp->sched.taskq) {
- pp->sched.taskq = port_taskq_init(port_taskq_alloc(), pp);
- enq_port = !pp->sched.exe_taskq;
- }
-
+ if (!pp->sched.taskq && !pp->sched.exe_taskq) {
#ifdef ERTS_SMP
- if (enq_port) {
ErtsRunQueue *xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL);
if (xrunq) {
/* Port emigrated ... */
@@ -587,12 +582,17 @@ erts_port_task_schedule(Eterm id,
erts_smp_runq_unlock(runq);
runq = xrunq;
}
- }
+ enq_port = !pp->sched.taskq && !pp->sched.exe_taskq;
+#else
+ enq_port = 1;
#endif
+ }
ASSERT(!enq_port || !(runq->flags & ERTS_RUNQ_FLG_SUSPENDED));
- ASSERT(pp->sched.taskq);
+ if (!pp->sched.taskq)
+ pp->sched.taskq = port_taskq_init(port_taskq_alloc(), pp);
+
ASSERT(ptp);
ptp->type = type;
@@ -984,8 +984,9 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp)
#endif
done:
- ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq));
+ runq->scheduler->reductions += reds;
+ ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq));
ERTS_PORT_REDUCTIONS_EXECUTED(runq, reds);
return res;
diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c
index 0fa2def5af..c5127bc29d 100644
--- a/erts/emulator/beam/erl_process.c
+++ b/erts/emulator/beam/erl_process.c
@@ -43,6 +43,9 @@
#include "erl_async.h"
#include "dtrace-wrapper.h"
+#define ERTS_DELAYED_WAKEUP_INFINITY (~(Uint64) 0)
+#define ERTS_DELAYED_WAKEUP_REDUCTIONS ((Uint64) CONTEXT_REDS/2)
+
#define ERTS_RUNQ_CHECK_BALANCE_REDS_PER_SCHED (2000*CONTEXT_REDS)
#define ERTS_RUNQ_CALL_CHECK_BALANCE_REDS \
(ERTS_RUNQ_CHECK_BALANCE_REDS_PER_SCHED/2)
@@ -932,9 +935,15 @@ haw_thr_prgr_current_check_progress(ErtsAuxWorkData *awdp)
}
static ERTS_INLINE erts_aint32_t
-handle_delayed_aux_work_wakeup(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
+handle_delayed_aux_work_wakeup(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting)
{
int jix, max_jix;
+
+ ASSERT(awdp->delayed_wakeup.next != ERTS_DELAYED_WAKEUP_INFINITY);
+
+ if (!waiting && awdp->delayed_wakeup.next > awdp->esdp->reductions)
+ return aux_work;
+
unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_DELAYED_AW_WAKEUP);
ERTS_THR_MEMORY_BARRIER;
@@ -950,11 +959,14 @@ handle_delayed_aux_work_wakeup(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
set_aux_work_flags_wakeup_nob(ERTS_SCHED_SLEEP_INFO_IX(sched-1),
aux_work);
}
+ awdp->delayed_wakeup.next = ERTS_DELAYED_WAKEUP_INFINITY;
return aux_work & ~ERTS_SSI_AUX_WORK_DELAYED_AW_WAKEUP;
}
static ERTS_INLINE void
-schedule_aux_work_wakeup(ErtsAuxWorkData *awdp, int sched, erts_aint32_t aux_work)
+schedule_aux_work_wakeup(ErtsAuxWorkData *awdp,
+ int sched,
+ erts_aint32_t aux_work)
{
int jix = awdp->delayed_wakeup.sched2jix[sched];
if (jix >= 0) {
@@ -967,7 +979,20 @@ schedule_aux_work_wakeup(ErtsAuxWorkData *awdp, int sched, erts_aint32_t aux_wor
awdp->delayed_wakeup.job[jix].sched = sched;
awdp->delayed_wakeup.job[jix].aux_work = aux_work;
}
- set_aux_work_flags_wakeup_nob(awdp->ssi, ERTS_SSI_AUX_WORK_DELAYED_AW_WAKEUP);
+
+ if (awdp->delayed_wakeup.next != ERTS_DELAYED_WAKEUP_INFINITY) {
+ ASSERT(erts_atomic32_read_nob(&awdp->ssi->aux_work)
+ & ERTS_SSI_AUX_WORK_DELAYED_AW_WAKEUP);
+ }
+ else {
+ awdp->delayed_wakeup.next = (awdp->esdp->reductions
+ + ERTS_DELAYED_WAKEUP_REDUCTIONS);
+
+ ASSERT(!(erts_atomic32_read_nob(&awdp->ssi->aux_work)
+ & ERTS_SSI_AUX_WORK_DELAYED_AW_WAKEUP));
+ set_aux_work_flags_wakeup_nob(awdp->ssi,
+ ERTS_SSI_AUX_WORK_DELAYED_AW_WAKEUP);
+ }
}
#endif
@@ -1046,7 +1071,8 @@ misc_aux_work_clean(ErtsThrQ_t *q,
static ERTS_INLINE erts_aint32_t
handle_misc_aux_work(ErtsAuxWorkData *awdp,
- erts_aint32_t aux_work)
+ erts_aint32_t aux_work,
+ int waiting)
{
ErtsThrQ_t *q = &misc_aux_work_queues[awdp->sched_id].q;
@@ -1066,7 +1092,8 @@ handle_misc_aux_work(ErtsAuxWorkData *awdp,
static ERTS_INLINE erts_aint32_t
handle_misc_aux_work_thr_prgr(ErtsAuxWorkData *awdp,
- erts_aint32_t aux_work)
+ erts_aint32_t aux_work,
+ int waiting)
{
if (!erts_thr_progress_has_reached_this(haw_thr_prgr_current(awdp),
awdp->misc.thr_prgr))
@@ -1145,7 +1172,8 @@ erts_notify_check_async_ready_queue(void *vno)
static ERTS_INLINE erts_aint32_t
handle_async_ready(ErtsAuxWorkData *awdp,
- erts_aint32_t aux_work)
+ erts_aint32_t aux_work,
+ int waiting)
{
ErtsSchedulerSleepInfo *ssi = awdp->ssi;
unset_aux_work_flags(ssi, ERTS_SSI_AUX_WORK_ASYNC_READY);
@@ -1167,7 +1195,8 @@ handle_async_ready(ErtsAuxWorkData *awdp,
static ERTS_INLINE erts_aint32_t
handle_async_ready_clean(ErtsAuxWorkData *awdp,
- erts_aint32_t aux_work)
+ erts_aint32_t aux_work,
+ int waiting)
{
void *thr_prgr_p;
@@ -1203,7 +1232,7 @@ handle_async_ready_clean(ErtsAuxWorkData *awdp,
#endif
static ERTS_INLINE erts_aint32_t
-handle_fix_alloc(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
+handle_fix_alloc(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting)
{
ErtsSchedulerSleepInfo *ssi = awdp->ssi;
erts_aint32_t res;
@@ -1237,7 +1266,7 @@ erts_alloc_notify_delayed_dealloc(int ix)
}
static ERTS_INLINE erts_aint32_t
-handle_delayed_dealloc(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
+handle_delayed_dealloc(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting)
{
ErtsSchedulerSleepInfo *ssi = awdp->ssi;
int need_thr_progress = 0;
@@ -1275,7 +1304,7 @@ handle_delayed_dealloc(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
}
static ERTS_INLINE erts_aint32_t
-handle_delayed_dealloc_thr_prgr(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
+handle_delayed_dealloc_thr_prgr(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting)
{
ErtsSchedulerSleepInfo *ssi;
int need_thr_progress;
@@ -1403,7 +1432,7 @@ erts_smp_notify_check_children_needed(void)
}
static ERTS_INLINE erts_aint32_t
-handle_check_children(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
+handle_check_children(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting)
{
unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_CHECK_CHILDREN);
erts_check_children();
@@ -1426,7 +1455,7 @@ erts_smp_atomic32_t erts_halt_progress;
int erts_halt_code;
static ERTS_INLINE erts_aint32_t
-handle_reap_ports(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
+handle_reap_ports(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting)
{
unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_REAP_PORTS);
awdp->esdp->run_queue->halt_in_progress = 1;
@@ -1474,7 +1503,7 @@ handle_reap_ports(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
#if HAVE_ERTS_MSEG
static ERTS_INLINE erts_aint32_t
-handle_mseg_cache_check(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
+handle_mseg_cache_check(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting)
{
unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_MSEG_CACHE_CHECK);
erts_mseg_cache_check();
@@ -1484,7 +1513,7 @@ handle_mseg_cache_check(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
#endif
static ERTS_INLINE erts_aint32_t
-handle_setup_aux_work_timer(ErtsAuxWorkData *awdp, erts_aint32_t aux_work)
+handle_setup_aux_work_timer(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting)
{
unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_SET_TMO);
setup_aux_work_timer();
@@ -1498,7 +1527,7 @@ handle_aux_work(ErtsAuxWorkData *awdp, erts_aint32_t orig_aux_work, int waiting)
#define HANDLE_AUX_WORK(FLG, HNDLR) \
ignore |= FLG; \
if (aux_work & FLG) { \
- aux_work = HNDLR(awdp, aux_work); \
+ aux_work = HNDLR(awdp, aux_work, waiting); \
ERTS_DBG_CHK_AUX_WORK_VAL(aux_work); \
if (!(aux_work & ~ignore)) { \
ERTS_DBG_CHK_AUX_WORK_VAL(aux_work); \
@@ -3917,6 +3946,7 @@ init_aux_work_data(ErtsAuxWorkData *awdp, ErtsSchedulerData *esdp, char *dawwp)
awdp->async_ready.queue = NULL;
#endif
#ifdef ERTS_SMP
+ awdp->delayed_wakeup.next = ERTS_DELAYED_WAKEUP_INFINITY;
if (!dawwp) {
awdp->delayed_wakeup.job = NULL;
awdp->delayed_wakeup.sched2jix = NULL;
@@ -4121,6 +4151,9 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online)
#ifdef ERTS_SMP
daww_ptr += daww_sz;
#endif
+
+ esdp->reductions = 0;
+
init_sched_wall_time(&esdp->sched_wall_time);
}
@@ -6664,6 +6697,9 @@ Process *schedule(Process *p, int calls)
|| p->rcount == 0);
}
#endif
+
+ esdp->reductions += reds;
+
erts_smp_runq_lock(rq);
ERTS_PROC_REDUCTIONS_EXECUTED(rq, p->prio, reds, actual_reds);
diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h
index 9e7a5a5c74..7c481a91dd 100644
--- a/erts/emulator/beam/erl_process.h
+++ b/erts/emulator/beam/erl_process.h
@@ -439,6 +439,7 @@ typedef struct {
#endif
#ifdef ERTS_SMP
struct {
+ Uint64 next;
int *sched2jix;
int jix;
ErtsDelayedAuxWorkWakeupJob *job;
@@ -481,6 +482,7 @@ struct ErtsSchedulerData_ {
ErtsSchedAllocData alloc_data;
+ Uint64 reductions;
ErtsSchedWallTime sched_wall_time;
#ifdef ERTS_DO_VERIFY_UNUSED_TEMP_ALLOC
diff --git a/erts/emulator/hipe/hipe_amd64_bifs.m4 b/erts/emulator/hipe/hipe_amd64_bifs.m4
index ec25c0b9b7..42e8b2a9b0 100644
--- a/erts/emulator/hipe/hipe_amd64_bifs.m4
+++ b/erts/emulator/hipe/hipe_amd64_bifs.m4
@@ -30,12 +30,12 @@ include(`hipe/hipe_amd64_asm.m4')
#define TEST_GOT_EXN cmpq $THE_NON_VALUE, %rax
#endif'
-define(TEST_GOT_MBUF,`movq P_MBUF(P), %rdx # `TEST_GOT_MBUF'
+define(TEST_GOT_MBUF,`movq P_MBUF(P), %rdx /* `TEST_GOT_MBUF' */
testq %rdx, %rdx
jnz 3f
2:')
define(HANDLE_GOT_MBUF,`
-3: call nbif_$1_gc_after_bif # `HANDLE_GOT_MBUF'
+3: call nbif_$1_gc_after_bif /* `HANDLE_GOT_MBUF' */
jmp 2b')
`#if defined(ERTS_ENABLE_LOCK_CHECK) && defined(ERTS_SMP)
diff --git a/erts/emulator/hipe/hipe_bif_list.m4 b/erts/emulator/hipe/hipe_bif_list.m4
index 942fa0c5cb..7f78ddd57f 100644
--- a/erts/emulator/hipe/hipe_bif_list.m4
+++ b/erts/emulator/hipe/hipe_bif_list.m4
@@ -245,7 +245,7 @@ noproc_primop_interface_5(nbif_bs_put_big_integer, hipe_bs_put_big_integer)
gc_bif_interface_0(nbif_check_get_msg, hipe_check_get_msg)
-#ifdef NO_FPE_SIGNALS
+#`ifdef' NO_FPE_SIGNALS
nocons_nofail_primop_interface_0(nbif_emulate_fpe, hipe_emulate_fpe)
#endif
diff --git a/erts/emulator/hipe/hipe_x86_bifs.m4 b/erts/emulator/hipe/hipe_x86_bifs.m4
index 3cb7d67be0..8cc6340933 100644
--- a/erts/emulator/hipe/hipe_x86_bifs.m4
+++ b/erts/emulator/hipe/hipe_x86_bifs.m4
@@ -35,12 +35,12 @@ include(`hipe/hipe_x86_asm.m4')
# define CALL_BIF(F) call CSYM(F)
#endif'
-define(TEST_GOT_MBUF,`movl P_MBUF(P), %edx # `TEST_GOT_MBUF'
+define(TEST_GOT_MBUF,`movl P_MBUF(P), %edx /* `TEST_GOT_MBUF' */
testl %edx, %edx
jnz 3f
2:')
define(HANDLE_GOT_MBUF,`
-3: call nbif_$1_gc_after_bif # `HANDLE_GOT_MBUF'
+3: call nbif_$1_gc_after_bif /* `HANDLE_GOT_MBUF' */
jmp 2b')
/*
@@ -70,7 +70,7 @@ ASYM($1):
NBIF_ARG_REG(0,P)
NBIF_ARG(2,1,0)
lea 8(%esp), %eax
- NBIF_ARG_REG(1,%eax) # BIF__ARGS
+ NBIF_ARG_REG(1,%eax) /* BIF__ARGS */
CALL_BIF($2)
TEST_GOT_MBUF
@@ -105,7 +105,7 @@ ASYM($1):
NBIF_ARG(2,2,0)
NBIF_ARG(3,2,1)
lea 8(%esp), %eax
- NBIF_ARG_REG(1,%eax) # BIF__ARGS
+ NBIF_ARG_REG(1,%eax) /* BIF__ARGS */
CALL_BIF($2)
TEST_GOT_MBUF
@@ -141,7 +141,7 @@ ASYM($1):
NBIF_ARG(3,3,1)
NBIF_ARG(4,3,2)
lea 8(%esp), %eax
- NBIF_ARG_REG(1,%eax) # BIF__ARGS
+ NBIF_ARG_REG(1,%eax) /* BIF__ARGS */
CALL_BIF($2)
TEST_GOT_MBUF
diff --git a/erts/emulator/test/bs_bit_binaries_SUITE.erl b/erts/emulator/test/bs_bit_binaries_SUITE.erl
index ff1088118d..1428387a65 100644
--- a/erts/emulator/test/bs_bit_binaries_SUITE.erl
+++ b/erts/emulator/test/bs_bit_binaries_SUITE.erl
@@ -177,14 +177,55 @@ append(Config) when is_list(Config) ->
cs_init(),
?line <<(-1):256/signed-unit:8>> = cs(do_append(id(<<>>), 256*8)),
?line <<(-1):256/signed-unit:8>> = cs(do_append2(id(<<>>), 256*4)),
+ <<(-1):256/signed-unit:8>> = cs(do_append3(id(<<>>), 256*8)),
cs_end().
do_append(Bin, N) when N > 0 -> do_append(<<Bin/bits,1:1>>, N-1);
do_append(Bin, 0) -> Bin.
-do_append2(Bin, N) when N > 0 -> do_append2(<<Bin/bits,3:2>>, N-1);
+do_append2(Bin, N) when N > 0 -> do_append2(<<Bin/binary-unit:2,3:2>>, N-1);
do_append2(Bin, 0) -> Bin.
+do_append3(Bin, N) when N > 0 ->
+ Bits = bit_size(Bin),
+ if
+ Bits rem 2 =:= 0 ->
+ do_append3(<<Bin/binary-unit:2,1:1>>, N-1);
+ Bits rem 3 =:= 0 ->
+ do_append3(<<Bin/binary-unit:3,1:1>>, N-1);
+ Bits rem 4 =:= 0 ->
+ do_append3(<<Bin/binary-unit:4,1:1>>, N-1);
+ Bits rem 5 =:= 0 ->
+ do_append3(<<Bin/binary-unit:5,1:1>>, N-1);
+ Bits rem 6 =:= 0 ->
+ do_append3(<<Bin/binary-unit:6,1:1>>, N-1);
+ Bits rem 7 =:= 0 ->
+ do_append3(<<Bin/binary-unit:7,1:1>>, N-1);
+ Bits rem 8 =:= 0 ->
+ do_append3(<<Bin/binary-unit:8,1:1>>, N-1);
+ Bits rem 9 =:= 0 ->
+ do_append3(<<Bin/binary-unit:9,1:1>>, N-1);
+ Bits rem 10 =:= 0 ->
+ do_append3(<<Bin/binary-unit:10,1:1>>, N-1);
+ Bits rem 11 =:= 0 ->
+ do_append3(<<Bin/binary-unit:11,1:1>>, N-1);
+ Bits rem 12 =:= 0 ->
+ do_append3(<<Bin/binary-unit:12,1:1>>, N-1);
+ Bits rem 13 =:= 0 ->
+ do_append3(<<Bin/binary-unit:13,1:1>>, N-1);
+ Bits rem 14 =:= 0 ->
+ do_append3(<<Bin/binary-unit:14,1:1>>, N-1);
+ Bits rem 15 =:= 0 ->
+ do_append3(<<Bin/binary-unit:15,1:1>>, N-1);
+ Bits rem 16 =:= 0 ->
+ do_append3(<<Bin/binary-unit:16,1:1>>, N-1);
+ Bits rem 17 =:= 0 ->
+ do_append3(<<Bin/binary-unit:17,1:1>>, N-1);
+ true ->
+ do_append3(<<Bin/binary-unit:1,1:1>>, N-1)
+ end;
+do_append3(Bin, 0) -> Bin.
+
cs_init() ->
erts_debug:set_internal_state(available_internal_state, true),
ok.
diff --git a/erts/emulator/test/bs_construct_SUITE.erl b/erts/emulator/test/bs_construct_SUITE.erl
index 8b5c82d968..9c88803fea 100644
--- a/erts/emulator/test/bs_construct_SUITE.erl
+++ b/erts/emulator/test/bs_construct_SUITE.erl
@@ -28,7 +28,7 @@
mem_leak/1, coerce_to_float/1, bjorn/1,
huge_float_field/1, huge_binary/1, system_limit/1, badarg/1,
copy_writable_binary/1, kostis/1, dynamic/1, bs_add/1,
- otp_7422/1, zero_width/1]).
+ otp_7422/1, zero_width/1, bad_append/1]).
-include_lib("test_server/include/test_server.hrl").
@@ -38,7 +38,8 @@ all() ->
[test1, test2, test3, test4, test5, testf, not_used,
in_guard, mem_leak, coerce_to_float, bjorn,
huge_float_field, huge_binary, system_limit, badarg,
- copy_writable_binary, kostis, dynamic, bs_add, otp_7422, zero_width].
+ copy_writable_binary, kostis, dynamic, bs_add, otp_7422, zero_width,
+ bad_append].
groups() ->
[].
@@ -827,5 +828,49 @@ zero_width(Config) when is_list(Config) ->
?line {'EXIT',{badarg,_}} = (catch <<(id(not_a_number)):0>>),
ok.
+
+bad_append(_) ->
+ do_bad_append(<<127:1>>, fun append_unit_3/1),
+ do_bad_append(<<127:2>>, fun append_unit_3/1),
+ do_bad_append(<<127:17>>, fun append_unit_3/1),
+
+ do_bad_append(<<127:3>>, fun append_unit_4/1),
+ do_bad_append(<<127:5>>, fun append_unit_4/1),
+ do_bad_append(<<127:7>>, fun append_unit_4/1),
+ do_bad_append(<<127:199>>, fun append_unit_4/1),
+
+ do_bad_append(<<127:7>>, fun append_unit_8/1),
+ do_bad_append(<<127:9>>, fun append_unit_8/1),
+
+ do_bad_append(<<0:8>>, fun append_unit_16/1),
+ do_bad_append(<<0:15>>, fun append_unit_16/1),
+ do_bad_append(<<0:17>>, fun append_unit_16/1),
+ ok.
+
+do_bad_append(Bin0, Appender) ->
+ {'EXIT',{badarg,_}} = (catch Appender(Bin0)),
+
+ Bin1 = id(<<0:3,Bin0/bitstring>>),
+ <<_:3,Bin2/bitstring>> = Bin1,
+ {'EXIT',{badarg,_}} = (catch Appender(Bin2)),
+
+ %% Create a writable binary.
+ Empty = id(<<>>),
+ Bin3 = <<Empty/bitstring,Bin0/bitstring>>,
+ {'EXIT',{badarg,_}} = (catch Appender(Bin3)),
+ ok.
+
+append_unit_3(Bin) ->
+ <<Bin/binary-unit:3,0:1>>.
+
+append_unit_4(Bin) ->
+ <<Bin/binary-unit:4,0:1>>.
+
+append_unit_8(Bin) ->
+ <<Bin/binary,0:1>>.
+
+append_unit_16(Bin) ->
+ <<Bin/binary-unit:16,0:1>>.
+
id(I) -> I.
diff --git a/erts/emulator/valgrind/suppress.halfword b/erts/emulator/valgrind/suppress.halfword
new file mode 100644
index 0000000000..8fe448d897
--- /dev/null
+++ b/erts/emulator/valgrind/suppress.halfword
@@ -0,0 +1,56 @@
+# Extra suppressions specific for the halfword emulator.
+
+# --- Suppress all offheap binaries ---
+# Valgrinds leak check does not recognize pointers that are stored
+# at unaligned addresses. In halfword emulator we store 64-bit pointers
+# to offheap data on 32-bit aligned heaps.
+# We solve this by suppressing allocation of all offheap structures
+# that are not referenced by other tables (ie binaries).
+
+{
+Halfword erts_bin_nrml_alloc
+Memcheck:Leak
+...
+fun:erts_bin_nrml_alloc
+...
+}
+
+{
+Halfword erts_bin_realloc
+Memcheck:Leak
+...
+fun:erts_bin_realloc
+...
+}
+
+{
+Halfword erts_bin_realloc_fnf
+Memcheck:Leak
+...
+fun:erts_bin_realloc_fnf
+...
+}
+
+{
+Halfword erts_bin_drv_alloc
+Memcheck:Leak
+...
+fun:erts_bin_drv_alloc
+...
+}
+
+{
+Halfword erts_bin_drv_alloc_fnf
+Memcheck:Leak
+...
+fun:erts_bin_drv_alloc_fnf
+...
+}
+
+{
+Halfword erts_create_magic_binary
+Memcheck:Leak
+...
+fun:erts_create_magic_binary
+...
+}
diff --git a/erts/emulator/valgrind/suppress.patched.3.6.0 b/erts/emulator/valgrind/suppress.patched.3.6.0
index 62ba032520..b3507bdba7 100644
--- a/erts/emulator/valgrind/suppress.patched.3.6.0
+++ b/erts/emulator/valgrind/suppress.patched.3.6.0
@@ -133,26 +133,18 @@ fun:pthread_create@@GLIBC_2.2.5
{
zlib; ok according to zlib developers
Memcheck:Cond
-fun:longest_match
+...
fun:deflate_slow
fun:deflate
}
{
zlib; ok according to zlib developers
Memcheck:Cond
-fun:longest_match
+...
fun:deflate_fast
fun:deflate
}
{
-zlib; ok accordnig to zlib (this one popped up with valgrind-3.6.0)
-Memcheck:Cond
-fun:deflate_slow
-fun:deflate
-fun:zlib_deflate
-fun:zlib_ctl
-}
-{
No leak; pointer into block
Memcheck:Leak
fun:malloc
@@ -275,6 +267,14 @@ obj:*/ssleay.*
}
{
+ Harmless assembler bug in openssl
+ Memcheck:Addr8
+ ...
+ fun:AES_cbc_encrypt
+ ...
+}
+
+{
erts_bits_init_state; Why is this needed?
Memcheck:Leak
PossiblyLost
diff --git a/erts/emulator/valgrind/suppress.standard b/erts/emulator/valgrind/suppress.standard
index 5a129bfd10..beecf1a7b5 100644
--- a/erts/emulator/valgrind/suppress.standard
+++ b/erts/emulator/valgrind/suppress.standard
@@ -120,14 +120,14 @@ fun:pthread_create@@GLIBC_2.2.5
{
zlib; ok according to zlib developers
Memcheck:Cond
-fun:longest_match
+...
fun:deflate_slow
fun:deflate
}
{
zlib; ok according to zlib developers
Memcheck:Cond
-fun:longest_match
+...
fun:deflate_fast
fun:deflate
}
@@ -252,6 +252,14 @@ obj:*/ssleay.*
}
{
+ Harmless assembler bug in openssl
+ Memcheck:Addr8
+ ...
+ fun:AES_cbc_encrypt
+ ...
+}
+
+{
Prebuilt constant terms in os_info_init (PossiblyLost)
Memcheck:Leak
fun:malloc
generated by cgit v1.2.3 (git 2.25.1) at 2025-01-15 05:21:28 +0000