| Age | Commit message (Collapse) | Author |
|---|
|
* rickard/port-emigrate-bug/maint/OTP-12084:
Verify run-queue asserts
Fix emigrate bug in erts_port_task_schedule()
|
|
In the #erlang IRC channel Anthony Ramine once mentioned the idea of
allowing a NIF to use an emulator trap, similar to a BIF trap, to schedule
another NIF for execution. This is exactly how dirty NIFs were implemented
for Erlang/OTP 17.0, so this commit refactors and generalizes that dirty
NIF code to support a new enif_schedule_nif() API function.
The enif_schedule_nif() function allows a long-running NIF to be broken
into separate NIF invocations. The NIF first executes part of the
long-running task, then calls enif_schedule_nif() to schedule a NIF for
later execution to continue the task. Any number of NIFs can be scheduled
in this manner, one after another. Since the emulator regains control
between invocations, this helps avoid problems caused by native code tying
up scheduler threads for too long.
The enif_schedule_nif() function also replaces the original experimental
dirty NIF API. The function takes a flags parameter that a caller can use
to indicate the NIF should be scheduled onto either a dirty CPU scheduler
thread, a dirty I/O scheduler thread, or scheduled as a regular NIF on a
regular scheduler thread. With this change, the original experimental
enif_schedule_dirty_nif(), enif_schedule_dirty_nif_finalizer() and
enif_dirty_nif_finalizer() API functions are no longer needed and have been
removed. Explicit scheduling of a dirty NIF finalization function is no
longer necessary; if an application wants similar functionality, it can
have a dirty NIF just invoke enif_schedule_nif() to schedule a non-dirty
NIF to complete its task.
Lift the restriction that dirty NIFs can't call enif_make_badarg() to raise
an exception. This was a problem with the original dirty NIF API because it
forced developers to get and check all incoming arguments in a regular NIF,
and then schedule the dirty NIF which then had to get all the arguments
again. Now, the argument checking can be done in the dirty NIF and it can
call enif_make_badarg() itself to flag incorrect arguments.
Extend the ErlNifFunc struct with a new flags field that allows NIFs to be
declared as dirty. The default value for this field is 0, indicating a
regular NIF, so it's backwards compatible with all existing statically
initialized ErlNifFunc struct instances, and so such instances require no
code changes. Defining the flags field with a value of
ERL_NIF_DIRTY_JOB_CPU_BOUND indicates that the NIF should execute on a
dirty CPU scheduler thread, or defining it with a value of
ERL_NIF_DIRTY_JOB_IO_BOUND indicates that the NIF should execute on a dirty
I/O scheduler thread. Any other flags field value causes a NIF library
loading error.
Extend the ErlNifEntry struct with a new options field that indicates
whether a NIF library was built with support for optional features such as
dirty NIFs. When a NIF library is loaded, the runtime checks the options
field to ensure compatibility. If a NIF library built with dirty NIF
support is loaded into a runtime that does not support dirty NIFs, and the
library defines one or more ErlNifFunc entries with non-zero flags fields
indicating dirty NIFs, a NIF library loading error results. There is no
error if a NIF library built with dirty NIF support is loaded into a
runtime that does not support dirty NIFs but the library does not have any
dirty NIFs. It is also not an error if a library without dirty NIF support
is loaded into a runtime built with dirty NIF support.
Add documentation and tests for enif_schedule_nif().
|
|
* egil/proc_lib-optimizations/OTP-12060:
stdlib: Update dependencies to erts-6.2
erts: Update preloaded erlang.beam
stdlib: Use erlang:fun_info_mfa/1 in proc_lib:init_p/3
erts: Test erlang:fun_info_mfa/1
erts: Introduce erlang:fun_info_mfa/1
|
|
* nox/reedr-logging/OTP-12115:
Add number of entries to mnesia copy debug message
Add thread index to allocator enomem dump slogan
Add run queue index to process dump info
Add missing error string to syslog logging in epmd
Demote rare debug slogan of message discarding to debug build
|
|
rickard/port-emigrate-bug/maint/OTP-12084
* rickard/port-emigrate-bug/OTP-12084:
Verify run-queue asserts
Fix emigrate bug in erts_port_task_schedule()
Conflicts:
erts/emulator/beam/erl_port_task.c
|
|
|
|
* lukas/erts/fix_neg_of_int64_min/OTP-12097:
erts: Fix neg int overflow when sint is min size
|
|
* nox/clang-ubsan/OTP-12097:
Properly handle SINT_MIN in small_to_big()
Use offsetof() in io.c
|
|
* rickard/nosuspend-bug/OTP-12082:
Fix build of test port program
Update Makefile.src
Add async_ports test
Fix abort of nosuspend-tasks in erts_port_task_schedule()
|
|
When INT64_MIN is the value of a Sint64 we have to first cast it to
an Uint64 before negating it. Otherwise we get an integer overflow
which is undefined behaviour and in gcc 4.9 this results in -0 instead
of -9223372036854775808 in gcc 4.8.
|
|
While current run-queue lock is unlocked in the call to
erts_check_emigration_need() from erts_port_task_schedule() the
port can be migrated to another run-queue by another thread.
The code in erts_port_task_schedule() needs to check if this
has occurred when returning from erts_check_emigration_need(), and
if so respect the migration decision. When this was not done,
the thread calling erts_port_task_schedule() held the wrong
run-queue lock which caused invalid updates of the port task queue.
This bug was automatically fixed by the rewrites in the branch
rickard/r16b/port-optimizations-fixes/OTP-10336 (commit
56cef897ca3ad2377e34a6ea5800a54a28cbeb6e) introduced in erts-5.10
and do not effect erts versions after that.
|
|
|
|
The counter for the amount of outstanding data in the port queue
became inconsistent when aborting nosuspend-signals in
erts_port_task_schedule(). This since the counter was subtracted
by the data size of the signal although the data size had never
been added to it. This inconsistency caused the port queue to
remain in a busy state forever.
|
|
|
|
Introduced for proc_lib:init_p/3
|
|
|
|
As there is no overflow for signed integers, -SINT_MIN is undefined behaviour
and the cast to unsigned needs to happen before negation. SINT_MIN denotes the
minimum value that can be stored in the Sint type.
beam/big.c:1512:6: runtime error: negation of -9223372036854775808 cannot be represented in type 'Sint' (aka 'long'); cast to an unsigned type to negate this value to itself
|
|
This silences the following UBSan errors:
beam/io.c:7131:27: runtime error: member access within null pointer of type 'ErlDrvSysInfo'
beam/io.c:7140:20: runtime error: member access within null pointer of type 'ErlDrvSysInfo'
beam/io.c:7166:20: runtime error: member access within null pointer of type 'ErlDrvSysInfo'
beam/io.c:7174:20: runtime error: member access within null pointer of type 'ErlDrvSysInfo'
|
|
|
|
* nox/fix-perms:
Fix permissions of some files in the repository
|
|
* rickard/binary_to_term_error/OTP-11931:
Replace erlang:binary_to_term() Erlang wrappers
Conflicts:
erts/preloaded/ebin/erts_internal.beam
|
|
* rickard/yielding-binary-list-conversions/OTP-11888:
Add test-case comparing old and new implementations
Make binary BIFs converting from lists yield on large input
Make binary BIFs converting to lists yield on large input
|
|
* rickard/disable-gc-fix/OTP-11887:
Do not GC other processes in non-smp runtime
Fix conversion of empty string in erts_convert_native_to_filename()
Add support for failing in BIF that has trapped
HiPE wrappers for BIFs disabling GC
|
|
Replace the 'erlang:binary_to_term/1' and 'erlang:binary_to_term/2'
Erlang wrappers taking care of failure after yield with management
of this in the hidden yield BIF.
|
|
* etsukata/system_info_tolerant_timeofday/OTP-11970:
Add erlang:system_info(tolerant_timeofday)
|
|
* lukas/erts/fix_trace_on_registered_procs/OTP-11968:
erts: tracing on send now works for registered processes
|
|
This bug was introduced in R16B. Testcases have been adapted to
verify the correct behaviour.
|
|
* fenek/fix/vstudio_macro:
Add Visual Studio macros to erl_driver.h and ei.h
|
|
These files aren't supposed to be executable. For reference, the command used to
find them was:
git ls-files -z | xargs -0 -J % find % -type f -perm ++x
|
|
- erlang:list_to_binary/1
- erlang:iolist_to_binary/1
- erlang:list_to_bitstring/1
- binary:list_to_bin/1
|
|
|
|
- erlang:binary_to_list/1
- erlang:binary_to_list/3
- erlang:bitstring_to_list/1
|
|
|
|
|
|
|
|
* egil/fix-erts_debug-size/OTP-11923:
erts: Update preloaded erts_internal.beam
erts: Add spec for erts_internal:map_to_tuple_keys/1
erts: Add testcase for erts_debug:size/1 Map terms
kernel: Fix erts_debug:size/1 to handle Map sizes
erts: Add erts_internal:map_to_tuple_keys/1
|
|
|
|
* sverk/ets-delete-unfix-race/OTP-11892:
Fix race between ETS table deletion and unfixation
erts: Add etp commands for alloc_util block and carrier inspection
|
|
* lukas/erts/suspend_assert/OTP-11906:
erts: Fix faulty process suspend assert
|
|
|
|
* lukas/erts/etp-lc-macros/OTP-11904:
erts: Add etp-lc-dump and etp-ppc-stacktrace macro
|
|
* Used for introspection.
* Will return the internal key tuple if applicable
* Not documented - not for public use
|
|
Symptom: VM crash running mnesia_SUITE
Scenario: Process A terminates while still having fixed table T
and process B "at the same time" deletes table T with ets:delete/1 or
by terminating.
Problem: A table scheduled for deallocation do only have a valid
'common' part. The unfix-table-at-process-exit code tried to read the
hash-specific part of such a table.
Solution: Must back off if DB_DELETE flag is set.
Since: R16B
|
|
Add erlang:system_info(tolerant_timeofday), an API to check whether
compensation for sudden changes of system time is enabled or not.
|
|
No message for large_heap monitoring was ever sent on non-smp VM.
Bug introduced in R16B.
|
|
* sverk/suspend-running-race:
erts: Fix bug of scheduling a suspended process
|
|
Reference-count the NIF before and after invoking a NIF on dirty schedulers
to prevent having the NIF purged during the call.
|
|
Dirty NIF support used an Export structure to facilitate calls to dirty
NIFs and finalizers, but Export isn't large enough to hold all necessary
data. This was causing an invalid memory read in beam_emu.c past the end of
the Export object. Add a local extended Export struct to erl_nif.c that can
hold all the necessary data.
|
|
|
|
This is a race that can cause RUNNING (instead of RUNNING_SYS)
set on a SUSPENDED process. The effect of this race happening is probably
quite benign.
The bug was discovered by process_SUITE:processes_gc_trap on debug VM
crashing on last assert in schedule():
/* Never run a suspended process */
ASSERT(!(ERTS_PSFLG_SUSPENDED & erts_smp_atomic32_read_nob(&p->state)));
|
