| Age | Commit message (Collapse) | Author |
|---|
|
* vinoski/ds:
initial support for dirty schedulers and dirty NIFs
|
|
Add initial support for dirty schedulers.
There are two types of dirty schedulers: CPU schedulers and I/O
schedulers. By default, there are as many dirty CPU schedulers as there are
normal schedulers and as many dirty CPU schedulers online as normal
schedulers online. There are 10 dirty I/O schedulers (similar to the choice
of 10 as the default for async threads).
By default, dirty schedulers are disabled and conditionally compiled
out. To enable them, you must pass --enable-dirty-schedulers to the
top-level configure script when building Erlang/OTP.
Current dirty scheduler support requires the emulator to be built with SMP
support. This restriction will be lifted in the future.
You can specify the number of dirty schedulers with the command-line
options +SDcpu (for dirty CPU schedulers) and +SDio (for dirty I/O
schedulers). The +SDcpu option is similar to the +S option in that it takes
two numbers separated by a colon: C1:C2, where C1 specifies the number of
dirty schedulers available and C2 specifies the number of dirty schedulers
online. The +SDPcpu option allows numbers of dirty CPU schedulers available
and dirty CPU schedulers online to be specified as percentages, similar to
the existing +SP option for normal schedulers. The number of dirty CPU
schedulers created and dirty CPU schedulers online may not exceed the
number of normal schedulers created and normal schedulers online,
respectively. The +SDio option takes only a single number specifying the
number of dirty I/O schedulers available and online. There is no support
yet for programmatically changing at run time the number of dirty CPU
schedulers online via erlang:system_flag/2. Also, changing the number of
normal schedulers online via erlang:system_flag(schedulers_online,
NewSchedulersOnline) should ensure that there are no more dirty CPU
schedulers than normal schedulers, but this is not yet implemented. You can
retrieve the number of dirty schedulers by passing dirty_cpu_schedulers,
dirty_cpu_schedulers_online, or dirty_io_schedulers to
erlang:system_info/1.
Currently only NIFs are able to access dirty scheduler
functionality. Neither drivers nor BIFs currently support dirty
schedulers. This restriction will be addressed in the future.
If dirty scheduler support is present in the runtime, the initial status
line Erlang prints before presenting its interactive prompt will include
the indicator "[ds:C1:C2:I]" where "ds" indicates "dirty schedulers", "C1"
indicates the number of dirty CPU schedulers available, "C2" indicates the
number of dirty CPU schedulers online, and "I" indicates the number of
dirty I/O schedulers.
Document The dirty NIF API in the erl_nif man page. The API closely follows
Rickard Green's presentation slides from his talk "Future Extensions to the
Native Interface", presented at the 2011 Erlang Factory held in the San
Francisco Bay Area. Rickard's slides are available online at
http://bit.ly/1m34UHB .
Document the new erl command-line options, the additions to
erlang:system_info/1, and also add the erlang:system_flag/2 dirty scheduler
documentation even though it's not yet implemented.
To determine whether the dirty NIF API is available, native code can check
to see whether the C preprocessor macro ERL_NIF_DIRTY_SCHEDULER_SUPPORT is
defined. To check if dirty schedulers are available at run time, native
code can call the boolean enif_have_dirty_schedulers() function, and Erlang
code can call erlang:system_info(dirty_cpu_schedulers), which raises
badarg if no dirty scheduler support is available.
Add a simple dirty NIF test to the emulator NIF suite.
|
|
OTP-11628
* vinoski/rm-drv-async-cancel:
remove deprecated driver_async_cancel function
|
|
OTP-11618
* sverk/misc-bug-fixes:
erts: Fix faulty assert in match spec engine.
erts: Fix crash when comparing very large floats with integers
erts: Refactor big-float compare on HALFWORD to use C-stack
erts: Fix halfword compile errors in ESTACK
|
|
* egil/gc-increase-tenure-rate/OTP-11617:
erts: Increase gc tenure rate
|
|
* rickard/load_balance/OTP-11385:
Fix usage of non-empty run-queue flag
|
|
|
|
* rickard/load_balance/OTP-11385:
Disable scheduler utilization balancing if +scl true is passed
|
|
|
|
* rickard/load_balance/OTP-11385:
Add support for scheduler utilization balancing
|
|
* rickard/default_acul/OTP-11604:
erts: Use "+Muacul de" as default
|
|
For more information see documentation of the new command line argument +sub
|
|
* vinoski/system-flag-warn-msg:
fix system_flag deprecation warnings
OTP-11602
|
|
|
|
big_buf was one word too short on 32-bit emulators causing
memory corruption.
Seems like this did not cause a problem before the ESTACK memory layout
was changed in 172ebf11dc455e22b87f.
|
|
for the temporary conversion from float to big.
Preparation for coming bugfix of 'big_buf' array size.
|
|
Errors introduced in 172ebf11dc455e2
|
|
* sverk/term2bin-simplify:
erts: Refactor ESTACK & WSTACK to use a struct easy to "export"
erts: Fix benign ESTACK/WSTACK typo
erts: Fix compiler warnings for NO_JUMP_TABLE
erts: Run binary_SUITE:trapping even for 32bit
erts: Extend binary_SUITE:ttb_trap to also cover binary_to_term
erts: Remove the extra_root feature from the process structure
erts: Simplify term_to_binary by removing saved ESTACK from root set
|
|
OTP-11585
* sverk/bin2term-int-size-estimation-bug:
erts: Fix useless comparisons in binary_SUITE:external_size
erts: Reduce heap usage for binary_SUITE:deep
erts: Remove overestimation of heap space in binary_to_term
|
|
* egil/fix-bs_get_integer/OTP-11581:
erts: Fix bs_get_integer instruction
|
|
This is not a clean refactor. It changes the behaviour slightly
of E/WSTACK_RESTORE. The allocated stack from E/WSTACK_SAVE is used
as-is and not copied into default_stack. This will hopefully fix an
illusive memory leak that valgrind is reporting.
|
|
Passing cpu_topology or scheduler_bind_type to erlang:system_flag/2 results
in an error report warning that the argument is deprecated and is slated
for removal in erts-5.10/OTP-R16. Since we're already past that version and
no substitute approach has yet been decided for these features, keep the
deprecation warning but bump the removal version it mentions up to Erlang
18.
|
|
|
|
|
|
Some time ago the driver_async_cancel function was deprecated and slated
for removal in R17. This commit removes the function along with its
associated tests and documentation, sets the
ERL_DRV_EXTENDED_MAJOR_VERSION to 3 and ERL_DRV_EXTENDED_MINOR_VERSION to
0, and modifies the sys_info_base_drv and sys_info_prev_drv tests in the
driver test suite to check version 3.0 instead of 2.0.
|
|
* rickard/otp-17-vsn-fix:
Fix issues with new versioning
|
|
as we don't use it and instead have the feature to disable GC
during trapping BIFs.
|
|
We disabled GC (in 522a29666088d5) during trapping and don't need to
include the saved ESTACK as part of root set.
|
|
|
|
The garbage collector tries to maintain the previous heap block size
during a minor gc, i.e. 'need' is not utilized in determining the size of
the new heap, instead it relies on tenure and garbage to be sufficiently large.
In instances during intense growing with exlusively live data on the heap
coupled with delayed tenure, fullsweeps would be triggered directly after
a minor gc to make room for 'need' since the new heap would be full.
To remedy this, the tenure of terms on the minor heap will always happen
(if it is below the high watermark) instead of every other minor gc.
|
|
|
|
for 32-bit integers (INTEGER_EXT) on 64-bit architectures.
|
|
The instruction bs_get_integer could unnecessarily trigger a garbage collection
in failure cases which is unwanted or outright dangerous.
Ex:
<<X:Sz,_/bits>> = <<"some binary">>
Previously, if Sz induced X to a bignum it would reserved memory size this on
the heap via a garbage collection before checking if the size could actually match.
It will now check the binary size before triggering a collection.
|
|
* sverk/re-valgrind-fix:
erts: Fix valgrind warning for re_SUITE:error_handling
|
|
OTP-11549
* sverk/unicode-driver-paths:
erts: Support loading of drivers with unicode paths
erts: Add 'extra' argument to erts_convert_filename_to_encoding
erts: Refactor remove erts_sys_dll_open2
erts: Fix compiler warning
|
|
Yielding binary_to_term.
OTP-11535
* tag 'OTP_R16B03_yielding_binary_to_term':
Increase versions for OTP_R16B03_yielding_binary_to_term
erts: Adjust term_to_binary reduction factors
erts: Yield after trapping term_to_binary if gc has been ordered
erts: Let term_to_binary disable gc while trapping
erts: Improve stress of binary_to_term in binary_SUITE
erts: Fix bug in binary_to_term for compressed on halfword
erts: Fix crash when binary_to_term throws badarg
erts: Trapping memcpy in binary_to_term
erts: Cleanup code for trapping binary_to_term
erts: Add erlang wrappers to binary_to_term
trapping uncompress
trapping size calculation
trapping binary_to_term/2
trapping STRING_EXT
trapping lists and tuples
trapping binary_to_term passing binary_SUITE
Conflicts:
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/erts_internal.beam
erts/vsn.mk
lib/kernel/vsn.mk
lib/stdlib/vsn.mk
|
|
|
|
|
|
|
|
* sverk/trapping-bin2term:
erts: Adjust term_to_binary reduction factors
erts: Yield after trapping term_to_binary if gc has been ordered
erts: Let term_to_binary disable gc while trapping
erts: Improve stress of binary_to_term in binary_SUITE
erts: Fix bug in binary_to_term for compressed on halfword
erts: Fix crash when binary_to_term throws badarg
erts: Trapping memcpy in binary_to_term
erts: Cleanup code for trapping binary_to_term
erts: Add erlang wrappers to binary_to_term
trapping uncompress
trapping size calculation
trapping binary_to_term/2
trapping STRING_EXT
trapping lists and tuples
trapping binary_to_term passing binary_SUITE
Parallel check_process_code when code_server purge a module
Functionality for disabling garbage collection
Use asynchronous check_process_code in code_parallel_SUITE
Execution of system tasks in context of another process
Conflicts:
erts/emulator/beam/external.c
erts/emulator/beam/sys.h
erts/emulator/test/binary_SUITE.erl
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/erts_internal.beam
|
|
by making sure we at least can read the first four bytes
of a compiled regexp where the 'magic_number' is located.
|
|
Update versions of OTP, erts, kernel, and stdlib to comply with
the new version scheme decided by the OTP technical board.
|
|
* rickard/garbage_collect/OTP-11388:
Parallel check_process_code when code_server purge a module
Functionality for disabling garbage collection
Use asynchronous check_process_code in code_parallel_SUITE
Execution of system tasks in context of another process
Conflicts:
bootstrap/lib/kernel/ebin/hipe_unified_loader.beam
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/erts_internal.beam
|
|
|
|
|
|
The test needs to be false to fail, not true. This was noticed with the
following warning, where marking erl_assert_error as non-returning didn't
help:
beam/erl_gc.c:2772:2: warning: variable 'refc' is used uninitialized whenever switch default is taken [-Wsometimes-uninitialized]
default:
^~~~~~~
beam/erl_gc.c:2775:26: note: uninitialized use occurs here
ERTS_CHK_OFFHEAP_ASSERT(refc >= 1);
^~~~
beam/erl_gc.c:2738:11: note: expanded from macro 'ERTS_CHK_OFFHEAP_ASSERT'
if (!(EXP)) \
^
beam/erl_gc.c:2759:18: note: initialize the variable 'refc' to silence this warning
erts_aint_t refc;
^
= 0
|
|
|
|
When the offending code is reached, new_crr is either uninitalized or
have been set to NULL. This patch removes the following warning:
beam/erl_alloc_util.c:3510:6: warning: variable 'new_crr' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
if (!(flags & CFLG_FORCE_MSEG)) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
beam/erl_alloc_util.c:3567:23: note: uninitialized use occurs here
DEBUG_SAVE_ALIGNMENT(new_crr);
^~~~~~~
beam/erl_alloc_util.c:674:51: note: expanded from macro 'DEBUG_SAVE_ALIGNMENT'
UWord algnmnt__ = sizeof(Unit_t) - (((UWord) (C)) % sizeof(Unit_t));\
^
beam/erl_alloc_util.c:3510:2: note: remove the 'if' if its condition is always true
if (!(flags & CFLG_FORCE_MSEG)) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
beam/erl_alloc_util.c:3438:23: note: initialize the variable 'new_crr' to silence this warning
Carrier_t *new_crr, *old_crr;
^
= NULL
|
|
|
|
* sverk/bit-cmp-bug/OTP-11501:
erts: Fix invalid read when appending binaries during call trace
erts: Optimize comparison for bitstrings with byte aligned start
erts: Fix invalid read in bitstring comparison
|
