aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam
AgeCommit message (Collapse)Author
2014-01-28Merge branch 'vinoski/ds'Rickard Green
* vinoski/ds: initial support for dirty schedulers and dirty NIFs
2014-01-28initial support for dirty schedulers and dirty NIFsSteve Vinoski
Add initial support for dirty schedulers. There are two types of dirty schedulers: CPU schedulers and I/O schedulers. By default, there are as many dirty CPU schedulers as there are normal schedulers and as many dirty CPU schedulers online as normal schedulers online. There are 10 dirty I/O schedulers (similar to the choice of 10 as the default for async threads). By default, dirty schedulers are disabled and conditionally compiled out. To enable them, you must pass --enable-dirty-schedulers to the top-level configure script when building Erlang/OTP. Current dirty scheduler support requires the emulator to be built with SMP support. This restriction will be lifted in the future. You can specify the number of dirty schedulers with the command-line options +SDcpu (for dirty CPU schedulers) and +SDio (for dirty I/O schedulers). The +SDcpu option is similar to the +S option in that it takes two numbers separated by a colon: C1:C2, where C1 specifies the number of dirty schedulers available and C2 specifies the number of dirty schedulers online. The +SDPcpu option allows numbers of dirty CPU schedulers available and dirty CPU schedulers online to be specified as percentages, similar to the existing +SP option for normal schedulers. The number of dirty CPU schedulers created and dirty CPU schedulers online may not exceed the number of normal schedulers created and normal schedulers online, respectively. The +SDio option takes only a single number specifying the number of dirty I/O schedulers available and online. There is no support yet for programmatically changing at run time the number of dirty CPU schedulers online via erlang:system_flag/2. Also, changing the number of normal schedulers online via erlang:system_flag(schedulers_online, NewSchedulersOnline) should ensure that there are no more dirty CPU schedulers than normal schedulers, but this is not yet implemented. You can retrieve the number of dirty schedulers by passing dirty_cpu_schedulers, dirty_cpu_schedulers_online, or dirty_io_schedulers to erlang:system_info/1. Currently only NIFs are able to access dirty scheduler functionality. Neither drivers nor BIFs currently support dirty schedulers. This restriction will be addressed in the future. If dirty scheduler support is present in the runtime, the initial status line Erlang prints before presenting its interactive prompt will include the indicator "[ds:C1:C2:I]" where "ds" indicates "dirty schedulers", "C1" indicates the number of dirty CPU schedulers available, "C2" indicates the number of dirty CPU schedulers online, and "I" indicates the number of dirty I/O schedulers. Document The dirty NIF API in the erl_nif man page. The API closely follows Rickard Green's presentation slides from his talk "Future Extensions to the Native Interface", presented at the 2011 Erlang Factory held in the San Francisco Bay Area. Rickard's slides are available online at http://bit.ly/1m34UHB . Document the new erl command-line options, the additions to erlang:system_info/1, and also add the erlang:system_flag/2 dirty scheduler documentation even though it's not yet implemented. To determine whether the dirty NIF API is available, native code can check to see whether the C preprocessor macro ERL_NIF_DIRTY_SCHEDULER_SUPPORT is defined. To check if dirty schedulers are available at run time, native code can call the boolean enif_have_dirty_schedulers() function, and Erlang code can call erlang:system_info(dirty_cpu_schedulers), which raises badarg if no dirty scheduler support is available. Add a simple dirty NIF test to the emulator NIF suite.
2014-01-28Merge branch 'vinoski/rm-drv-async-cancel'Rickard Green
OTP-11628 * vinoski/rm-drv-async-cancel: remove deprecated driver_async_cancel function
2014-01-27Merge branch 'sverk/misc-bug-fixes'Sverker Eriksson
OTP-11618 * sverk/misc-bug-fixes: erts: Fix faulty assert in match spec engine. erts: Fix crash when comparing very large floats with integers erts: Refactor big-float compare on HALFWORD to use C-stack erts: Fix halfword compile errors in ESTACK
2014-01-27Merge branch 'egil/gc-increase-tenure-rate/OTP-11617'Björn-Egil Dahlberg
* egil/gc-increase-tenure-rate/OTP-11617: erts: Increase gc tenure rate
2014-01-27Merge branch 'rickard/load_balance/OTP-11385'Rickard Green
* rickard/load_balance/OTP-11385: Fix usage of non-empty run-queue flag
2014-01-27Fix usage of non-empty run-queue flagRickard Green
2014-01-24Merge branch 'rickard/load_balance/OTP-11385'Rickard Green
* rickard/load_balance/OTP-11385: Disable scheduler utilization balancing if +scl true is passed
2014-01-24Disable scheduler utilization balancing if +scl true is passedRickard Green
2014-01-24Merge branch 'rickard/load_balance/OTP-11385'Rickard Green
* rickard/load_balance/OTP-11385: Add support for scheduler utilization balancing
2014-01-23Merge branch 'rickard/default_acul/OTP-11604'Rickard Green
* rickard/default_acul/OTP-11604: erts: Use "+Muacul de" as default
2014-01-23Add support for scheduler utilization balancingRickard Green
For more information see documentation of the new command line argument +sub
2014-01-23Merge branch 'vinoski/system-flag-warn-msg'Henrik Nord
* vinoski/system-flag-warn-msg: fix system_flag deprecation warnings OTP-11602
2014-01-23erts: Fix faulty assert in match spec engine.Sverker Eriksson
2014-01-22erts: Fix crash when comparing very large floats with integersSverker Eriksson
big_buf was one word too short on 32-bit emulators causing memory corruption. Seems like this did not cause a problem before the ESTACK memory layout was changed in 172ebf11dc455e22b87f.
2014-01-22erts: Refactor big-float compare on HALFWORD to use C-stackSverker Eriksson
for the temporary conversion from float to big. Preparation for coming bugfix of 'big_buf' array size.
2014-01-22erts: Fix halfword compile errors in ESTACKSverker Eriksson
Errors introduced in 172ebf11dc455e2
2014-01-20Merge branch 'sverk/term2bin-simplify'Sverker Eriksson
* sverk/term2bin-simplify: erts: Refactor ESTACK & WSTACK to use a struct easy to "export" erts: Fix benign ESTACK/WSTACK typo erts: Fix compiler warnings for NO_JUMP_TABLE erts: Run binary_SUITE:trapping even for 32bit erts: Extend binary_SUITE:ttb_trap to also cover binary_to_term erts: Remove the extra_root feature from the process structure erts: Simplify term_to_binary by removing saved ESTACK from root set
2014-01-20Merge branch 'sverk/bin2term-int-size-estimation-bug'Sverker Eriksson
OTP-11585 * sverk/bin2term-int-size-estimation-bug: erts: Fix useless comparisons in binary_SUITE:external_size erts: Reduce heap usage for binary_SUITE:deep erts: Remove overestimation of heap space in binary_to_term
2014-01-20Merge branch 'egil/fix-bs_get_integer/OTP-11581'Björn-Egil Dahlberg
* egil/fix-bs_get_integer/OTP-11581: erts: Fix bs_get_integer instruction
2014-01-17erts: Refactor ESTACK & WSTACK to use a struct easy to "export"Sverker Eriksson
This is not a clean refactor. It changes the behaviour slightly of E/WSTACK_RESTORE. The allocated stack from E/WSTACK_SAVE is used as-is and not copied into default_stack. This will hopefully fix an illusive memory leak that valgrind is reporting.
2014-01-15fix system_flag deprecation warningsSteve Vinoski
Passing cpu_topology or scheduler_bind_type to erlang:system_flag/2 results in an error report warning that the argument is deprecated and is slated for removal in erts-5.10/OTP-R16. Since we're already past that version and no substitute approach has yet been decided for these features, keep the deprecation warning but bump the removal version it mentions up to Erlang 18.
2014-01-15erts: Fix benign ESTACK/WSTACK typoSverker Eriksson
2014-01-15erts: Fix compiler warnings for NO_JUMP_TABLESverker Eriksson
2014-01-15remove deprecated driver_async_cancel functionSteve Vinoski
Some time ago the driver_async_cancel function was deprecated and slated for removal in R17. This commit removes the function along with its associated tests and documentation, sets the ERL_DRV_EXTENDED_MAJOR_VERSION to 3 and ERL_DRV_EXTENDED_MINOR_VERSION to 0, and modifies the sys_info_base_drv and sys_info_prev_drv tests in the driver test suite to check version 3.0 instead of 2.0.
2014-01-13Merge branch 'rickard/otp-17-vsn-fix'Rickard Green
* rickard/otp-17-vsn-fix: Fix issues with new versioning
2014-01-10erts: Remove the extra_root feature from the process structureSverker Eriksson
as we don't use it and instead have the feature to disable GC during trapping BIFs.
2014-01-10erts: Simplify term_to_binary by removing saved ESTACK from root setSverker Eriksson
We disabled GC (in 522a29666088d5) during trapping and don't need to include the saved ESTACK as part of root set.
2014-01-10erts: Replace tab with space for proper alignmentLukas Larsson
2014-01-09erts: Increase gc tenure rateBjörn-Egil Dahlberg
The garbage collector tries to maintain the previous heap block size during a minor gc, i.e. 'need' is not utilized in determining the size of the new heap, instead it relies on tenure and garbage to be sufficiently large. In instances during intense growing with exlusively live data on the heap coupled with delayed tenure, fullsweeps would be triggered directly after a minor gc to make room for 'need' since the new heap would be full. To remedy this, the tenure of terms on the minor heap will always happen (if it is below the high watermark) instead of every other minor gc.
2014-01-09Fix issues with new versioningRickard Green
2014-01-08erts: Remove overestimation of heap space in binary_to_termSverker Eriksson
for 32-bit integers (INTEGER_EXT) on 64-bit architectures.
2014-01-07erts: Fix bs_get_integer instructionBjörn-Egil Dahlberg
The instruction bs_get_integer could unnecessarily trigger a garbage collection in failure cases which is unwanted or outright dangerous. Ex: <<X:Sz,_/bits>> = <<"some binary">> Previously, if Sz induced X to a bignum it would reserved memory size this on the heap via a garbage collection before checking if the size could actually match. It will now check the binary size before triggering a collection.
2013-12-20Merge branch 'sverk/re-valgrind-fix'Sverker Eriksson
* sverk/re-valgrind-fix: erts: Fix valgrind warning for re_SUITE:error_handling
2013-12-20Merge branch 'sverk/unicode-driver-paths'Sverker Eriksson
OTP-11549 * sverk/unicode-driver-paths: erts: Support loading of drivers with unicode paths erts: Add 'extra' argument to erts_convert_filename_to_encoding erts: Refactor remove erts_sys_dll_open2 erts: Fix compiler warning
2013-12-17Merge tag 'OTP_R16B03_yielding_binary_to_term'Sverker Eriksson
Yielding binary_to_term. OTP-11535 * tag 'OTP_R16B03_yielding_binary_to_term': Increase versions for OTP_R16B03_yielding_binary_to_term erts: Adjust term_to_binary reduction factors erts: Yield after trapping term_to_binary if gc has been ordered erts: Let term_to_binary disable gc while trapping erts: Improve stress of binary_to_term in binary_SUITE erts: Fix bug in binary_to_term for compressed on halfword erts: Fix crash when binary_to_term throws badarg erts: Trapping memcpy in binary_to_term erts: Cleanup code for trapping binary_to_term erts: Add erlang wrappers to binary_to_term trapping uncompress trapping size calculation trapping binary_to_term/2 trapping STRING_EXT trapping lists and tuples trapping binary_to_term passing binary_SUITE Conflicts: erts/preloaded/ebin/erlang.beam erts/preloaded/ebin/erts_internal.beam erts/vsn.mk lib/kernel/vsn.mk lib/stdlib/vsn.mk
2013-12-16erts: Support loading of drivers with unicode pathsSverker Eriksson
2013-12-16erts: Add 'extra' argument to erts_convert_filename_to_encodingSverker Eriksson
2013-12-13erts: Refactor remove erts_sys_dll_open2Sverker Eriksson
2013-12-12Merge branch 'sverk/trapping-bin2term' into OTP_R16B03Sverker Eriksson
* sverk/trapping-bin2term: erts: Adjust term_to_binary reduction factors erts: Yield after trapping term_to_binary if gc has been ordered erts: Let term_to_binary disable gc while trapping erts: Improve stress of binary_to_term in binary_SUITE erts: Fix bug in binary_to_term for compressed on halfword erts: Fix crash when binary_to_term throws badarg erts: Trapping memcpy in binary_to_term erts: Cleanup code for trapping binary_to_term erts: Add erlang wrappers to binary_to_term trapping uncompress trapping size calculation trapping binary_to_term/2 trapping STRING_EXT trapping lists and tuples trapping binary_to_term passing binary_SUITE Parallel check_process_code when code_server purge a module Functionality for disabling garbage collection Use asynchronous check_process_code in code_parallel_SUITE Execution of system tasks in context of another process Conflicts: erts/emulator/beam/external.c erts/emulator/beam/sys.h erts/emulator/test/binary_SUITE.erl erts/preloaded/ebin/erlang.beam erts/preloaded/ebin/erts_internal.beam
2013-12-11erts: Fix valgrind warning for re_SUITE:error_handlingSverker Eriksson
by making sure we at least can read the first four bytes of a compiled regexp where the 'magic_number' is located.
2013-12-10Update versions of OTP, erts, kernel, and stdlibRickard Green
Update versions of OTP, erts, kernel, and stdlib to comply with the new version scheme decided by the OTP technical board.
2013-12-07Merge branch 'rickard/garbage_collect/OTP-11388'Rickard Green
* rickard/garbage_collect/OTP-11388: Parallel check_process_code when code_server purge a module Functionality for disabling garbage collection Use asynchronous check_process_code in code_parallel_SUITE Execution of system tasks in context of another process Conflicts: bootstrap/lib/kernel/ebin/hipe_unified_loader.beam erts/preloaded/ebin/erlang.beam erts/preloaded/ebin/erts_internal.beam
2013-12-02Merge branch 'maint'Henrik Nord
2013-12-02Compile in_heapfrag() only in debug modeAnthony Ramine
2013-12-02Fix erts_check_off_heap2 assertionAnthony Ramine
The test needs to be false to fail, not true. This was noticed with the following warning, where marking erl_assert_error as non-returning didn't help: beam/erl_gc.c:2772:2: warning: variable 'refc' is used uninitialized whenever switch default is taken [-Wsometimes-uninitialized] default: ^~~~~~~ beam/erl_gc.c:2775:26: note: uninitialized use occurs here ERTS_CHK_OFFHEAP_ASSERT(refc >= 1); ^~~~ beam/erl_gc.c:2738:11: note: expanded from macro 'ERTS_CHK_OFFHEAP_ASSERT' if (!(EXP)) \ ^ beam/erl_gc.c:2759:18: note: initialize the variable 'refc' to silence this warning erts_aint_t refc; ^ = 0
2013-12-02Properly mark erl_assert_error as non-returningAnthony Ramine
2013-12-02Remove uninitialized use of new_crr in erl_alloc_utilAnthony Ramine
When the offending code is reached, new_crr is either uninitalized or have been set to NULL. This patch removes the following warning: beam/erl_alloc_util.c:3510:6: warning: variable 'new_crr' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] if (!(flags & CFLG_FORCE_MSEG)) { ^~~~~~~~~~~~~~~~~~~~~~~~~~ beam/erl_alloc_util.c:3567:23: note: uninitialized use occurs here DEBUG_SAVE_ALIGNMENT(new_crr); ^~~~~~~ beam/erl_alloc_util.c:674:51: note: expanded from macro 'DEBUG_SAVE_ALIGNMENT' UWord algnmnt__ = sizeof(Unit_t) - (((UWord) (C)) % sizeof(Unit_t));\ ^ beam/erl_alloc_util.c:3510:2: note: remove the 'if' if its condition is always true if (!(flags & CFLG_FORCE_MSEG)) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ beam/erl_alloc_util.c:3438:23: note: initialize the variable 'new_crr' to silence this warning Carrier_t *new_crr, *old_crr; ^ = NULL
2013-11-29Merge branch 'maint'Sverker Eriksson
2013-11-29Merge branch 'sverk/bit-cmp-bug/OTP-11501' into maintSverker Eriksson
* sverk/bit-cmp-bug/OTP-11501: erts: Fix invalid read when appending binaries during call trace erts: Optimize comparison for bitstrings with byte aligned start erts: Fix invalid read in bitstring comparison