| Age | Commit message (Collapse) | Author |
|---|
|
Add initial support for dirty schedulers.
There are two types of dirty schedulers: CPU schedulers and I/O
schedulers. By default, there are as many dirty CPU schedulers as there are
normal schedulers and as many dirty CPU schedulers online as normal
schedulers online. There are 10 dirty I/O schedulers (similar to the choice
of 10 as the default for async threads).
By default, dirty schedulers are disabled and conditionally compiled
out. To enable them, you must pass --enable-dirty-schedulers to the
top-level configure script when building Erlang/OTP.
Current dirty scheduler support requires the emulator to be built with SMP
support. This restriction will be lifted in the future.
You can specify the number of dirty schedulers with the command-line
options +SDcpu (for dirty CPU schedulers) and +SDio (for dirty I/O
schedulers). The +SDcpu option is similar to the +S option in that it takes
two numbers separated by a colon: C1:C2, where C1 specifies the number of
dirty schedulers available and C2 specifies the number of dirty schedulers
online. The +SDPcpu option allows numbers of dirty CPU schedulers available
and dirty CPU schedulers online to be specified as percentages, similar to
the existing +SP option for normal schedulers. The number of dirty CPU
schedulers created and dirty CPU schedulers online may not exceed the
number of normal schedulers created and normal schedulers online,
respectively. The +SDio option takes only a single number specifying the
number of dirty I/O schedulers available and online. There is no support
yet for programmatically changing at run time the number of dirty CPU
schedulers online via erlang:system_flag/2. Also, changing the number of
normal schedulers online via erlang:system_flag(schedulers_online,
NewSchedulersOnline) should ensure that there are no more dirty CPU
schedulers than normal schedulers, but this is not yet implemented. You can
retrieve the number of dirty schedulers by passing dirty_cpu_schedulers,
dirty_cpu_schedulers_online, or dirty_io_schedulers to
erlang:system_info/1.
Currently only NIFs are able to access dirty scheduler
functionality. Neither drivers nor BIFs currently support dirty
schedulers. This restriction will be addressed in the future.
If dirty scheduler support is present in the runtime, the initial status
line Erlang prints before presenting its interactive prompt will include
the indicator "[ds:C1:C2:I]" where "ds" indicates "dirty schedulers", "C1"
indicates the number of dirty CPU schedulers available, "C2" indicates the
number of dirty CPU schedulers online, and "I" indicates the number of
dirty I/O schedulers.
Document The dirty NIF API in the erl_nif man page. The API closely follows
Rickard Green's presentation slides from his talk "Future Extensions to the
Native Interface", presented at the 2011 Erlang Factory held in the San
Francisco Bay Area. Rickard's slides are available online at
http://bit.ly/1m34UHB .
Document the new erl command-line options, the additions to
erlang:system_info/1, and also add the erlang:system_flag/2 dirty scheduler
documentation even though it's not yet implemented.
To determine whether the dirty NIF API is available, native code can check
to see whether the C preprocessor macro ERL_NIF_DIRTY_SCHEDULER_SUPPORT is
defined. To check if dirty schedulers are available at run time, native
code can call the boolean enif_have_dirty_schedulers() function, and Erlang
code can call erlang:system_info(dirty_cpu_schedulers), which raises
badarg if no dirty scheduler support is available.
Add a simple dirty NIF test to the emulator NIF suite.
|
|
* rickard/load_balance/OTP-11385:
Fix usage of non-empty run-queue flag
|
|
|
|
* rickard/load_balance/OTP-11385:
Add support for scheduler utilization balancing
|
|
For more information see documentation of the new command line argument +sub
|
|
as we don't use it and instead have the feature to disable GC
during trapping BIFs.
|
|
* sverk/trapping-bin2term:
erts: Adjust term_to_binary reduction factors
erts: Yield after trapping term_to_binary if gc has been ordered
erts: Let term_to_binary disable gc while trapping
erts: Improve stress of binary_to_term in binary_SUITE
erts: Fix bug in binary_to_term for compressed on halfword
erts: Fix crash when binary_to_term throws badarg
erts: Trapping memcpy in binary_to_term
erts: Cleanup code for trapping binary_to_term
erts: Add erlang wrappers to binary_to_term
trapping uncompress
trapping size calculation
trapping binary_to_term/2
trapping STRING_EXT
trapping lists and tuples
trapping binary_to_term passing binary_SUITE
Parallel check_process_code when code_server purge a module
Functionality for disabling garbage collection
Use asynchronous check_process_code in code_parallel_SUITE
Execution of system tasks in context of another process
Conflicts:
erts/emulator/beam/external.c
erts/emulator/beam/sys.h
erts/emulator/test/binary_SUITE.erl
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/erts_internal.beam
|
|
or if "too much" offheap binaries has been built
|
|
Being able to disable garbage collection over context
switches vastly simplifies implementation of yielding
native code that builds large or complex data structures
on the heap. This since the heap can be left in an
inconsistent state over the context switch.
|
|
A process requesting a system task to be executed in the context of
another process will be notified by a message when the task has
executed. This message will be on the form:
{RequestType, RequestId, Pid, Result}.
A process requesting a system task to be executed can set priority
on the system task. The requester typically set the same priority
on the task as its own process priority, and by this avoiding
priority inversion. A request for execution of a system task is
made by calling the statically linked in NIF
erts_internal:request_system_task(Pid, Prio, Request). This is an
undocumented ERTS internal function that should remain so. It
should *only* be called from BIF implementations.
Currently defined system tasks are:
* garbage_collect
* check_process_code
Further system tasks can and will be implemented in the future.
The erlang:garbage_collect/[1,2] and erlang:check_process_code/[2,3]
BIFs are now implemented using system tasks. Both the
'garbage_collect' and the 'check_process_code' operations perform
or may perform garbage_collections. By doing these via the
system task functionality all garbage collect operations in the
system will be performed solely in the context of the process
being garbage collected. This makes it possible to later implement
functionality for disabling garbage collection of a process over
context switches.
Newly introduced BIFs:
* erlang:garbage_collect/2 - The new second argument is an option
list. Introduced option:
* {async, RequestId} - making it possible for users to issue
asynchronous garbage collect requests.
* erlang:check_process_code/3 - The new third argument is an
option list. Introduced options:
* {async, RequestId} - making it possible for users to issue
asynchronous check process code requests.
* {allow_gc, boolean()} - making it possible to issue requests
that aren't allowed to garbage collect (operation will abort
if gc should be needed).
These options have been introduced as a preparation for
parallelization of check_process_code operations when the
code_server is about to purge a module.
|
|
* egil/fix-crashdump-pointer-crash/OTP-11420:
erts: Fix segfaulting crashdump writing
|
|
Crashdumps initiated by out-of-memory on spawn could cause the beam
to segfault during crashdump writing due to invalid pointers.
The pointers are invalid since the process creation never finished.
This commit remedies this problem by removing the process from
crashdump printout.
|
|
|
|
Clear the "enabled" flag for scheduling statistics when disable is
specified.
|
|
* rickard/+sfwi/OTP-11164:
erts: Add the +sfwi system flag
|
|
+sfwi Interval
Set scheduler forced wakeup interval. All run queues will be scanned
each Interval milliseconds. While there are sleeping schedulers in
the system, one scheduler will be woken for each non-empty run queue
found. An Interval of zero disables this feature, which also is the
default.
This feature has been introduced as a temporary workaround for lengthy
executing native code, and native code that do not bump reductions
properly in OTP. When these bugs have be fixed the +sfwi flag will
be removed.
|
|
* pan/happi/yield_in_term_to_binary:
Add testcase to stress extra_root
term_to_binary: Remove debug code and set production trap levels
Teach erl_gc:offset_rootset about extra_root
Teach external.c to handle reallocs before compression
Make all steps ofterm_to_binary work in chunks and yield
Make term_to_binary yield (trap).
OTP-11163
|
|
* pan/r16b01/system_monitor_long_schedule/OTP-11067:
Minor spelling correction
Add system_monitor of long_schedule
|
|
Rewrite and extend of Happi's initial work
Extra_root to process structure to enable GC of state - Changed the
process structure to point to a separate struct, the struct also
contains a destructor function to allow for proper cleanup.
Rewrote encode_size_struct and enc_term to have internal versions with
reduction counters which will result in interrupt for later restart
when the counter reaches zero - removed the EWA_STACK from Happis
version and directly save the ESTACK's and WSTACK's in the above
mentioned struct (or array thereof) that are pointed out from the
process structure. The destructor will take care of the deallocation
in case of process death.
Added ESTACK and WSTACK macros to save and restore stack and to change
allocator, which makes the previously mentioned stack-save work.
Rewrote enc_term to not store pointers on the stack, and use one WSTACK
for commands etc and another ESTACK for Eterms - Slightly different than
Happis version to make halfword code simpler.
Rewrote encode_size_struct2 so that it does not store pointers on the
stack, also switched to ESTACK instead of WSTACK, this also handles
halfword correctly.
Added interfaces for chunkwise compression, that are
used from term_to_binary/2 when the compressed option is given.
|
|
|
|
|
|
* lukas/erts/gc_stat_contention/OTP-10271:
Replace gc stat lock with sched spec data
|
|
|
|
* rickard/ptab-id-alloc/OTP-11077:
Introduce a better id allocation algorithm for PTabs
|
|
|
|
* rickard/nosuspend/OTP-11076:
Only verify not busy for erlang:send(Port, Msg, [nosuspend]) until scheduled
|
|
|
|
|
|
* maint-r16:
Prepare release
Prepare release
Do not count offline run-queues as empty
Prepare release
Prevent loss of objects after the extension marker
Don't lose the extension mark for object set parameters
Conflicts:
erts/vsn.mk
|
|
|
|
Conflicts:
erts/vsn.mk
|
|
|
|
|
|
This only applied to non-smp systems
|
|
* pan/fix-compiler-warnings-clang-and-new-gcc:
Fix compiler warnings from GCC 4.7.1 on ARCH Linux
Fix clang compiler warnings on FreeBSD in erts
|
|
|
|
|
|
|
|
rickard/r16/port-optimizations/OTP-10336
* rickard/port-optimizations/OTP-10336:
Change annotate level for emacs-22 in cerl
Update etp-commands
Add documentation on communication in Erlang
Add support for busy port message queue
Add driver callback epilogue
Implement true asynchronous signaling between processes and ports
Add erl_drv_[send|output]_term
Move busy port flag
Use rwlock for driver list
Optimize management of port tasks
Improve configuration of process and port tables
Remove R9 compatibility features
Use ptab functionality also for ports
Prepare for use of ptab functionality also for ports
Atomic port state
Generalize process table implementation
Implement functionality for delaying thread progress from unmanaged threads
Conflicts:
erts/doc/src/erl_driver.xml
erts/doc/src/erlang.xml
erts/emulator/beam/beam_bif_load.c
erts/emulator/beam/beam_bp.c
erts/emulator/beam/beam_emu.c
erts/emulator/beam/bif.c
erts/emulator/beam/copy.c
erts/emulator/beam/erl_alloc.c
erts/emulator/beam/erl_alloc.types
erts/emulator/beam/erl_bif_info.c
erts/emulator/beam/erl_bif_port.c
erts/emulator/beam/erl_bif_trace.c
erts/emulator/beam/erl_init.c
erts/emulator/beam/erl_message.c
erts/emulator/beam/erl_port_task.c
erts/emulator/beam/erl_process.c
erts/emulator/beam/erl_process.h
erts/emulator/beam/erl_process_lock.c
erts/emulator/beam/erl_trace.c
erts/emulator/beam/export.h
erts/emulator/beam/global.h
erts/emulator/beam/io.c
erts/emulator/sys/unix/sys.c
erts/emulator/sys/vxworks/sys.c
erts/emulator/test/port_SUITE.erl
erts/etc/unix/cerl.src
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/prim_inet.beam
erts/preloaded/src/prim_inet.erl
lib/hipe/cerl/erl_bif_types.erl
lib/kernel/doc/src/inet.xml
lib/kernel/src/inet.erl
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* maint:
ssl: Adopt test case to not take so long
Fix bug in erts_port_task_schedule()
Use reduction limit in order to determine when to do wakeup
Conflicts:
erts/emulator/beam/erl_port_task.c
erts/emulator/beam/erl_process.c
|
|
|
|
* sverk/code-load-refactor-later-op:
erts: Fix faulty lock check assert
erts: Refactor naming regarding code_write_permission
erts: Refactor tracing to use erts_schedule_thr_prgr_later_op
erts: Allow thr_prgr_later_op to reschedule
erts: Refactor code loading to use erts_schedule_thr_prgr_later_op
erts: Remove some compiler warnings
|
