| Age | Commit message (Collapse) | Author |
|---|
|
Add initial support for dirty schedulers.
There are two types of dirty schedulers: CPU schedulers and I/O
schedulers. By default, there are as many dirty CPU schedulers as there are
normal schedulers and as many dirty CPU schedulers online as normal
schedulers online. There are 10 dirty I/O schedulers (similar to the choice
of 10 as the default for async threads).
By default, dirty schedulers are disabled and conditionally compiled
out. To enable them, you must pass --enable-dirty-schedulers to the
top-level configure script when building Erlang/OTP.
Current dirty scheduler support requires the emulator to be built with SMP
support. This restriction will be lifted in the future.
You can specify the number of dirty schedulers with the command-line
options +SDcpu (for dirty CPU schedulers) and +SDio (for dirty I/O
schedulers). The +SDcpu option is similar to the +S option in that it takes
two numbers separated by a colon: C1:C2, where C1 specifies the number of
dirty schedulers available and C2 specifies the number of dirty schedulers
online. The +SDPcpu option allows numbers of dirty CPU schedulers available
and dirty CPU schedulers online to be specified as percentages, similar to
the existing +SP option for normal schedulers. The number of dirty CPU
schedulers created and dirty CPU schedulers online may not exceed the
number of normal schedulers created and normal schedulers online,
respectively. The +SDio option takes only a single number specifying the
number of dirty I/O schedulers available and online. There is no support
yet for programmatically changing at run time the number of dirty CPU
schedulers online via erlang:system_flag/2. Also, changing the number of
normal schedulers online via erlang:system_flag(schedulers_online,
NewSchedulersOnline) should ensure that there are no more dirty CPU
schedulers than normal schedulers, but this is not yet implemented. You can
retrieve the number of dirty schedulers by passing dirty_cpu_schedulers,
dirty_cpu_schedulers_online, or dirty_io_schedulers to
erlang:system_info/1.
Currently only NIFs are able to access dirty scheduler
functionality. Neither drivers nor BIFs currently support dirty
schedulers. This restriction will be addressed in the future.
If dirty scheduler support is present in the runtime, the initial status
line Erlang prints before presenting its interactive prompt will include
the indicator "[ds:C1:C2:I]" where "ds" indicates "dirty schedulers", "C1"
indicates the number of dirty CPU schedulers available, "C2" indicates the
number of dirty CPU schedulers online, and "I" indicates the number of
dirty I/O schedulers.
Document The dirty NIF API in the erl_nif man page. The API closely follows
Rickard Green's presentation slides from his talk "Future Extensions to the
Native Interface", presented at the 2011 Erlang Factory held in the San
Francisco Bay Area. Rickard's slides are available online at
http://bit.ly/1m34UHB .
Document the new erl command-line options, the additions to
erlang:system_info/1, and also add the erlang:system_flag/2 dirty scheduler
documentation even though it's not yet implemented.
To determine whether the dirty NIF API is available, native code can check
to see whether the C preprocessor macro ERL_NIF_DIRTY_SCHEDULER_SUPPORT is
defined. To check if dirty schedulers are available at run time, native
code can call the boolean enif_have_dirty_schedulers() function, and Erlang
code can call erlang:system_info(dirty_cpu_schedulers), which raises
badarg if no dirty scheduler support is available.
Add a simple dirty NIF test to the emulator NIF suite.
|
|
* rickard/load_balance/OTP-11385:
Add support for scheduler utilization balancing
|
|
For more information see documentation of the new command line argument +sub
|
|
* egil/etp-commands/OTP-11582:
erts: Update etp-commands with heap-dump
|
|
* rickard/otp-17-vsn-fix:
Fix issues with new versioning
|
|
|
|
|
|
|
|
Update versions of OTP, erts, kernel, and stdlib to comply with
the new version scheme decided by the OTP technical board.
|
|
* rickard/garbage_collect/OTP-11388:
Parallel check_process_code when code_server purge a module
Functionality for disabling garbage collection
Use asynchronous check_process_code in code_parallel_SUITE
Execution of system tasks in context of another process
Conflicts:
bootstrap/lib/kernel/ebin/hipe_unified_loader.beam
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/erts_internal.beam
|
|
Conflicts:
erts/etc/win32/Install.c
|
|
* dgud/sasl/no_dot_erlang_boot/OTP-8479:
sasl: Add no_dot_erlang documentation and tests
sasl: Add no_dot_erlang start script
|
|
* maint:
Fix observer retrieval of alloc info
Fix documentation of the +MMsco switch
Replace the +MMscmgc switch with +MMscrfsd
Add switch for disabling sys_alloc carriers
Add support for locking mappings to physical memory
|
|
* rickard/supercarrier-fix/OTP-11149:
Fix observer retrieval of alloc info
Fix documentation of the +MMsco switch
Replace the +MMscmgc switch with +MMscrfsd
Add switch for disabling sys_alloc carriers
Add support for locking mappings to physical memory
|
|
Being able to disable garbage collection over context
switches vastly simplifies implementation of yielding
native code that builds large or complex data structures
on the heap. This since the heap can be left in an
inconsistent state over the context switch.
|
|
A process requesting a system task to be executed in the context of
another process will be notified by a message when the task has
executed. This message will be on the form:
{RequestType, RequestId, Pid, Result}.
A process requesting a system task to be executed can set priority
on the system task. The requester typically set the same priority
on the task as its own process priority, and by this avoiding
priority inversion. A request for execution of a system task is
made by calling the statically linked in NIF
erts_internal:request_system_task(Pid, Prio, Request). This is an
undocumented ERTS internal function that should remain so. It
should *only* be called from BIF implementations.
Currently defined system tasks are:
* garbage_collect
* check_process_code
Further system tasks can and will be implemented in the future.
The erlang:garbage_collect/[1,2] and erlang:check_process_code/[2,3]
BIFs are now implemented using system tasks. Both the
'garbage_collect' and the 'check_process_code' operations perform
or may perform garbage_collections. By doing these via the
system task functionality all garbage collect operations in the
system will be performed solely in the context of the process
being garbage collected. This makes it possible to later implement
functionality for disabling garbage collection of a process over
context switches.
Newly introduced BIFs:
* erlang:garbage_collect/2 - The new second argument is an option
list. Introduced option:
* {async, RequestId} - making it possible for users to issue
asynchronous garbage collect requests.
* erlang:check_process_code/3 - The new third argument is an
option list. Introduced options:
* {async, RequestId} - making it possible for users to issue
asynchronous check process code requests.
* {allow_gc, boolean()} - making it possible to issue requests
that aren't allowed to garbage collect (operation will abort
if gc should be needed).
These options have been introduced as a preparation for
parallelization of check_process_code operations when the
code_server is about to purge a module.
|
|
|
|
* sverk/cerl-gdb-fix:
erts: Fix cerl -gdb
|
|
by replacing all newlines in $beam_args with space
|
|
* maint:
erts: Add cerl -dump and dumping in z_SUITE
|
|
|
|
Replaced the +MMscmgc switch with the +MMscrfsd switch. The old switch
didn't reflect what it controlled.
|
|
The switch "+Musac <boolean>" controls if sys_alloc carriers
are allowed.
|
|
Using "+Mlpm all" switch all mappings made by the emulator will
be locked into physical memory.
|
|
Conflicts:
erts/preloaded/ebin/erlang.beam
|
|
* rickard-sverker/supercarrier/OTP-11149: (29 commits)
erts: Add test case for erts_mmap
erts: Add mutex to init_atoms in erts_mmap.c
erts: Fix lock violation for init_atoms in erl_mmap.c
erts: Fix misc minor bugs in supercarrier initialization
erts: Add erts_mmap stats
erts: Add erts_bld_tupleX macros
erts: Rename erts_bld_atom_uint_2tup_list to *_uword_*
erts: Fix bug in lookup_free_seg
erts: Fix race bug in erts_munmap
erts: Add HARD_DBG_MSEG
erts: Refactor rbt_insert in erl_mmap
erts: erts_mmap improved free seg desc management
erts: Add documentation for +MMsc* system flags
erts: Allow page aligned erts_munmap()
erts: Sort tree in super aligned sizes (SA_SZ_ADDR_ORDER)
erts: Fix ASSERT bug and void* arithmetics
erts: Add mmap argument to erts_debug:get_internal_state
erts: Improve erts_mmap out of free descriptor management
erts: Cleanup erl_mmap
erts: Add __func__ to ERTS_ASSERT macro
...
|
|
|
|
* nox/silent-rules-fixes/OTP-11351:
Fix two small silent rules omissions
|
|
Sometimes it is wanted to start erlang without loading the user dependent
.erlang file, for example in scripts and configure tests.
|
|
* Coalescing and trimming of free segments in supercarrier
* Management of super aligned and super unaligned areas in
supercarrier
* Management of reservation of physical memory
* erts_mseg usage of erts_mmap
|
|
|
|
|
|
|
|
The 'erlc' program passes options to the 'erl' program using
the '-s' option. The '-s' option causes all options to be converted
to atoms, which implies that UTF-8 file names may not be given on
the command line.
We could solve just the UTF-8 problem by using '-run' and change
the erl_compile module to expect strings instead of atoms, but since
that is an incompatible change, we should take the opportunity to
make more incompatible changes while we are at it.
Specifically, when 'erlc' was first written, there was no way to pass
command line arguments starting with '-' to Erlang, so 'erlc' did all
parsing of arguments and translated options to atoms starting with a
'@' character (for example, -I was translated to @i). Since then,
the '-extra' option has been introduced which allows us to pass
anything to Erlang at the end of the command line.
Therefore, while at it, do the minimum of necessary command line
parsing in the 'erlc' program (e.g. the '-smp' option), passing the
command line essentially unchanged to 'erl' using the '-extra' option,
and rewrite the option parsing in Erlang.
|
|
|
|
config.h defines HAVE_SYSLOG_H whereas the sources are looking for
NO_SYSLOG to be undefined. As the logic of "if feature is available"
makes more sense than "if feature is not unavailable", I opted for the
config.h define.
|
|
|
|
* lukas/erts/erl_exec_e:
+e should be passed through erlexec
|
|
|
|
* sverk/valgrind-single-core:
erts: Speed up valgrind with asynch threads
|
|
|
|
* maint:
add erl option to set schedulers by percentages
|
|
For applications where measurements show enhanced performance from the use
of a non-default number of emulator scheduler threads, having to accurately
set the right number of scheduler threads across multiple hosts each with
different numbers of logical processors is difficult because the erl +S
option requires absolute numbers of scheduler threads and scheduler threads
online to be specified.
To address this issue, add a +SP option to erl, similar to the existing +S
option but allowing the number of scheduler threads and scheduler threads
online to be set as percentages of logical processors configured and
logical processors available, respectively. For example, "+SP 50:25" sets
the number of scheduler threads to 50% of the logical processors
configured, and the number of scheduler threads online to 25% of the
logical processors available. The +SP option also interacts with any
settings specified with the +S option, such that the combination of options
"+S 4:4 +SP 50:50" (in either order) results in 2 scheduler threads and 2
scheduler threads online.
Add documentation for the +SP option.
Add tests for the +SP option to scheduler_SUITE.
Add tests and documentation for two existing features of the +S option: +S
0:0 resets the scheduler thread count and scheduler threads online count to
their defaults, and specifying negative numbers for +S results in those
values being subtracted from the default values for the host.
|
|
by only letting it run on one core. Valgrind only let one thread
at a time execute anyway.
|
|
* maint:
etp: Do not use name as beam.smp is the name on Linux
|
|
|
|
* maint:
erts: Create gdb pything script for thread listing
|
|
|
|
|
|
* weisslj/run-erl-redirect-dev-null/OTP-11215:
run_erl: Redirect standard streams to /dev/null
|
