| Age | Commit message (Collapse) | Author |
|---|
|
jhogberg/john/erts/cross-type-carrier-migration/OTP-15063
Allow carrier migration between different allocator types
|
|
|
|
into john/erts/merge-OTP-15067
|
|
The fail state wasn't re-checked in the state change loop; only
the FREE state was checked. In addition to that, we would leave
the task in the queue when bailing out which could lead to a
double-free.
This commit backports active_sys_enqueue from master to make it
easier to merge onwards.
|
|
This commit replaces the old memory instrumentation with a new
implementation that scans carriers instead of wrapping
erts_alloc/erts_free. The old implementation could not extract
information without halting the emulator, had considerable runtime
overhead, and the memory maps it produced were noisy and lacked
critical information.
Since the new implementation walks through existing data structures
there's no longer a need to start the emulator with special flags to
get information about carrier utilization/fragmentation. Memory
fragmentation is also easier to diagnose as it's presented on a
per-carrier basis which eliminates the need to account for "holes"
between mmap segments.
To help track allocations, each allocation can now be tagged with
what it is and who allocated it at the cost of one extra word per
allocation. This is controlled on a per-allocator basis with the
+M<S>atags option, and is enabled by default for binary_alloc and
driver_alloc (which is also used by NIFs).
|
|
|
|
Communication between Erlang processes has conceptually always been
performed through asynchronous signaling. The runtime system
implementation has however previously preformed most operation
synchronously. In a system with only one true thread of execution, this
is not problematic (often the opposite). In a system with multiple threads
of execution (as current runtime system implementation with SMP support)
it becomes problematic. This since it often involves locking of structures
when updating them which in turn cause resource contention. Utilizing
true asynchronous communication often avoids these resource contention
issues.
The case that triggered this change was contention on the link lock due
to frequent updates of the monitor trees during communication with a
frequently used server. The signal order delivery guarantees of the
language makes it hard to change the implementation of only some signals
to use true asynchronous signaling. Therefore the implementations
of (almost) all signals have been changed.
Currently the following signals have been implemented as true
asynchronous signals:
- Message signals
- Exit signals
- Monitor signals
- Demonitor signals
- Monitor triggered signals (DOWN, CHANGE, etc)
- Link signals
- Unlink signals
- Group leader signals
All of the above already defined as asynchronous signals in the
language. The implementation of messages signals was quite
asynchronous to begin with, but had quite strict delivery constraints
due to the ordering guarantees of signals between a pair of processes.
The previously used message queue partitioned into two halves has been
replaced by a more general signal queue partitioned into three parts
that service all kinds of signals. More details regarding the signal
queue can be found in comments in the erl_proc_sig_queue.h file.
The monitor and link implementations have also been completely replaced
in order to fit the new asynchronous signaling implementation as good
as possible. More details regarding the new monitor and link
implementations can be found in the erl_monitor_link.h file.
|
|
* sverker/systask-reqid-bug:
erts: Fix bug in systask scheduling
|
|
* sverker/systask-reqid-bug:
erts: Fix bug in systask scheduling
|
|
when request id is an immediate.
Ex:
erlang:garbage_collect(P, [{async,Immediate}]).
may crash the VM.
|
|
|
|
This reverts commit d8c8e0c66d6faf5402682f3a8568362eedebdfee.
|
|
|
|
|
|
process_SUITE starts os_mon in init_per_suite/1. Therefore,
there may be occasional alarm info messages received. Make
sure that they are ignored and don't cause the test fail to
fail.
Also, get rid of the flush/0 function that discards all
messages in the message queue. Instead, be more selective
and only discard {error, ...} messages.
|
|
If process_SUITE:spawn_opt_max_heap_size/1 failed, the default
value for the maximum heap size could have been changed. That
would cause other test cases that spawned huge processes to
fail.
Contain the damage by always restoring the default value
for max_heap_size in end_per_testcase/2.
|
|
It is expected that low-priority processes should be allowed
to run about 1/8 of the time of normal-priority processes.
The proportion calculated was usually considerably lower than
1/8. It seems that the main reason is the punishement in
reductions when sending to a process that alreday has many
messages in the message queue. I have verified that by running
the emulator with the +snsp to turn off the send punishment.
To avoid that punishment, let the server process run at high
priority so that it can keep its message queue short.
It seems that the proportion is now very close to 1/8. Therefore,
we can tighten the margins for the proportion.
|
|
|
|
* maint:
Fix processes() BIF test cases
|
|
* rickard/test-fix:
Fix processes() BIF test cases
|
|
|
|
|
|
The BIF process_info(Pid, current_stacktrace) truncates the
stacktrace. The old behavior was to truncate long stacktraces to max
8 items. And this was hard coded. Now it is truncated to the value of
system_flag(backtrace_depth) instead. The backtrace_depth defaults to
8, but is configurable.
|
|
* kvakvs/erts/gc_minor_option/OTP-11695:
erts: Fix req_system_task gc typespec
Fix process_SUITE system_task_blast and no_priority_inversion2
Option to erlang:garbage_collect to request minor (generational) GC
Conflicts:
erts/emulator/beam/erl_process.c
erts/preloaded/src/erts_internal.erl
|
|
|
|
* rickard/time-unit/OTP-13735:
Update test-cases to use new symbolic time units
Replace misspelled symbolic time units
Conflicts:
erts/doc/src/erlang.xml
erts/emulator/test/long_timers_test.erl
|
|
|
|
|
|
* lukas/erts/testfixes-19:
erts: Increase bif and nif call_time trace test
erts: Fix distribution_SUITE:bulk_send_bigbig on windows
erts: Ensure bs_add_overflow test has enough memory
kernel: Better explain controlling_process' tcp behaviour
kernel: Fix t_recv_delim on bsd
os_mon: Make sure to start/stop os_mon in tests correctly
ssl: Fix use_interface dist_SSL test
erl_interface: Fix signed int overflow tc bug
erts: fix atom_roundtrip_r15b tc
erts: Require more memory for debug tests
|
|
|
|
|
|
The max_heap_size process flag can be used to limit the
growth of a process heap by killing it before it becomes
too large to handle. It is possible to set the maximum
using the `erl +hmax` option, `system_flag(max_heap_size, ...)`,
`spawn_opt(Fun, [{max_heap_size, ...}])` and
`process_flag(max_heap_size, ...)`.
It is possible to configure the behaviour of the process
when the maximum heap size is reached. The process may be
sent an untrappable exit signal with reason kill and/or
send an error_logger message with details on the process
state. A new trace event called gc_max_heap_size is
also triggered for the garbage_collection trace flag
when the heap grows larger than the configured size.
If kill and error_logger are disabled, it is still
possible to see that the maximum has been reached by
doing garbage collection tracing on the process.
The heap size is defined as the sum of the heap memory
that the process is currently using. This includes
all generational heaps, the stack, any messages that
are considered to be part of the heap and any extra
memory the garbage collector may need during collection.
In the current implementation this means that when a process
is set using on_heap message queue data mode, the messages
that are in the internal message queue are counted towards
this value. For off_heap, only matched messages count towards
the size of the heap. For mixed, it depends on race conditions
within the VM whether a message is part of the heap or not.
Below is an example run of the new behaviour:
Eshell V8.0 (abort with ^G)
1> f(P),P = spawn_opt(fun() -> receive ok -> ok end end, [{max_heap_size, 512}]).
<0.60.0>
2> erlang:trace(P, true, [garbage_collection, procs]).
1
3> [P ! lists:duplicate(M,M) || M <- lists:seq(1,15)],ok.
ok
4>
=ERROR REPORT==== 26-Apr-2016::16:25:10 ===
Process: <0.60.0>
Context: maximum heap size reached
Max heap size: 512
Total heap size: 723
Kill: true
Error Logger: true
GC Info: [{old_heap_block_size,0},
{heap_block_size,609},
{mbuf_size,145},
{recent_size,0},
{stack_size,9},
{old_heap_size,0},
{heap_size,211},
{bin_vheap_size,0},
{bin_vheap_block_size,46422},
{bin_old_vheap_size,0},
{bin_old_vheap_block_size,46422}]
flush().
Shell got {trace,<0.60.0>,gc_start,
[{old_heap_block_size,0},
{heap_block_size,233},
{mbuf_size,145},
{recent_size,0},
{stack_size,9},
{old_heap_size,0},
{heap_size,211},
{bin_vheap_size,0},
{bin_vheap_block_size,46422},
{bin_old_vheap_size,0},
{bin_old_vheap_block_size,46422}]}
Shell got {trace,<0.60.0>,gc_max_heap_size,
[{old_heap_block_size,0},
{heap_block_size,609},
{mbuf_size,145},
{recent_size,0},
{stack_size,9},
{old_heap_size,0},
{heap_size,211},
{bin_vheap_size,0},
{bin_vheap_block_size,46422},
{bin_old_vheap_size,0},
{bin_old_vheap_block_size,46422}]}
Shell got {trace,<0.60.0>,exit,killed}
|
|
* henrik/update-copyrightyear:
update copyright-year
|
|
|
|
|
|
Those clause are obsolete and never used by common_test.
|
|
The macro ?t is deprecated. Replace its use with 'test_server'.
|
|
|
|
|
|
|
|
|
|
As a first step to removing the test_server application as
as its own separate application, change the inclusion of
test_server.hrl to an inclusion of ct.hrl and remove the
inclusion of test_server_line.hrl.
|
|
This info request returns greater details about the current
gc state. This info is not included in the default process_info/1
as it would clutter the default printout with too much information.
|
|
|
|
|
|
|
|
Being able to disable garbage collection over context
switches vastly simplifies implementation of yielding
native code that builds large or complex data structures
on the heap. This since the heap can be left in an
inconsistent state over the context switch.
|
|
A process requesting a system task to be executed in the context of
another process will be notified by a message when the task has
executed. This message will be on the form:
{RequestType, RequestId, Pid, Result}.
A process requesting a system task to be executed can set priority
on the system task. The requester typically set the same priority
on the task as its own process priority, and by this avoiding
priority inversion. A request for execution of a system task is
made by calling the statically linked in NIF
erts_internal:request_system_task(Pid, Prio, Request). This is an
undocumented ERTS internal function that should remain so. It
should *only* be called from BIF implementations.
Currently defined system tasks are:
* garbage_collect
* check_process_code
Further system tasks can and will be implemented in the future.
The erlang:garbage_collect/[1,2] and erlang:check_process_code/[2,3]
BIFs are now implemented using system tasks. Both the
'garbage_collect' and the 'check_process_code' operations perform
or may perform garbage_collections. By doing these via the
system task functionality all garbage collect operations in the
system will be performed solely in the context of the process
being garbage collected. This makes it possible to later implement
functionality for disabling garbage collection of a process over
context switches.
Newly introduced BIFs:
* erlang:garbage_collect/2 - The new second argument is an option
list. Introduced option:
* {async, RequestId} - making it possible for users to issue
asynchronous garbage collect requests.
* erlang:check_process_code/3 - The new third argument is an
option list. Introduced options:
* {async, RequestId} - making it possible for users to issue
asynchronous check process code requests.
* {allow_gc, boolean()} - making it possible to issue requests
that aren't allowed to garbage collect (operation will abort
if gc should be needed).
These options have been introduced as a preparation for
parallelization of check_process_code operations when the
code_server is about to purge a module.
|
|
|
|
|
