| Age | Commit message (Collapse) | Author |
|---|
|
This commit replaces the old memory instrumentation with a new
implementation that scans carriers instead of wrapping
erts_alloc/erts_free. The old implementation could not extract
information without halting the emulator, had considerable runtime
overhead, and the memory maps it produced were noisy and lacked
critical information.
Since the new implementation walks through existing data structures
there's no longer a need to start the emulator with special flags to
get information about carrier utilization/fragmentation. Memory
fragmentation is also easier to diagnose as it's presented on a
per-carrier basis which eliminates the need to account for "holes"
between mmap segments.
To help track allocations, each allocation can now be tagged with
what it is and who allocated it at the cost of one extra word per
allocation. This is controlled on a per-allocator basis with the
+M<S>atags option, and is enabled by default for binary_alloc and
driver_alloc (which is also used by NIFs).
|
|
Sharing these settings for all schedulers can degrade performance,
so it makes sense to be able to configure them separately.
This also changes the default busy-wait time to "short" for both
kinds of dirty schedulers.
|
|
* john/erts/async-thread-defaults/OTP-14928:
Change default async thread count to 1
|
|
All uses of async threads in the built-in drivers have been
replaced with dirty IO, so it no longer makes sense to leave the
default at 10.
|
|
Communication between Erlang processes has conceptually always been
performed through asynchronous signaling. The runtime system
implementation has however previously preformed most operation
synchronously. In a system with only one true thread of execution, this
is not problematic (often the opposite). In a system with multiple threads
of execution (as current runtime system implementation with SMP support)
it becomes problematic. This since it often involves locking of structures
when updating them which in turn cause resource contention. Utilizing
true asynchronous communication often avoids these resource contention
issues.
The case that triggered this change was contention on the link lock due
to frequent updates of the monitor trees during communication with a
frequently used server. The signal order delivery guarantees of the
language makes it hard to change the implementation of only some signals
to use true asynchronous signaling. Therefore the implementations
of (almost) all signals have been changed.
Currently the following signals have been implemented as true
asynchronous signals:
- Message signals
- Exit signals
- Monitor signals
- Demonitor signals
- Monitor triggered signals (DOWN, CHANGE, etc)
- Link signals
- Unlink signals
- Group leader signals
All of the above already defined as asynchronous signals in the
language. The implementation of messages signals was quite
asynchronous to begin with, but had quite strict delivery constraints
due to the ordering guarantees of signals between a pair of processes.
The previously used message queue partitioned into two halves has been
replaced by a more general signal queue partitioned into three parts
that service all kinds of signals. More details regarding the signal
queue can be found in comments in the erl_proc_sig_queue.h file.
The monitor and link implementations have also been completely replaced
in order to fit the new asynchronous signaling implementation as good
as possible. More details regarding the new monitor and link
implementations can be found in the erl_monitor_link.h file.
|
|
* john/erts/assert-on-memcpy-memset-etc:
Always use sys_memcpy/cmp/etc instead of plain memcpy/cmp/etc
Check the arguments to sys_memcpy and friends
|
|
|
|
This is a fix for a bug introduced in 22cde2bda
|
|
The reduction cost of sending messages is now constant and will no
longer scale according to the length of the receiving process'
message queue.
|
|
putenv(3) and friends aren't thread-safe regardless of how you slice
it; a global lock around all environment operations (like before)
keeps things safe as far as our own operations go, but we have
absolutely no control over what libc or a library dragged in by a
driver/NIF does -- they're free to call getenv(3) or putenv(3)
without honoring our lock.
This commit solves this by setting up an "emulated" environment which
can't be touched without going through our interfaces. Third-party
libraries can still shoot themselves in the foot but benign uses of
os:putenv/2 will no longer risk crashing the emulator.
|
|
It is not longer relevant when using the poll thread
|
|
|
|
|
|
|
|
This refactor was done using the unifdef tool like this:
for file in $(find erts/ -name *.[ch]); do unifdef -t -f defile -o $file $file; done
where defile contained:
#define ERTS_SMP 1
#define USE_THREADS 1
#define DDLL_SMP 1
#define ERTS_HAVE_SMP_EMU 1
#define SMP 1
#define ERL_BITS_REENTRANT 1
#define ERTS_USE_ASYNC_READY_Q 1
#define FDBLOCK 1
#undef ERTS_POLL_NEED_ASYNC_INTERRUPT_SUPPORT
#define ERTS_POLL_ASYNC_INTERRUPT_SUPPORT 0
#define ERTS_POLL_USE_WAKEUP_PIPE 1
#define ERTS_POLL_USE_UPDATE_REQUESTS_QUEUE 1
#undef ERTS_HAVE_PLAIN_EMU
#undef ERTS_SIGNAL_STATE
|
|
* john/erts/runtime-lcnt:
Document rt_mask and add warnings about copy_save
Add an emulator test suite for lock counting
Break erts_debug:lock_counters/1 into separate BIFs
Allow toggling lock counting at runtime
Move lock flags to a common header
Enable register_SUITE for lcnt builds
Enable lcnt smoke test on all builds that have lcnt enabled
Make lock counter info independent of the locks being counted
OTP-14412
OTP-13170
OTP-14413
|
|
The implementation is still hidden behind ERTS_ENABLE_LOCK_COUNT, and
all categories are still enabled by default, but the actual counting can be
toggled at will.
OTP-13170
|
|
|
|
erts: Remove old unused functions
|
|
The functions have been found using: https://github.com/caolanm/callcatcher
|
|
|
|
OTP-14380
* rickard/ds-stack-size:
Suggested stack size options for dirty schedulers
|
|
|
|
|
|
|
|
OTP-14331
* rickard/pcre-8.40:
Update documentation
Update README.pcre_update.md
Stack guard for PCRE
Adjust for incompatibility between PCRE 8.40 and perl 5.22.1
Generate re replacement and split tests with perl vsn 5.22.1
Fix re_SUITE:pcre_compile_workspace_overflow/1
Skip line with lockout of modifiers in PCRE tests
Update tests for PCRE version 8.40
Update PCRE to version 8.40
Conflicts:
erts/emulator/beam/beam_debug.c
|
|
|
|
|
|
|
|
Magic references are *intentionally* indistinguishable from ordinary
references for the Erlang software. Magic references do not change
the language, and are intended as a pure runtime internal optimization.
An ordinary reference is typically used as a key in some table. A
magic reference has a direct pointer to a reference counted magic
binary. This makes it possible to implement various things without
having to do lookups in a table, but instead access the data directly.
Besides very fast lookups this can also improve scalability by
removing a potentially contended table. A couple of examples of
planned future usage of magic references are ETS table identifiers,
and BIF timer identifiers.
Besides future optimizations using magic references it should also
be possible to replace the exposed magic binary cludge with magic
references. That is, magic binaries that are exposed as empty
binaries to the Erlang software.
|
|
* maint:
Use a hole-marker that cannot be mistaken for a valid term on the heap
|
|
* rickard/hole-marker:
Use a hole-marker that cannot be mistaken for a valid term on the heap
|
|
|
|
|
|
This fixes commit f0867aa2ccbbf5677e0577bba08f8b7bc53ec0ed
|
|
Conflicts:
OTP_VERSION
|
|
Conflicts:
OTP_VERSION
erts/doc/src/notes.xml
erts/emulator/sys/unix/erl_unix_sys.h
erts/emulator/sys/unix/sys.c
erts/vsn.mk
lib/crypto/c_src/crypto.c
lib/crypto/doc/src/notes.xml
lib/crypto/vsn.mk
lib/inets/doc/src/notes.xml
lib/inets/vsn.mk
lib/ssh/doc/src/notes.xml
lib/ssh/src/ssh.app.src
lib/ssh/src/ssh_connection_handler.erl
lib/ssh/vsn.mk
otp_versions.table
|
|
maint-18
* egil/erts/fix_scheduler_suspend/ERL-94/PR-978/OTP-13425:
erts: Fix install of suspend handler
|
|
|
|
* move signal handler setup
|
|
* maint:
Fix initialization dependencies between time, lock-check and lcnt
|
|
* rickard/init-bug/OTP-13911:
Fix initialization dependencies between time, lock-check and lcnt
Conflicts:
erts/emulator/sys/unix/sys.c
|
|
|
|
to limit crash dump size
|
|
|
|
* bjorn/erts/help-clang:
Slightly reduce code size of process_main()
|
|
clang will generate smaller code if we avoid reassigning the
reg and freg arrays. That can be easily arranged by passing them
as arguments to process_main().
|
|
This commit makes sure to setup the suspend handler
to matter what +B option is given at the command line.
|
|
|
|
* rickard/ds-purge-module/OTP-13808:
Perform check_process_code while process is executing dirty
Conflicts:
erts/doc/src/erl_nif.xml
|
