aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/erl_gc.c
AgeCommit message (Collapse)Author
2019-03-25erts: Include external msg in need of GCLukas Larsson
When many external messages suddenly appear in the mailbox the young gen size is adjusted accordingly but it was immediately shrunk as the data was not counted towards the shrink size. This commit includes the ext dist size in the shrink calculation which means that the decode of the external messages will not trigger as many GCs which means much better performance.
2019-02-22erts: Add ERL_NODE_BOOKKEEP to node tables refcLukas Larsson
2018-12-20Merge branch 'lukas/erts/fix-seq_trace-reset_trace/OTP-15490' into maintLukas Larsson
* lukas/erts/fix-seq_trace-reset_trace/OTP-15490: erts: Fix seq_trace:reset_trace dirty gc bug erts: Use sys_memcpy in copy_one_frag
2018-12-13erts: Use sys_memcpy in copy_one_fragLukas Larsson
2018-12-10erts: Use ptr_val for pointer in gc msg copyLukas Larsson
Without this fix asserts would trigger in debug build but nothing else would break.
2018-12-04erts: Fix copy of literal msg during gcLukas Larsson
A copy has to be made of the message as there is a trace token. There was a bug where the actual message was incorrectly modified even if it was a literal.
2018-11-02erts: Fix msacc GC states on dirty schedsLukas Larsson
A dirty scheduler is an un-managed thread so we need to lock the msacc state on those.
2018-10-19Honor the max heap size when copying literals after purgingBjörn Gustavsson
When a module has been purged from memory, any literals belonging to that module will be copied to all processes that hold references to them. The max heap size limit would be ignored in the garbage collection initiated when copying literals to a process. If the max heap size was exceeded, the process would typically be terminated in the following garbage collection. Since the process would be killed anyway later, kill the process before copying a literal that would make it exceed its max heap size. While at it, also fix a potential bug in `erlang:garbage_collect/0`. If it was found that the max heap sized had been exceeded while executing `erlang:garbage_collect/0`, the process would enter a kind of zombie state instead of being properly terminated.
2018-05-16Teach HiPE to yield from receiveRickard Green
2018-04-26Merge branch 'rickard/process_info/OTP-14966'Rickard Green
* rickard/process_info/OTP-14966: Restore merge of signal queues in queue_messages() if main lock is held Fix message tracing
2018-04-25Restore merge of signal queues in queue_messages() if main lock is heldRickard Green
2018-04-16Fix deadlock in HiPE gc after receiveRickard Green
2018-04-12New process_info() implementation using signalsRickard Green
2018-03-21Implementation of true asynchronous signaling between processesRickard Green
Communication between Erlang processes has conceptually always been performed through asynchronous signaling. The runtime system implementation has however previously preformed most operation synchronously. In a system with only one true thread of execution, this is not problematic (often the opposite). In a system with multiple threads of execution (as current runtime system implementation with SMP support) it becomes problematic. This since it often involves locking of structures when updating them which in turn cause resource contention. Utilizing true asynchronous communication often avoids these resource contention issues. The case that triggered this change was contention on the link lock due to frequent updates of the monitor trees during communication with a frequently used server. The signal order delivery guarantees of the language makes it hard to change the implementation of only some signals to use true asynchronous signaling. Therefore the implementations of (almost) all signals have been changed. Currently the following signals have been implemented as true asynchronous signals: - Message signals - Exit signals - Monitor signals - Demonitor signals - Monitor triggered signals (DOWN, CHANGE, etc) - Link signals - Unlink signals - Group leader signals All of the above already defined as asynchronous signals in the language. The implementation of messages signals was quite asynchronous to begin with, but had quite strict delivery constraints due to the ordering guarantees of signals between a pair of processes. The previously used message queue partitioned into two halves has been replaced by a more general signal queue partitioned into three parts that service all kinds of signals. More details regarding the signal queue can be found in comments in the erl_proc_sig_queue.h file. The monitor and link implementations have also been completely replaced in order to fit the new asynchronous signaling implementation as good as possible. More details regarding the new monitor and link implementations can be found in the erl_monitor_link.h file.
2018-02-22Merge 'sverker/erts/more-crash-dump-info/OTP-14820'Sverker Eriksson
2018-02-20erts: Include failing garbing process in crash dumpSverker Eriksson
Exclude garbing processes, EXCEPT if run by crash dumping thread in which case we assume the heap is healthy without any move markers yet/left. Switched order between (allocating) setup_rootset() and (move marking) collect_live_heap_frags().
2018-02-20erts: Remove unused args to collect_live_heap_fragsSverker Eriksson
2018-01-19erts: Polish GC rootset interationSverker Eriksson
2018-01-19erts: Optimize move_cons & move_boxedSverker Eriksson
Replace double pointer with return that can mostly be ignored. Use restrict pointers.
2017-11-27erts: Fix bad merge of PR #1644Björn Gustavsson
2017-11-27Merge branch 'maint'John Högberg
2017-11-24Fix purging of modules with "fake literals"Björn Gustavsson
When compiling Erlang source code, the literal area for the module can only contain data types that have a literal syntax. However, it is possible to sneak in other data types (such as references) in the literal pool by compiling from abstract or assembly code. Those "fake literals" would work fine, but would crash the runtime system when the module containing the literals was purged. Although fake literals are not officially supported, the runtime should not crash when attempting to use them. Therefore, fix the garbage collection of literals and releasing of literal areas. https://bugs.erlang.org/browse/ERL-508
2017-09-28Eliminate MY_IS_SSMALL()Björn Gustavsson
For a long time, there has been the two macros IS_SSMALL() and MY_IS_SSMALL() that do exactly the same thing. There should only be one, and it should be called IS_SSMALL(). However, we must decide which implementation to use. When MY_IS_SSMALL() was introduced a long time ago, it was the most efficient. In a modern C compiler, there might not be any difference. To find out, I used the following small C program to examine the code generation: #include <stdio.h> typedef unsigned int Uint32; typedef unsigned long Uint64; typedef long Sint; #define SWORD_CONSTANT(Const) Const##L #define SMALL_BITS (64-4) #define MAX_SMALL ((SWORD_CONSTANT(1) << (SMALL_BITS-1))-1) #define MIN_SMALL (-(SWORD_CONSTANT(1) << (SMALL_BITS-1))) #define MY_IS_SSMALL32(x) (((Uint32) ((((x)) >> (SMALL_BITS-1)) + 1)) < 2) #define MY_IS_SSMALL64(x) (((Uint64) ((((x)) >> (SMALL_BITS-1)) + 1)) < 2) #define MY_IS_SSMALL(x) (sizeof(x) == sizeof(Uint32) ? MY_IS_SSMALL32(x) : MY_IS_SSMALL64(x)) #define IS_SSMALL(x) (((x) >= MIN_SMALL) && ((x) <= MAX_SMALL)) void original(Sint n) { if (IS_SSMALL(n)) { printf("yes\n"); } } void enhanced(Sint n) { if (MY_IS_SSMALL(n)) { printf("yes\n"); } } gcc 7.2 produced the following code for the original() function: .LC0: .string "yes" original(long): movabs rax, 576460752303423488 add rdi, rax movabs rax, 1152921504606846975 cmp rdi, rax jbe .L4 rep ret .L4: mov edi, OFFSET FLAT:.LC0 jmp puts clang 5.0.0 produced the following code which is slightly better: original(long): movabs rax, 576460752303423488 add rax, rdi shr rax, 60 jne .LBB0_1 mov edi, .Lstr jmp puts # TAILCALL .LBB0_1: ret .Lstr: .asciz "yes" However, in the context of beam_emu.c, clang could produce similar to what gcc produced. gcc 7.2 produced the following code when MY_IS_SSMALL() was used: .LC0: .string "yes" enhanced(long): sar rdi, 59 add rdi, 1 cmp rdi, 1 jbe .L4 rep ret .L4: mov edi, OFFSET FLAT:.LC0 jmp puts clang produced similar code. This code seems to be the cheapest. There are four instructions, and there is no loading of huge integer constants.
2017-09-11erts: Remove possibility to disable dirty schedulersLukas Larsson
2017-07-17erts: Replace usage of all erts_smp prefixes to just ertsLukas Larsson
2017-07-17erts: Cleanup removal of non-smp emulatorsLukas Larsson
2017-07-17erts: Remove ERTS_SMP and USE_THREAD definesLukas Larsson
This refactor was done using the unifdef tool like this: for file in $(find erts/ -name *.[ch]); do unifdef -t -f defile -o $file $file; done where defile contained: #define ERTS_SMP 1 #define USE_THREADS 1 #define DDLL_SMP 1 #define ERTS_HAVE_SMP_EMU 1 #define SMP 1 #define ERL_BITS_REENTRANT 1 #define ERTS_USE_ASYNC_READY_Q 1 #define FDBLOCK 1 #undef ERTS_POLL_NEED_ASYNC_INTERRUPT_SUPPORT #define ERTS_POLL_ASYNC_INTERRUPT_SUPPORT 0 #define ERTS_POLL_USE_WAKEUP_PIPE 1 #define ERTS_POLL_USE_UPDATE_REQUESTS_QUEUE 1 #undef ERTS_HAVE_PLAIN_EMU #undef ERTS_SIGNAL_STATE
2017-07-06Merge branch 'john/erts/runtime-lcnt' into maintJohn Högberg
* john/erts/runtime-lcnt: Document rt_mask and add warnings about copy_save Add an emulator test suite for lock counting Break erts_debug:lock_counters/1 into separate BIFs Allow toggling lock counting at runtime Move lock flags to a common header Enable register_SUITE for lcnt builds Enable lcnt smoke test on all builds that have lcnt enabled Make lock counter info independent of the locks being counted OTP-14412 OTP-13170 OTP-14413
2017-07-06Allow toggling lock counting at runtimeJohn Högberg
The implementation is still hidden behind ERTS_ENABLE_LOCK_COUNT, and all categories are still enabled by default, but the actual counting can be toggled at will. OTP-13170
2017-06-01Merge branch 'rickard/purge-hibernated-19' into rickard/purge-hibernated-20Rickard Green
* rickard/purge-hibernated-19: Do not GC hibernated process from other processes Fix check_process_code() on hibernated process Conflicts: erts/emulator/beam/beam_bif_load.c erts/emulator/beam/erl_process.c erts/emulator/beam/erl_process.h
2017-06-01Merge branch 'rickard/purge-hibernated' into rickard/purge-hibernated-19Rickard Green
* rickard/purge-hibernated: Do not GC hibernated process from other processes Fix check_process_code() on hibernated process Conflicts: erts/emulator/beam/beam_bif_load.c erts/emulator/beam/erl_gc.c erts/emulator/beam/erl_process.h
2017-06-01Fix check_process_code() on hibernated processRickard Green
2017-05-16Merge pull request #1436 from garazdawi/lukas/erts/remove_unused_functionsLukas Larsson
erts: Remove old unused functions
2017-05-16erts: Remove old unused functionsLukas Larsson
The functions have been found using: https://github.com/caolanm/callcatcher
2017-05-04Update copyright yearRaimo Niskanen
2017-04-25Merge branch 'sverker/refactor'Sverker Eriksson
* sverker/refactor: erts: Introduce struct binary_internals erts: Introduce erts_bin_release erts: Init refc=1 in erts_bin_drv_alloc* erts: Init refc=1 in erts_bin_nrml_alloc erts: Remove deliberate leak of hipe fun entries erts: Remove hipe_bifs:remove_refs_from/1 Refactor hipe specific code to use ErtsCodeInfo erts: Refactor ErtsCodeInfo.native
2017-04-12erts: Introduce struct binary_internalsSverker Eriksson
to replace macro ERTS_INTERNAL_BINARY_FIELDS as header in Binary and friends.
2017-04-11Fix dirty GC implementationRickard Green
2017-04-11erts: Introduce erts_bin_releaseSverker Eriksson
2017-02-28erts: Refactor MOVE_CONS to inline functionBjörn-Egil Dahlberg
2017-02-28erts: Refactor MOVE_BOXED to inline functionBjörn-Egil Dahlberg
2017-02-28erts: Convert small sub-binaries to heap-binariesBjörn-Egil Dahlberg
In many cases sub-binaries costs more memory than converting them to heap-binaries. Sub-binaries also has a hidden cost of pinning larger binaries in memory. By converting binaries this cost is reduced. Byte aligned sub-binaries upto 24 bytes (64-bit) or 12 bytes (32-bit) are converted.
2017-02-14Fixed typos in ertsAndrew Dryga
2017-02-06Add binary overhead for magic ref/binariesRickard Green
2017-02-06Implement magic referencesRickard Green
Magic references are *intentionally* indistinguishable from ordinary references for the Erlang software. Magic references do not change the language, and are intended as a pure runtime internal optimization. An ordinary reference is typically used as a key in some table. A magic reference has a direct pointer to a reference counted magic binary. This makes it possible to implement various things without having to do lookups in a table, but instead access the data directly. Besides very fast lookups this can also improve scalability by removing a potentially contended table. A couple of examples of planned future usage of magic references are ETS table identifiers, and BIF timer identifiers. Besides future optimizations using magic references it should also be possible to replace the exposed magic binary cludge with magic references. That is, magic binaries that are exposed as empty binaries to the Erlang software.
2017-02-06Merge branch 'maint'Rickard Green
* maint: Atomic reference count of binaries also in non-SMP Conflicts: erts/emulator/beam/erl_fun.c
2017-02-06Merge branch 'rickard/binary-refc' into maintRickard Green
OTP-14202 * rickard/binary-refc: Atomic reference count of binaries also in non-SMP Conflicts: erts/emulator/beam/beam_bp.c
2017-02-06Atomic reference count of binaries also in non-SMPRickard Green
NIF resources was not handled in a thread-safe manner in the runtime system without SMP support. As a consequence of this fix, the following driver functions are now thread-safe also in the runtime system without SMP support: - driver_free_binary() - driver_realloc_binary() - driver_binary_get_refc() - driver_binary_inc_refc() - driver_binary_dec_refc()
2017-01-23Merge branch 'sverker/ASSERT_IN_ENV'Sverker Eriksson
* sverker/ASSERT_IN_ENV: erts: Add macro ERTS_PROC_LOCKS_HIGHER_THAN erts: Cleanup and extra assertions in nif_SUITE.c erts: Cleanup enif_make_reverse_list erts: Add assertions for correct ErlNifEnv erts: Make erts_dbg_within_proc available # Conflicts: # erts/emulator/beam/erl_gc.h
2017-01-17Merge branch 'maint'Sverker Eriksson