aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam
AgeCommit message (Collapse)Author
2019-04-10Decentralized counters for ETS ordered_set with write_concurrencyKjell Winblad
Previously, all ETS tables used centralized counter variables to keep track of the number of items stored and the amount of memory consumed. These counters can cause scalability problems (especially on big NUMA systems). This commit adds an implementation of a decentralized counter and modifies the implementation of ETS so that ETS tables of type ordered_set with write_concurrency enabled use the decentralized counter. [Experiments][1] indicate that this change substantially improves the scalability of ETS ordered_set tables with write_concurrency enabled in scenarios with frequent `ets:insert/2` and `ets:delete/2` calls. The new counter is implemented in the module erts_flxctr (`erts_flxctr.h` and `erts_flxctr.c`). The module has the suffix flxctr as it contains the implementation of a flexible counter (i.e., counter instances can be configured to be either centralized or decentralized). Counters that are configured to be centralized are implemented with a single counter variable which is modified with atomic operations. Decentralized counters are spread over several cache lines (how many can be configured with the parameter `+dcg`). The scheduler threads are mapped to cache lines so that there is no single point of contention when decentralized counters are updated. The thread progress functionality of the Erlang VM is utilized to implement support for linearizable snapshots of decentralized counters. The snapshot functionality is used by the `ets:info/1` and `ets:info/2` functions. [1]: http://winsh.me/ets_catree_benchmark/flxctr_res.html
2019-04-10Merge pull request #2204 from jhogberg/john/erts/optimize-arith-ops/OTP-15740John Högberg
erts: Optimize arithmetic ops using overflow intrinsics
2019-04-10Merge branch 'john/merge-re-replace-fix'John Högberg
* john/merge-re-replace-fix: erts: Skip ERTS_NOINLINE on non-GCC-compatible compilers cleanup: beam_emu.c: use ERTS_NOINLINE instead of NOINLINE stdlib: fix re:replace on LTO builds
2019-04-09erts: Fix dist sequence yield state allocator typeLukas Larsson
The wrong allocator type was given to erts_free.
2019-04-09erts: Yield when exiting/free process is suspended by deLukas Larsson
Complementory fix to 922fd355831575965
2019-04-09erts: Fix alignment of ErtsDistExternal on sparcLukas Larsson
The edep->data values need to be 64-bit aligned as it contains 64-bit values.
2019-04-09erts: Fix incorrect gcc may be used init warningLukas Larsson
2019-04-09erts: Fix lock order violation in system_info(procs)Lukas Larsson
2019-04-09erts: Fix buffer alignment bug in alloc_dist_obufLukas Larsson
Cannot do unaligned word writes on sparc!
2019-04-09erts: Fix build of dist_ctrl_get_data return termLukas Larsson
2019-04-08erts: Optimize arithmetic ops using overflow intrinsicsJohn Högberg
2019-04-08erts: Skip ERTS_NOINLINE on non-GCC-compatible compilersJohn Högberg
__declspec(noinline) works fine on MSVC but requires us to place the macro before a function rather than after, which in turn causes early versions of GCC to puke since they only accept __attribute__ at the end of a function declaration. Since this is a new macro that previously only saw use in beam_emu, I figured it's easiest to leave it disabled on MSVC.
2019-04-05Optimize the i_minus instructionBjörn Gustavsson
Measurements show that i_minus instructions (subtraction) are frequent enough to warrant creating specialized i_minus instructions. Thanks to José Valim for doing instruction counting on Elixir code.
2019-04-05Merge remote-tracking branch 'trofi/master' into john/merge-re-replace-fixJohn Högberg
PR 2194, stdlib: fix re:replace on LTO builds * trofi/master: cleanup: beam_emu.c: use ERTS_NOINLINE instead of NOINLINE stdlib: fix re:replace on LTO builds
2019-04-01Merge branch 'john/kernel/cuddle-file-suite'John Högberg
* john/kernel/cuddle-file-suite: file_name_SUITE: Darwin 18 broke support for 'icky' names file_SUITE: Ignore EILSEQ on Darwin in +fnu/latin1 test
2019-03-29cleanup: beam_emu.c: use ERTS_NOINLINE instead of NOINLINESergei Trofimovich
Signed-off-by: Sergei Trofimovich <[email protected]>
2019-03-29stdlib: fix re:replace on LTO buildsSergei Trofimovich
Fabio Coatti reported elixir build failure in https://bugs.gentoo.org/681778. The minimal reproducer looks like that (from otp git tree): $ ./configure CFLAGS='-O2 -flto' LDFLAGS='-O2 -flto=8' $ make $ ERL_TOP=$PWD \ PATH=$ERL_TOP/bin:$PATH \ \ bin/erl \ \ -noshell -eval 're:replace("a","b","c",[{return,list}]).' \ -s erlang halt {"init terminating in do_boot",{badarg,[{re,replace,["a","b","c",[{return,list}]], [{file,"re.erl"},{line,362}]}, {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,680}]}, {init,start_it,1,[]}, {init,start_em,1,[]}, {init,do_boot,3,[]}]}} init terminating in do_boot ({badarg,[{re,replace,[[_],[_],[_],[_]],[{_},{_}]}, {erl_eval,do_apply,6,[{_},{_}]},{init,start_it,1,[]},{init,start_em,1,[]},{init,do_boot,3,[]}]}) Crash dump is being written to: erl_crash.dump...done The failure happens in libpcre2 where stack overflow is mis-identified at function entry of erts_pcre_compile2() compile_regex() if (PUBL(stack_guard) != NULL && PUBL(stack_guard)()) { *errorcodeptr= ERR85; return FALSE; } The stack "overflow" detection happens in thr_wrapper() ethr_set_stacklimit__() because the stack usage code relies on the fact that ethr_set_stacklimit__() and similar functions don't get inlined into callers for stack growth measurement. Before the change inlining avoidance was achieved by putting functions into standalone translation units. LTO makes this technique inefficient. The change marks functions explicitly as __attribute__((__noinline__)) on gcc. Reported-by: Fabio Coatti Bug: https://bugs.gentoo.org/681778 Signed-off-by: Sergei Trofimovich <[email protected]>
2019-03-29Merge branch 'maint'Sverker Eriksson
2019-03-29Merge branch 'sverker/process_info-reductions-fix/OTP-15709' into maintSverker Eriksson
* sverker/process_info-reductions-fix/OTP-15709: erts: Fix bug in process_info(reductions) erts: Use ptr_val for pointer in gc msg copy
2019-03-29Fix out of memory bug in the implementation of mapsKjell Winblad
This commit fixes a bug that could cause a crash or memory usage to grow until the machine ran out of memory when adding a key-value pair to a map. This could happen when inserting a new key-value pair with a key K1 containing a binary B1 into a map M having a key K2 with a binary B2 if the following conditions were met: * size(B1) >= 4294967296 * size(B2) >= 4294967296 * size(M) >= 32 * (size(B1) rem 4294967296) == (size(B2) rem 4294967296) * the first (size(B1) rem 4294967296) bytes are the same both in B1 and B2 * substituting B1 in K1 with B2 would result in a term with the same value as K2 The root cause of the bug is that the map implementation only hashed the first (X modulo 4294967296) bytes of binaries so that different binaries could get hashed to the same hash value independently of the hash seed.
2019-03-29file_SUITE: Ignore EILSEQ on Darwin in +fnu/latin1 testJohn Högberg
There's no way to create the file in question, so we may as well skip the test.
2019-03-28erts: Fix bug in process_info(reductions)Sverker Eriksson
returning incorrect result as * current process might not be RUNNING in which case REDS_IN is actually used as def_arg_reg[5] * FCALLS might not have been swapped out * the SAVED_CALLS case was wrong and returned number of reds left
2019-03-28erts: Use ptr_val for pointer in gc msg copySverker Eriksson
Cherry-pick: 3d7d66b84b69f3eb42e8bcd8dc510b6676e31cac Without this fix asserts would trigger in debug build but nothing else would break.
2019-03-28Merge branch 'john/erts/remove-destructive-bs_get_binary2/ERL-901'John Högberg
* john/erts/remove-destructive-bs_get_binary2/ERL-901: erts: Remove unsafe bs_get_binary2 optimization from loader
2019-03-27erts: Remove unsafe bs_get_binary2 optimization from loaderJohn Högberg
A load-time optimization assumed that match contexts had no further uses when a bs_get_binary2 overwrote the match context's register, and figured it would be safe to reuse the match context's memory for the resulting binary. This is no longer safe as of OTP 22, as a match context may be reused after being passed to another function.
2019-03-27Merge branch 'john/erts/seq_trace-sigq-clearing/OTP-15704'John Högberg
* john/erts/seq_trace-sigq-clearing/OTP-15704: erts: Handle all signal types when clearing seq_trace token
2019-03-27erts: Handle all signal types when clearing seq_trace tokenJohn Högberg
We walk through all signal queues in the system when clearing seq_trace, so we must be prepared to encounter all kinds of signals.
2019-03-27Fix unsafe optimization made by the loaderBjörn Gustavsson
Fix the unsafe load-time optimization introduced in 07bdbb3a1edc. https://bugs.erlang.org/browse/ERL-899
2019-03-26Merge branch 'john/erts/explain-empty-stack-check-in-next_catch'John Högberg
* john/erts/explain-empty-stack-check-in-next_catch: erts: Add an explanation for the empty stack check in next_catch
2019-03-26Revert "Prepare release"Henrik Nord
This reverts commit 0118046f9bda8dcf3594184fa439b79267db8017.
2019-03-25Prepare releaseErlang/OTP
2019-03-25Merge branch 'sverker/bug-fixing'Sverker Eriksson
* sverker/bug-fixing: erts: Fix erts_debug:set_internal_state(reds_left) erts: Fix binary_SUITE:cmp_old_impl
2019-03-25erts: Fix erts_debug:set_internal_state(reds_left)Sverker Eriksson
to avoid failed ERTS_DBG_CHK_REDS by clearing virtual_reds.
2019-03-25Merge branch ↵Lukas Larsson
'lukas/erts/fragment-dist-messages/OTP-13397/OTP-15610/OTP-15611/OTP-15612/OTP-15613/OTP-15703' * lukas/erts/fragment-dist-messages/OTP-13397/OTP-15610/OTP-15611/OTP-15612/OTP-15613/OTP-15703: (26 commits) erts: Yield when exiting/free process is suspended by de doc: Fix Design Princ, statem incorrect anchor erts: Include external msg in need of GC erts: Include dist header in return from dist_ctrl_get_data erts: Fix so that exit/down terms stay alive erts: Pending signals can be for free processes erts: Fix faulty assert in reference_table_term etp: Don't crash etp-stack* when c_p->i is null erts: Adjust dist obuf size correctly after hdr fin erts: Fix non-payload dist exit signals erts: Always do trylock on proc locks when crash dumping Fix tests to work better in debug emulator erts: erts_factory_proc_init should not set hole marker erts: Skip heavy process tab tests in debug emu ts: Use same dynlinking logic for all bsd Revert "erts: Always run fds check after each testcase" erts: Always run fds check after each testcase erts: Always stop any testnodes before testcase exits erts: Add crash dumping of EXITING and FREE processes wx: Remove ERL_FLAGS from Makefile erlc command ...
2019-03-25erts: Yield when exiting/free process is suspended by deLukas Larsson
2019-03-25erts: Add an explanation for the empty stack check in next_catchJohn Högberg
2019-03-25erts: Include external msg in need of GCLukas Larsson
When many external messages suddenly appear in the mailbox the young gen size is adjusted accordingly but it was immediately shrunk as the data was not counted towards the shrink size. This commit includes the ext dist size in the shrink calculation which means that the decode of the external messages will not trigger as many GCs which means much better performance.
2019-03-25erts: Include dist header in return from dist_ctrl_get_dataLukas Larsson
2019-03-25erts: Fix so that exit/down terms stay aliveLukas Larsson
When the GC is run or the link/monitor node is deleted, the terms associated with it will also be deallocated so we have to make sure that we don't do any GCs and that all link/monitor nodes are copied onto the heap before the node is deallocated.
2019-03-25erts: Pending signals can be for free processesLukas Larsson
Since we no schedule free processes multiple times, pending signals can be from free as well as current processes.
2019-03-25erts: Fix faulty assert in reference_table_termLukas Larsson
2019-03-25erts: Adjust dist obuf size correctly after hdr finLukas Larsson
2019-03-25erts: Fix non-payload dist exit signalsLukas Larsson
When interacting with jinterface and erl_interface the old dist messages are used and they incorrectly used the external data.
2019-03-25erts: Always do trylock on proc locks when crash dumpingLukas Larsson
2019-03-25erts: erts_factory_proc_init should not set hole markerLukas Larsson
Setting the hole marker in debug on the entire heap is very slow so instead we do it in erts_produce_heap.
2019-03-25erts: Add crash dumping of EXITING and FREE processesLukas Larsson
2019-03-25erts: Fix incorrect ticket example in commentJohn Högberg
2019-03-25Revert "erts: Add debug dist obuf memory leak check"Lukas Larsson
This reverts commit f4c121b1d98bf3db7e6eecbb9fb5b292f2bc3bb0.
2019-03-25erts: Fix version error in erl_nif.hJohn Högberg
2019-03-22Merge branch 'sverker/enable-big-creation/OTP-15603'Sverker Eriksson
* sverker/enable-big-creation/OTP-15603: epmd: Support 32-bit creation values in local node erts: Robustify epmd reply function erts: Reject decoded local refs with too large first word erts: Fix bug in list_to_ref erl_interface: Remove old encoding of pid,port,refs erts: Remove old encoding of pids, ports and refs erts: Make DFLAG_BIG_CREATION mandatory