| Age | Commit message (Collapse) | Author |
|---|
|
by populating the table with the help of a random number generator
creating series of unique integers.
|
|
|
|
of repeated table opts
and waiting for workers
|
|
The current ETS ordered_set implementation can quickly become a
scalability bottleneck on multicore machines when an application updates
an ordered_set table from concurrent processes [1][2]. The current
implementation is based on an AVL tree protected from concurrent writes
by a single readers-writer lock. Furthermore, the current implementation
has an optimization, called the stack optimization [3], that can improve
the performance when only a single process accesses a table but can
cause bad scalability even in read-only scenarios. It is possible to
pass the option {write_concurrency, true} to ets:new/2 when creating an
ETS table of type ordered_set but this option has no effect for tables
of type ordered_set without this commit. The new ETS ordered_set
implementation, added by this commit, is only activated when one passes
the options ordered_set and {write_concurrency, true} to the ets:new/2
function. Thus, the previous ordered_set implementation (from here on
called the default implementation) can still be used in applications
that do not benefit from the new implementation. The benchmark results
on the following web page show that the new implementation is many times
faster than the old implementation in some scenarios and that the old
implementation is still better than the new implementation in some
scenarios.
http://winsh.me/ets_catree_benchmark/ets_ca_tree_benchmark_results.html
The new implementation is expected to scale better than the default
implementation when concurrent processes use the following ETS
operations to operate on a table:
delete/2, delete_object/2, first/1, insert/2 (single object),
insert_new/2 (single object), lookup/2, lookup_element/2, member/2,
next/2, take/2 and update_element/3 (single object).
Currently, the new implementation does not have scalable support for the
other operations (e.g., select/2). However, when these operations are
used infrequently, the new implantation may still scale better than the
default implementation as the benchmark results at the URL above shows.
Description of the New Implementation
----------------------------------
The new implementation is based on a data structure which is called the
contention adapting search tree (CA tree for short). The following
publication contains a detailed description of the CA tree:
A Contention Adapting Approach to Concurrent Ordered Sets
Journal of Parallel and Distributed Computing, 2018
Kjell Winblad and Konstantinos Sagonas
https://doi.org/10.1016/j.jpdc.2017.11.007
http://www.it.uu.se/research/group/languages/software/ca_tree/catree_proofs.pdf
A discussion of how the CA tree can be used as an ETS back-end can be
found in another publication [1]. The CA tree is a data structure that
dynamically changes its synchronization granularity based on detected
contention. Internally, the CA tree uses instances of a sequential data
structure to store items. The CA tree implementation contained in this
commit uses the same AVL tree implementation as is used for the default
ordered set implementation. This AVL tree implementation is reused so
that much of the existing code to implement the ETS operations can be
reused.
Tests
-----
The ETS tests in `lib/stdlib/test/ets_SUITE.erl` have been extended to
also test the new ordered_set implementation. The function
ets_SUITE:throughput_benchmark/0 has also been added to this file. This
function can be used to measure and compare the performance of the
different ETS table types and options. This function writes benchmark
data to standard output that can be visualized by the HTML page
`lib/stdlib/test/ets_SUITE_data/visualize_throughput.html`.
[1]
More Scalable Ordered Set for ETS Using Adaptation.
In Thirteenth ACM SIGPLAN workshop on Erlang (2014).
Kjell Winblad and Konstantinos Sagonas.
https://doi.org/10.1145/2633448.2633455
http://www.it.uu.se/research/group/languages/software/ca_tree/erlang_paper.pdf
[2]
On the Scalability of the Erlang Term Storage
In Twelfth ACM SIGPLAN workshop on Erlang (2013)
Kjell Winblad, David Klaftenegger and Konstantinos Sagonas
https://doi.org/10.1145/2505305.2505308
http://winsh.me/papers/erlang_workshop_2013.pdf
[3]
The stack optimization works by keeping one preallocated stack instance
in every ordered_set table. This stack is updated so that it contains
the search path in some read operations (e.g., ets:next/2). This makes
it possible for a subsequent ets:next/2 to avoid traversing some nodes
in some cases. Unfortunately, the preallocated stack needs to be flagged
so that it is not updated concurrently by several threads which cause
bad scalability.
|
|
|
|
* sverker/erts/ets-memstat-false-leak/ERL-720/OTP-15278:
erts: Refactor ets FixedDeletion allocations
erts: Fix ets memstat false leak of FixedDeletion
|
|
|
|
Introduce a put_tuple2 instruction
|
|
* ingela/ssl/initial-tls-1.3-functions:
ssl: Initial cipher suites adoption for TLS-1.3
ssl: Add new TLS-1.3 Alerts
ssl: Add initial TLS 1.3 hanshake encode/decode support
|
|
This commit filters out cipher suites not to be used in TLS-1.3
We still need to add new cipher suites for TLS-1.3 and possible
add new information to the suite data structure.
|
|
|
|
|
|
|
|
|
|
Erl compare ext lists bug (ERL-705)
|
|
|
|
Fix bug in compact representation of float_to_list/2
|
|
Fix type spec of ms_transform:parse_trans/2
|
|
|
|
causing erlang:memory to report too much ets memory.
|
|
Sometimes when building a tuple, there is no way to avoid an
extra `move` instruction. Consider this code:
make_tuple(A) -> {ok,A}.
The corresponding BEAM code looks like this:
{test_heap,3,1}.
{put_tuple,2,{x,1}}.
{put,{atom,ok}}.
{put,{x,0}}.
{move,{x,1},{x,0}}.
return.
To avoid overwriting the source register `{x,0}`, a `move`
instruction is necessary.
The problem doesn't exist when building a list:
%% build_list(A) -> [A].
{test_heap,2,1}.
{put_list,{x,0},nil,{x,0}}.
return.
Introduce a new `put_tuple2` instruction that builds a tuple in a
single instruction, so that the `move` instruction can be eliminated:
%% make_tuple(A) -> {ok,A}.
{test_heap,3,1}.
{put_tuple2,{x,0},{list,[{atom,ok},{x,0}]}}.
return.
Note that the BEAM loader already combines `put_tuple` and `put`
instructions into an internal instruction similar to `put_tuple2`.
Therefore the introduction of the new instruction will not speed up
execution of tuple building itself, but it will be less work for
the loader to load the new instruction.
|
|
* maint:
ops.tab: Fix potentially unsafe optimization of raise/2
|
|
The operands for the raise/2 instruction are almost always in x(2) and
x(1). Therefore the loader translates the raise/2 instruction to an
i_raise/0 instruction which uses the values in x(2) and x(1). If the
operands happens to be in other registers, the loader inserts move/2
instruction to move them to x(2) and x(1).
The problem is that x(3) is used as a temporary register when
generating the move/2 instructions. That is unsafe if the
Value operand for raise/2 is x(3).
Thus:
raise x(0) x(3)
will be translated to:
move x(0) x(3)
move x(3) x(1)
move x(3) x(2)
i_raise
The Trace will be written to both x(2) and x(1).
The current compiler will never use x(3) for the Value operand,
so there is no need to patch previous releases. But a future compiler
version might allocate registers differently.
|
|
* maint:
crypto: Let otp_test_engine only add what is needed OpenSSL_add_all_algorithms hangs on some test machines
|
|
* hans/crypto/init_test_engine_fix:
crypto: Let otp_test_engine only add what is needed OpenSSL_add_all_algorithms hangs on some test machines
|
|
* maint:
|
|
* hasse/dialyzer/improve_guards/OTP-15268/ERL-680:
dialyzer: Improve handling of complex guards
|
|
uabboli/hasse/dialyzer/improve_guards/OTP-15268/ERL-680
dialyzer: Improve handling of complex guards
|
|
It can also return errors and warnings.
|
|
Conflicts:
lib/ssl/src/ssl_connection.erl
lib/ssl/src/tls_connection.erl
|
|
* ingela/ssl/send-recv-dead-lock/ERL-622:
ssl: Improve close handling
ssl: Adopt distribution over TLS to use new sender process
ssl: Add new sender process for TLS state machine
|
|
* maint:
Updated OTP version
Update release notes
Update version numbers
Fix missing 'in' trace events during 'running' trace
|
|
* maint-21:
Updated OTP version
Update release notes
Update version numbers
Fix missing 'in' trace events during 'running' trace
|
|
OpenSSL_add_all_algorithms hangs on some test machines
|
|
|
|
|
|
|
|
* rickard/running-trace-fix/ERL-713/OTP-15269:
Fix missing 'in' trace events during 'running' trace
|
|
* rickard/fix-suspend-monitor-down/OTP-15237/ERL-704:
Fix incoming suspend monitor down
|
|
* rickard/crypto-configure/OTP-15129:
Fix crypto configure on Darwin
|
|
* maint:
Updated OTP version
Update release notes
stdlib: Let dets:open_file() crash when given raw file name
Fix kernel_app doc logger_level default from info to notice
Change-Id: I581946ac5cec6574ed79017e2987039c1fdcf80a
|
|
* maint-20:
Updated OTP version
Update release notes
Change-Id: I78586395e5784dc76b8a803a25f8054a251e1fd8
|
|
Update httpc_manager.erl to fix typo
|
|
Fix kernel_app doc logger_level default from info to notice
|
|
|
|
|
|
uabboli/hb/stdlib/fix_dets_file_name/OTP-15253/ERL-555
stdlib: Let dets:open_file() crash when given raw file name
|
|
See also https://bugs.erlang.org/browse/ERL-680.
The right associative short circuit expressions 'andalso' and 'orelse'
are expanded by the Compiler (see v3_core) into 'case' expressions. If
parentheses are used to enforce left associativeness, variables are
introduced, and the time needed by Dialyzer increases exponentially.
Rather than trying to fix Dialyzer itself, v3_core now rewrites
repeated use of 'andalso' ('orelse') into right associative
expressions before creating the 'case' expressions.
|
|
|
|
* maint:
ssl: Fix dialyzer errors detected when crypto.erl is typed
|
