| Age | Commit message (Collapse) | Author |
|---|
|
|
|
of repeated table opts
and waiting for workers
|
|
The current ETS ordered_set implementation can quickly become a
scalability bottleneck on multicore machines when an application updates
an ordered_set table from concurrent processes [1][2]. The current
implementation is based on an AVL tree protected from concurrent writes
by a single readers-writer lock. Furthermore, the current implementation
has an optimization, called the stack optimization [3], that can improve
the performance when only a single process accesses a table but can
cause bad scalability even in read-only scenarios. It is possible to
pass the option {write_concurrency, true} to ets:new/2 when creating an
ETS table of type ordered_set but this option has no effect for tables
of type ordered_set without this commit. The new ETS ordered_set
implementation, added by this commit, is only activated when one passes
the options ordered_set and {write_concurrency, true} to the ets:new/2
function. Thus, the previous ordered_set implementation (from here on
called the default implementation) can still be used in applications
that do not benefit from the new implementation. The benchmark results
on the following web page show that the new implementation is many times
faster than the old implementation in some scenarios and that the old
implementation is still better than the new implementation in some
scenarios.
http://winsh.me/ets_catree_benchmark/ets_ca_tree_benchmark_results.html
The new implementation is expected to scale better than the default
implementation when concurrent processes use the following ETS
operations to operate on a table:
delete/2, delete_object/2, first/1, insert/2 (single object),
insert_new/2 (single object), lookup/2, lookup_element/2, member/2,
next/2, take/2 and update_element/3 (single object).
Currently, the new implementation does not have scalable support for the
other operations (e.g., select/2). However, when these operations are
used infrequently, the new implantation may still scale better than the
default implementation as the benchmark results at the URL above shows.
Description of the New Implementation
----------------------------------
The new implementation is based on a data structure which is called the
contention adapting search tree (CA tree for short). The following
publication contains a detailed description of the CA tree:
A Contention Adapting Approach to Concurrent Ordered Sets
Journal of Parallel and Distributed Computing, 2018
Kjell Winblad and Konstantinos Sagonas
https://doi.org/10.1016/j.jpdc.2017.11.007
http://www.it.uu.se/research/group/languages/software/ca_tree/catree_proofs.pdf
A discussion of how the CA tree can be used as an ETS back-end can be
found in another publication [1]. The CA tree is a data structure that
dynamically changes its synchronization granularity based on detected
contention. Internally, the CA tree uses instances of a sequential data
structure to store items. The CA tree implementation contained in this
commit uses the same AVL tree implementation as is used for the default
ordered set implementation. This AVL tree implementation is reused so
that much of the existing code to implement the ETS operations can be
reused.
Tests
-----
The ETS tests in `lib/stdlib/test/ets_SUITE.erl` have been extended to
also test the new ordered_set implementation. The function
ets_SUITE:throughput_benchmark/0 has also been added to this file. This
function can be used to measure and compare the performance of the
different ETS table types and options. This function writes benchmark
data to standard output that can be visualized by the HTML page
`lib/stdlib/test/ets_SUITE_data/visualize_throughput.html`.
[1]
More Scalable Ordered Set for ETS Using Adaptation.
In Thirteenth ACM SIGPLAN workshop on Erlang (2014).
Kjell Winblad and Konstantinos Sagonas.
https://doi.org/10.1145/2633448.2633455
http://www.it.uu.se/research/group/languages/software/ca_tree/erlang_paper.pdf
[2]
On the Scalability of the Erlang Term Storage
In Twelfth ACM SIGPLAN workshop on Erlang (2013)
Kjell Winblad, David Klaftenegger and Konstantinos Sagonas
https://doi.org/10.1145/2505305.2505308
http://winsh.me/papers/erlang_workshop_2013.pdf
[3]
The stack optimization works by keeping one preallocated stack instance
in every ordered_set table. This stack is updated so that it contains
the search path in some read operations (e.g., ets:next/2). This makes
it possible for a subsequent ets:next/2 to avoid traversing some nodes
in some cases. Unfortunately, the preallocated stack needs to be flagged
so that it is not updated concurrently by several threads which cause
bad scalability.
|
|
|
|
* sverker/erts/ets-memstat-false-leak/ERL-720/OTP-15278:
erts: Refactor ets FixedDeletion allocations
erts: Fix ets memstat false leak of FixedDeletion
|
|
Introduce a put_tuple2 instruction
|
|
* ingela/ssl/initial-tls-1.3-functions:
ssl: Initial cipher suites adoption for TLS-1.3
ssl: Add new TLS-1.3 Alerts
ssl: Add initial TLS 1.3 hanshake encode/decode support
|
|
This commit filters out cipher suites not to be used in TLS-1.3
We still need to add new cipher suites for TLS-1.3 and possible
add new information to the suite data structure.
|
|
|
|
|
|
|
|
Erl compare ext lists bug (ERL-705)
|
|
|
|
Fix type spec of ms_transform:parse_trans/2
|
|
causing erlang:memory to report too much ets memory.
|
|
Sometimes when building a tuple, there is no way to avoid an
extra `move` instruction. Consider this code:
make_tuple(A) -> {ok,A}.
The corresponding BEAM code looks like this:
{test_heap,3,1}.
{put_tuple,2,{x,1}}.
{put,{atom,ok}}.
{put,{x,0}}.
{move,{x,1},{x,0}}.
return.
To avoid overwriting the source register `{x,0}`, a `move`
instruction is necessary.
The problem doesn't exist when building a list:
%% build_list(A) -> [A].
{test_heap,2,1}.
{put_list,{x,0},nil,{x,0}}.
return.
Introduce a new `put_tuple2` instruction that builds a tuple in a
single instruction, so that the `move` instruction can be eliminated:
%% make_tuple(A) -> {ok,A}.
{test_heap,3,1}.
{put_tuple2,{x,0},{list,[{atom,ok},{x,0}]}}.
return.
Note that the BEAM loader already combines `put_tuple` and `put`
instructions into an internal instruction similar to `put_tuple2`.
Therefore the introduction of the new instruction will not speed up
execution of tuple building itself, but it will be less work for
the loader to load the new instruction.
|
|
* maint:
crypto: Let otp_test_engine only add what is needed OpenSSL_add_all_algorithms hangs on some test machines
|
|
* hans/crypto/init_test_engine_fix:
crypto: Let otp_test_engine only add what is needed OpenSSL_add_all_algorithms hangs on some test machines
|
|
* hasse/dialyzer/improve_guards/OTP-15268/ERL-680:
dialyzer: Improve handling of complex guards
|
|
uabboli/hasse/dialyzer/improve_guards/OTP-15268/ERL-680
dialyzer: Improve handling of complex guards
|
|
It can also return errors and warnings.
|
|
Conflicts:
lib/ssl/src/ssl_connection.erl
lib/ssl/src/tls_connection.erl
|
|
* ingela/ssl/send-recv-dead-lock/ERL-622:
ssl: Improve close handling
ssl: Adopt distribution over TLS to use new sender process
ssl: Add new sender process for TLS state machine
|
|
OpenSSL_add_all_algorithms hangs on some test machines
|
|
* rickard/crypto-configure/OTP-15129:
Fix crypto configure on Darwin
|
|
* maint:
Updated OTP version
Update release notes
stdlib: Let dets:open_file() crash when given raw file name
Fix kernel_app doc logger_level default from info to notice
Change-Id: I581946ac5cec6574ed79017e2987039c1fdcf80a
|
|
* maint-20:
Updated OTP version
Update release notes
Change-Id: I78586395e5784dc76b8a803a25f8054a251e1fd8
|
|
Update httpc_manager.erl to fix typo
|
|
Fix kernel_app doc logger_level default from info to notice
|
|
|
|
uabboli/hb/stdlib/fix_dets_file_name/OTP-15253/ERL-555
stdlib: Let dets:open_file() crash when given raw file name
|
|
See also https://bugs.erlang.org/browse/ERL-680.
The right associative short circuit expressions 'andalso' and 'orelse'
are expanded by the Compiler (see v3_core) into 'case' expressions. If
parentheses are used to enforce left associativeness, variables are
introduced, and the time needed by Dialyzer increases exponentially.
Rather than trying to fix Dialyzer itself, v3_core now rewrites
repeated use of 'andalso' ('orelse') into right associative
expressions before creating the 'case' expressions.
|
|
* maint:
ssl: Fix dialyzer errors detected when crypto.erl is typed
|
|
* hans/ssl/dialyzer_crypto_typed/OTP-15271:
ssl: Fix dialyzer errors detected when crypto.erl is typed
|
|
* maint:
inets: Prepare for release
inets: Robust handling of 204, 304, 1xx responses
inets: Do not use chunked encoding with 1xx, 204, 304
Change-Id: Ia739a7b1a7f1482194def99d039b5a7c8133a6ca
|
|
* maint-20:
inets: Prepare for release
inets: Robust handling of 204, 304, 1xx responses
inets: Do not use chunked encoding with 1xx, 204, 304
Change-Id: Ia88eb1d217b46fb01b1d7544d9ef34788c87abbd
|
|
* maint:
Document allowed integer ops in types
stdlib: Fix specs of filename:basedir/2,2
|
|
* peterdmv/inets/fix_http_client/OTP-15242:
inets: Prepare for release
inets: Robust handling of 204, 304, 1xx responses
Change-Id: I12dced982907c3462fefb8a4ffaae8b365821f97
|
|
* hasse/stdlib/fix_basedir_specs/OTP-15252/ERL-667:
stdlib: Fix specs of filename:basedir/2,2
|
|
* peterdmv/inets/fix_http_server/OTP-15241:
inets: Do not use chunked encoding with 1xx, 204, 304
Change-Id: I4dc1cb6dc62cc5a090d49248c5fbfbb23f33004f
|
|
We want to make sure that the sender process that may get stuck in
prim_inet:send will die if the tls_connection process is
terminated. And we also like to make sure that it terminates as
gracefully as possible. So when the tls_connection process dies it
spawns a killer process that will brutaly kill the sender if it is
unresponsive and does not terminate due to its monitor of the
tls_connetion process triggering.
When the sender process also acts as distribution controller it
may also have other processess that it is linked with that it
should bring down or that could bring the connection down.
|
|
|
|
Separate sending and receiving when using TCP as transport
as prim_inet:send may block which in turn may result
in a deadlock between two Erlang processes communicating over
TLS, this is especially likely to happen when running Erlang distribution
over TLS.
|
|
|
|
Change-Id: I891cc997475780f22a60119778984739d560f203
|
|
All 1xx (informational), 204 (no content), and 304 (not modified)
responses MUST NOT include a message-body, and thus are always
terminated by the first empty line after the header fields.
This implies that chunked encoding MUST NOT be used for these
status codes.
This commit updates the client to gracefully handle responses from
faulty server implementations that can send chunked encoded 204,
304 or 1xx responses.
Change-Id: I2dd502e28b3c6e121640083118fa5c3e479f5194
|
|
|
|
* bjorn/compiler/ssa:
Travis CI: Run the SSA linter in the Linux64SmokeTest build
Remove retired compiler passes
Introduce a new SSA-based intermediate format
hipe_beam_to_icode: Correct translation of get_map_elements
beam_dead: Remove shortcut of binary matching instruction
beam_bs: Remove optimizations that are easier done on SSA format
Don't run unsafe compiler passes
Simplify optimizations by introducing is_nil late
beam_utils: Make is_tagged_tuple a pure test
beam_except: Enhance recognition of function_clause exceptions
beam_validator: Infer the types of copies in a smarter way
beam_validator: Improve merge of cons and literal list
beam_validator: Strengthen validation of func_info
beam_validator: Allow get_tuple_element before dsetelement
beam_validator: Don't transfer state to labels that can't be reached
beam_validator: Improve type analysis for tuples
beam_validator: Be more careful when updating try/catch state
beam_trim: Handle an empty list of instructions
v3_core: Number argument variables in ascending order
Teach binary instructions to use Y registers as destination
OTP-14894
|
|
* maint:
Updated OTP version
Prepare release
Updated the engine load functionality
inets: Prepare for release
inets: Use status code 501 when no mod_* handles the request
ssl: Prepare for release
ssl: Make sure that a correct cipher suite is selected
ssl: Make sure that a correct cipher suite is selected
|
|
* maint-21:
Updated OTP version
Prepare release
Updated the engine load functionality
inets: Prepare for release
inets: Use status code 501 when no mod_* handles the request
ssl: Prepare for release
ssl: Make sure that a correct cipher suite is selected
ssl: Make sure that a correct cipher suite is selected
|
