diff options
-rw-r--r-- | erts/doc/src/notes.xml | 18 | ||||
-rw-r--r-- | lib/common_test/doc/src/ct_hooks.xml | 6 | ||||
-rw-r--r-- | lib/common_test/doc/src/notes.xml | 60 | ||||
-rw-r--r-- | lib/eldap/doc/src/notes.xml | 20 | ||||
-rw-r--r-- | lib/eldap/vsn.mk | 2 | ||||
-rw-r--r-- | lib/erl_interface/doc/src/notes.xml | 33 | ||||
-rw-r--r-- | lib/erl_interface/vsn.mk | 2 | ||||
-rw-r--r-- | lib/public_key/doc/src/notes.xml | 20 | ||||
-rw-r--r-- | lib/public_key/vsn.mk | 2 | ||||
-rw-r--r-- | lib/ssl/doc/src/notes.xml | 34 | ||||
-rw-r--r-- | lib/ssl/src/tls_connection.erl | 16 | ||||
-rw-r--r-- | lib/ssl/test/inet_crypto_dist.erl | 940 | ||||
-rw-r--r-- | lib/ssl/test/ssl_dist_bench_SUITE.erl | 14 | ||||
-rw-r--r-- | lib/ssl/vsn.mk | 2 | ||||
-rw-r--r-- | lib/stdlib/doc/src/notes.xml | 18 | ||||
-rw-r--r-- | lib/stdlib/src/stdlib.appup.src | 6 | ||||
-rw-r--r-- | lib/stdlib/vsn.mk | 2 | ||||
-rw-r--r-- | make/otp_patch_solve_forward_merge_version | 2 | ||||
-rw-r--r-- | otp_versions.table | 1 |
19 files changed, 806 insertions, 392 deletions
diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index da470b51ec..f0f97a6775 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -31,6 +31,24 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 10.3.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed more bugs in <c>process_info(reductions)</c> + causing it to sometimes behave non-monotonic. That is, a + subsequent call toward the same process could return a + lower reduction value.</p> + <p> + Own Id: OTP-15793 Aux Id: ERIERL-337, OTP-15709 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.3.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/doc/src/ct_hooks.xml b/lib/common_test/doc/src/ct_hooks.xml index ff9969ebc3..100be85cac 100644 --- a/lib/common_test/doc/src/ct_hooks.xml +++ b/lib/common_test/doc/src/ct_hooks.xml @@ -109,7 +109,7 @@ </func> <func> - <name since="OTP @OTP-14746@">Module:post_groups(SuiteName, GroupDefs) -> NewGroupDefs</name> + <name since="OTP 21.3.8">Module:post_groups(SuiteName, GroupDefs) -> NewGroupDefs</name> <fsummary>Called after groups/0.</fsummary> <type> <v>SuiteName = atom()</v> @@ -165,7 +165,7 @@ </func> <func> - <name since="OTP @OTP-14746@">Module:post_all(SuiteName, Return, GroupDefs) -> NewReturn</name> + <name since="OTP 21.3.8">Module:post_all(SuiteName, Return, GroupDefs) -> NewReturn</name> <fsummary>Called after all/0.</fsummary> <type> <v>SuiteName = atom()</v> @@ -787,5 +787,3 @@ </funcs> </erlref> - - diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index a68cc3cca7..9456b6dbba 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -33,6 +33,66 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.17.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The test result when a hook function fails is in general + the same as if the function that the hook is associated + with fails. For example, if <c>post_init_per_testcase</c> + fails the result is that the test case is skipped, as is + the case when <c>init_per_testcase</c> fails.This, + however, was earlier not true for timetrap timeouts or + other error situations where the process running the hook + function was killed. This is now corrected, so the error + handling should be the same no matter how the hook + function fails.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15717 Aux Id: ERIERL-334 </p> + </item> + <item> + <p> + In some rare cases, when two common_test nodes used the + same log directory, a timing problem could occur which + caused common_test to crash because it's log cache file + was unexpectedly empty. This is now corrected.</p> + <p> + Own Id: OTP-15758 Aux Id: ERIERL-342 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Two new common_test hook functions are introduced:</p> + <p> + <c>post_groups/2</c>, which is called after + <c>Suite:groups/0</c><br/> <c>post_all/3</c>, which is + called after <c>Suite:all/0</c></p> + <p> + These functions allow modifying the return values from + the <c>groups/0</c> and <c>all/0</c> functions, + respectively.</p> + <p> + A new term, <c>{testcase,TestCase,RepeatProperties}</c> + is now also allowed in the return from <c>all/0</c>. This + can be used for repeating a single test case a specific + number of times, or until it fails or succeeds once.</p> + <p> + Own Id: OTP-14746 Aux Id: ERIERL-143 </p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.17.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/eldap/doc/src/notes.xml b/lib/eldap/doc/src/notes.xml index bf9358c4d1..975a25d7a8 100644 --- a/lib/eldap/doc/src/notes.xml +++ b/lib/eldap/doc/src/notes.xml @@ -31,6 +31,26 @@ </header> <p>This document describes the changes made to the Eldap application.</p> +<section><title>Eldap 1.2.7</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Back port of bug fix ERL-893 from OTP-22 and document + enhancements that will solve dialyzer warnings for users + of the ssl application.</p> + <p> + This change also affects public_key, eldap (and inet + doc).</p> + <p> + Own Id: OTP-15785 Aux Id: ERL-929, ERL-893, PR-2215 </p> + </item> + </list> + </section> + +</section> + <section><title>Eldap 1.2.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/eldap/vsn.mk b/lib/eldap/vsn.mk index 6d541e4689..7f03fbd1b2 100644 --- a/lib/eldap/vsn.mk +++ b/lib/eldap/vsn.mk @@ -1 +1 @@ -ELDAP_VSN = 1.2.6 +ELDAP_VSN = 1.2.7 diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index fc6a1bb548..b331fd81b6 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -31,6 +31,39 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.11.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + <c>erl_interface</c>/<c>ei</c> refused to use node names + with an alive name (the part of the node name preceding + the @ sign) longer than 63 characters and a host name + longer than 64 characters. The total amount of characters + allowed in a node name (alivename@hostname) was thus + limited to 128 characters. These limits applied both to + the own node name as well as node names of other nodes. + Ordinary Erlang nodes limit the node name length to 256 + characters, which meant that you could not communicate + with certain Erlang nodes due to their node name used.</p> + <p> + <c>erl_interface</c>/<c>ei</c> now allow the total amount + of characters in a node name to be up to 256 characters. + These characters may be distributed between alive name + and host name in whatever way needed. That is, the + maximum amount of characters in the alive name may be 254 + and the maximum amount of characters in the host name may + be 254, but in total the node name must not exceed 256 + characters.</p> + <p> + Own Id: OTP-15781 Aux Id: ERIERL-356 </p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.11.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 5e63f75ab5..4e31b3835d 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1,2 +1,2 @@ -EI_VSN = 3.11.2 +EI_VSN = 3.11.3 ERL_INTERFACE_VSN = $(EI_VSN) diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index f6bc0dc797..d83dd24f41 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -35,6 +35,26 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 1.6.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Back port of bug fix ERL-893 from OTP-22 and document + enhancements that will solve dialyzer warnings for users + of the ssl application.</p> + <p> + This change also affects public_key, eldap (and inet + doc).</p> + <p> + Own Id: OTP-15785 Aux Id: ERL-929, ERL-893, PR-2215 </p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 1.6.5</title> <section><title>Improvements and New Features</title> diff --git a/lib/public_key/vsn.mk b/lib/public_key/vsn.mk index 11c06fb158..c68806d856 100644 --- a/lib/public_key/vsn.mk +++ b/lib/public_key/vsn.mk @@ -1 +1 @@ -PUBLIC_KEY_VSN = 1.6.5 +PUBLIC_KEY_VSN = 1.6.6 diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index f0231da2ad..7947049a04 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -27,6 +27,40 @@ </header> <p>This document describes the changes made to the SSL application.</p> +<section><title>SSL 9.2.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Missing check of size of user_data_buffer made internal + socket behave as an active socket instead of active N. + This could cause memory problems.</p> + <p> + Own Id: OTP-15802 Aux Id: ERL-934 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Back port of bug fix ERL-893 from OTP-22 and document + enhancements that will solve dialyzer warnings for users + of the ssl application.</p> + <p> + This change also affects public_key, eldap (and inet + doc).</p> + <p> + Own Id: OTP-15785 Aux Id: ERL-929, ERL-893, PR-2215 </p> + </item> + </list> + </section> + +</section> + <section><title>SSL 9.2.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index 872a557e67..a05858221a 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -171,21 +171,19 @@ next_record(#state{protocol_buffers = connection_states = ConnectionStates, ssl_options = #ssl_options{padding_check = Check}} = State) -> next_record(State, CipherTexts, ConnectionStates, Check); -next_record(#state{user_data_buffer = {_,0,_}, - protocol_buffers = #protocol_buffers{tls_cipher_texts = []}, - protocol_specific = #{active_n_toggle := true, - active_n := N} = ProtocolSpec, +next_record(#state{protocol_buffers = #protocol_buffers{tls_cipher_texts = []}, + protocol_specific = #{active_n_toggle := true, active_n := N} = ProtocolSpec, static_env = #static_env{socket = Socket, close_tag = CloseTag, transport_cb = Transport} - } = State) -> + } = State) -> case tls_socket:setopts(Transport, Socket, [{active, N}]) of ok -> - {no_record, State#state{protocol_specific = ProtocolSpec#{active_n_toggle => false}}}; + {no_record, State#state{protocol_specific = ProtocolSpec#{active_n_toggle => false}}}; _ -> - self() ! {CloseTag, Socket}, - {no_record, State} - end; + self() ! {CloseTag, Socket}, + {no_record, State} + end; next_record(State) -> {no_record, State}. diff --git a/lib/ssl/test/inet_crypto_dist.erl b/lib/ssl/test/inet_crypto_dist.erl index 5aafaac983..63c19d9438 100644 --- a/lib/ssl/test/inet_crypto_dist.erl +++ b/lib/ssl/test/inet_crypto_dist.erl @@ -29,14 +29,8 @@ -define(DRIVER, inet_tcp). -define(FAMILY, inet). --define(PROTOCOL, inet_crypto_dist_v1). --define(DEFAULT_BLOCK_CRYPTO, aes_128_gcm). --define(DEFAULT_HASH_ALGORITHM, sha256). --define(DEFAULT_REKEY_INTERVAL, 32768). - -export([listen/1, accept/1, accept_connection/5, setup/5, close/1, select/1, is_node_name/1]). --export([is_supported/0]). %% Generalized dist API, for sibling IPv6 module inet6_crypto_dist -export([gen_listen/2, gen_accept/2, gen_accept_connection/6, @@ -52,20 +46,136 @@ -include_lib("kernel/include/dist.hrl"). -include_lib("kernel/include/dist_util.hrl"). -%% Test if crypto has got enough capabilities for this module to run +-define(PACKET_SIZE, 65536). +-define(BUFFER_SIZE, (?PACKET_SIZE bsl 4)). + +%% ------------------------------------------------------------------------- + +-record(params, + {socket, + dist_handle, + hmac_algorithm = sha256, + aead_cipher = aes_gcm, + rekey_key, + iv = 12, + key = 16, + tag_len = 16, + rekey_interval = 262144 + }). + +params(Socket) -> + #params{socket = Socket}. + + +-record(key_pair, + {type = ecdh, + %% The curve choice greatly affects setup time, + %% we really want an Edwards curve but that would + %% require a very new openssl version. + %% Twisted brainpool curves (*t1) are faster than + %% non-twisted (*r1), 256 is much faster than 384, + %% and so on... +%%% params = brainpoolP384t1, + params = brainpoolP256t1, + public, + private}). + +-define(KEY_PAIR_LIFE_TIME, 3600000). % 1 hour +-define(KEY_PAIR_LIFE_COUNT, 256). % Number of connection setups + + +%% ------------------------------------------------------------------------- +%% Keep the node's public/private key pair in the process state +%% of a key pair server linked to the acceptor process. +%% Create the key pair the first time it is needed +%% so crypto gets time to start first. %% -is_supported() -> - try {crypto:cipher_info(?DEFAULT_BLOCK_CRYPTO), - crypto:hash_info(?DEFAULT_HASH_ALGORITHM)} - of - {#{block_size := _, iv_length := _, key_length := _}, - #{size := _}} -> - true - catch - error:undef -> - false + +start_key_pair_server() -> + monitor_dist_proc( + spawn_link( + fun () -> + register(?MODULE, self()), + key_pair_server() + end)). + +key_pair_server() -> + key_pair_server(undefined, undefined, undefined). +%% +key_pair_server(KeyPair) -> + key_pair_server( + KeyPair, + erlang:start_timer(?KEY_PAIR_LIFE_TIME, self(), discard), + ?KEY_PAIR_LIFE_COUNT). +%% +key_pair_server(_KeyPair, Timer, 0) -> + cancel_timer(Timer), + key_pair_server(); +key_pair_server(KeyPair, Timer, Count) -> + receive + {Pid, Tag, get_key_pair} -> + case KeyPair of + undefined -> + KeyPair_1 = generate_key_pair(), + Pid ! {Tag, KeyPair_1}, + key_pair_server(KeyPair_1); + #key_pair{} -> + Pid ! {Tag, KeyPair}, + key_pair_server(KeyPair, Timer, Count - 1) + end; + {Pid, Tag, get_new_key_pair} -> + cancel_timer(Timer), + KeyPair_1 = generate_key_pair(), + Pid ! {Tag, KeyPair_1}, + key_pair_server(KeyPair_1); + {timeout, Timer, discard} when is_reference(Timer) -> + key_pair_server() + end. + +generate_key_pair() -> + #key_pair{type = Type, params = Params} = #key_pair{}, + {Public, Private} = + crypto:generate_key(Type, Params), + #key_pair{public = Public, private = Private}. + +cancel_timer(undefined) -> + ok; +cancel_timer(Timer) -> + case erlang:cancel_timer(Timer) of + false -> + receive + {timeout, Timer, _} -> ok + end; + _RemainingTime -> + ok + end. + +get_key_pair() -> + call_key_pair_server(get_key_pair). + +get_new_key_pair() -> + call_key_pair_server(get_new_key_pair). + +call_key_pair_server(Request) -> + Pid = whereis(?MODULE), + Ref = erlang:monitor(process, Pid), + Pid ! {self(), Ref, Request}, + receive + {Ref, Reply} -> + erlang:demonitor(Ref, [flush]), + Reply; + {'DOWN', Ref, process, Pid, Reason} -> + error(Reason) end. +compute_shared_secret( + #key_pair{ + type = PublicKeyType, + params = PublicKeyParams, + private = PrivKey}, PubKey) -> + %% + crypto:compute_key(PublicKeyType, PubKey, PrivKey, PublicKeyParams). + %% ------------------------------------------------------------------------- %% Erlang distribution plugin structure explained to myself %% ------- @@ -80,7 +190,7 @@ is_supported() -> %% is not one or two processes, but one port - a gen_tcp socket %% %% When the VM is started with the argument "-proto_dist inet_crypto" -%% net_kernel registers the module inet_crypto_dist as distribution +%% net_kernel registers the module inet_crypto_dist acli,oams distribution %% module. net_kernel calls listen/1 to create a listen socket %% and then accept/1 with the listen socket as argument to spawn %% the Acceptor process, which is linked to net_kernel. Apparently @@ -159,6 +269,12 @@ is_supported() -> %% terminate with reason 'normal'. %% ------------------------------------------------------------------------- +-compile({inline, [socket_options/0]}). +socket_options() -> + [binary, {active, false}, {packet, 2}, {nodelay, true}, + {sndbuf, ?BUFFER_SIZE}, {recbuf, ?BUFFER_SIZE}, + {buffer, ?BUFFER_SIZE}]. + %% ------------------------------------------------------------------------- %% select/1 is called by net_kernel to ask if this distribution protocol %% is willing to handle Node @@ -195,7 +311,7 @@ listen(Name) -> gen_listen(Name, Driver) -> case inet_tcp_dist:gen_listen(Driver, Name) of {ok, {Socket, Address, Creation}} -> - inet:setopts(Socket, [binary, {nodelay, true}]), + inet:setopts(Socket, socket_options()), {ok, {Socket, Address#net_address{protocol = ?DIST_PROTO}, Creation}}; Other -> @@ -217,24 +333,24 @@ gen_accept(Listen, Driver) -> %% %% Spawn Acceptor process %% - Config = config(), monitor_dist_proc( spawn_opt( fun () -> - accept_loop(Listen, Driver, NetKernel, Config) + start_key_pair_server(), + accept_loop(Listen, Driver, NetKernel) end, [link, {priority, max}])). -accept_loop(Listen, Driver, NetKernel, Config) -> - case Driver:accept(Listen) of +accept_loop(Listen, Driver, NetKernel) -> + case Driver:accept(trace(Listen)) of {ok, Socket} -> wait_for_code_server(), Timeout = net_kernel:connecttime(), - DistCtrl = start_dist_ctrl(Socket, Config, Timeout), + DistCtrl = start_dist_ctrl(trace(Socket), Timeout), %% DistCtrl is a "socket" NetKernel ! - {accept, - self(), DistCtrl, Driver:family(), ?DIST_PROTO}, + trace({accept, + self(), DistCtrl, Driver:family(), ?DIST_PROTO}), receive {NetKernel, controller, Controller} -> call_dist_ctrl(DistCtrl, {controller, Controller, self()}), @@ -242,7 +358,7 @@ accept_loop(Listen, Driver, NetKernel, Config) -> {NetKernel, unsupported_protocol} -> exit(unsupported_protocol) end, - accept_loop(Listen, Driver, NetKernel, Config); + accept_loop(Listen, Driver, NetKernel); AcceptError -> exit({accept, AcceptError}) end. @@ -292,12 +408,13 @@ gen_accept_connection( fun() -> do_accept( Acceptor, DistCtrl, - MyNode, Allowed, SetupTime, Driver, NetKernel) + trace(MyNode), Allowed, SetupTime, Driver, NetKernel) end, [link, {priority, max}])). do_accept( Acceptor, DistCtrl, MyNode, Allowed, SetupTime, Driver, NetKernel) -> + %% receive {Acceptor, controller, Socket} -> Timer = dist_util:start_timer(SetupTime), @@ -337,40 +454,42 @@ gen_setup(Node, Type, MyNode, LongOrShortNames, SetupTime, Driver) -> -spec setup_fun(_,_,_,_,_,_,_) -> fun(() -> no_return()). setup_fun( Node, Type, MyNode, LongOrShortNames, SetupTime, Driver, NetKernel) -> + %% fun() -> do_setup( - Node, Type, MyNode, LongOrShortNames, SetupTime, + trace(Node), Type, MyNode, LongOrShortNames, SetupTime, Driver, NetKernel) end. -spec do_setup(_,_,_,_,_,_,_) -> no_return(). do_setup( Node, Type, MyNode, LongOrShortNames, SetupTime, Driver, NetKernel) -> + %% {Name, Address} = split_node(Driver, Node, LongOrShortNames), ErlEpmd = net_kernel:epmd_module(), {ARMod, ARFun} = get_address_resolver(ErlEpmd, Driver), Timer = trace(dist_util:start_timer(SetupTime)), case ARMod:ARFun(Name, Address, Driver:family()) of - {ok, Ip, TcpPort, Version} -> - do_setup_connect( - Node, Type, MyNode, Timer, Driver, NetKernel, - Ip, TcpPort, Version); + {ok, Ip, TcpPort, Version} -> + do_setup_connect( + Node, Type, MyNode, Timer, Driver, NetKernel, + Ip, TcpPort, Version); {ok, Ip} -> case ErlEpmd:port_please(Name, Ip) of {port, TcpPort, Version} -> do_setup_connect( Node, Type, MyNode, Timer, Driver, NetKernel, - Ip, TcpPort, Version); + Ip, TcpPort, trace(Version)); Other -> - ?shutdown2( - Node, - trace( - {port_please_failed, ErlEpmd, Name, Ip, Other})) + _ = trace( + {ErlEpmd, port_please, [Name, Ip], Other}), + ?shutdown(Node) end; Other -> - ?shutdown2( - Node, - trace({getaddr_failed, Driver, Address, Other})) + _ = trace( + {ARMod, ARFun, [Name, Address, Driver:family()], + Other}), + ?shutdown(Node) end. -spec do_setup_connect(_,_,_,_,_,_,_,_,_) -> no_return(). @@ -379,15 +498,15 @@ do_setup_connect( Node, Type, MyNode, Timer, Driver, NetKernel, Ip, TcpPort, Version) -> dist_util:reset_timer(Timer), - ConnectOpts = - trace( - connect_options( - [binary, {active, false}, {packet, 2}, {nodelay, true}])), + ConnectOpts = trace(connect_options(socket_options())), case Driver:connect(Ip, TcpPort, ConnectOpts) of {ok, Socket} -> - Config = config(), DistCtrl = - start_dist_ctrl(Socket, Config, net_kernel:connecttime()), + try start_dist_ctrl(Socket, net_kernel:connecttime()) + catch error : {dist_ctrl, _} = DistCtrlError -> + _ = trace(DistCtrlError), + ?shutdown(Node) + end, %% DistCtrl is a "socket" HSData = hs_data_common( @@ -401,8 +520,10 @@ do_setup_connect( request_type = Type}, dist_util:handshake_we_started(trace(HSData_1)); ConnectError -> - ?shutdown2(Node, - trace({connect_failed, Ip, TcpPort, ConnectError})) + _ = trace( + {Driver, connect, [Ip, TcpPort, ConnectOpts], + ConnectError}), + ?shutdown(Node) end. %% ------------------------------------------------------------------------- @@ -412,7 +533,7 @@ close(Socket) -> gen_close(Socket, ?DRIVER). gen_close(Socket, Driver) -> - trace(Driver:close(Socket)). + Driver:close(trace(Socket)). %% ------------------------------------------------------------------------- @@ -428,17 +549,23 @@ hs_data_common(NetKernel, MyNode, DistCtrl, Timer, Socket, Family) -> socket = DistCtrl, timer = Timer, %% - f_send = + f_send = % -> ok | {error, closed}=>?shutdown() fun (S, Packet) when S =:= DistCtrl -> - call_dist_ctrl(S, {send, Packet}) + try call_dist_ctrl(S, {send, Packet}) + catch error : {dist_ctrl, Reason} -> + _ = trace(Reason), + {error, closed} + end end, - f_recv = + f_recv = % -> {ok, List} | Other=>?shutdown() fun (S, 0, infinity) when S =:= DistCtrl -> - case call_dist_ctrl(S, recv) of + try call_dist_ctrl(S, recv) of {ok, Bin} when is_binary(Bin) -> {ok, binary_to_list(Bin)}; Error -> Error + catch error : {dist_ctrl, Reason} -> + {error, trace(Reason)} end end, f_setopts_pre_nodeup = @@ -453,9 +580,9 @@ hs_data_common(NetKernel, MyNode, DistCtrl, Timer, Socket, Family) -> fun (S) when S =:= DistCtrl -> {ok, S} %% DistCtrl is the distribution port end, - f_address = + f_address = % -> #net_address{} | ?shutdown() fun (S, Node) when S =:= DistCtrl -> - case call_dist_ctrl(S, peername) of + try call_dist_ctrl(S, peername) of {ok, Address} -> case dist_util:split_node(Node) of {node, _, Host} -> @@ -465,13 +592,23 @@ hs_data_common(NetKernel, MyNode, DistCtrl, Timer, Socket, Family) -> protocol = ?DIST_PROTO, family = Family}; _ -> - {error, no_node} - end + ?shutdown(Node) + end; + Error -> + _ = trace(Error), + ?shutdown(Node) + catch error : {dist_ctrl, Reason} -> + _ = trace(Reason), + ?shutdown(Node) end end, - f_handshake_complete = - fun (S, _Node, DistHandle) when S =:= DistCtrl -> - call_dist_ctrl(S, {handshake_complete, DistHandle}) + f_handshake_complete = % -> ok | ?shutdown() + fun (S, Node, DistHandle) when S =:= DistCtrl -> + try call_dist_ctrl(S, {handshake_complete, DistHandle}) + catch error : {dist_ctrl, Reason} -> + _ = trace(Reason), + ?shutdown(Node) + end end, %% %% mf_tick/1, mf_getstat/1, mf_setopts/2 and mf_getopts/2 @@ -481,7 +618,7 @@ hs_data_common(NetKernel, MyNode, DistCtrl, Timer, Socket, Family) -> fun (S) when S =:= DistCtrl -> S ! dist_tick end, - mf_getstat = + mf_getstat = % -> {ok, RecvCnt, SendCnt, SendPend} | Other=>ignore_it fun (S) when S =:= DistCtrl -> case inet:getstat(Socket, [recv_cnt, send_cnt, send_pend]) @@ -489,7 +626,7 @@ hs_data_common(NetKernel, MyNode, DistCtrl, Timer, Socket, Family) -> {ok, Stat} -> split_stat(Stat, 0, 0, 0); Error -> - Error + trace(Error) end end, mf_setopts = @@ -596,25 +733,6 @@ nodelay() -> {nodelay, true} end. -config() -> - case init:get_argument(?DIST_NAME) of - error -> - error({missing_argument, ?DIST_NAME}); - {ok, [[String]]} -> - {ok, Tokens, _} = erl_scan:string(String ++ "."), - case erl_parse:parse_term(Tokens) of - {ok, #{secret := Secret} = Config} - when is_binary(Secret); is_list(Secret) -> - Config; - {ok, #{} = Config} -> - error({missing_secret, [{?DIST_NAME,Config}]}); - _ -> - error({bad_argument_value, [{?DIST_NAME,String}]}) - end; - {ok, Value} -> - error({malformed_argument, [{?DIST_NAME,Value}]}) - end. - %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% %% The DistCtrl process(es). @@ -625,40 +743,36 @@ config() -> %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% XXX Missing to "productified": -%%% * Cryptoanalysis by experts -%%% * Proof of usefulness -%%% * Unifying exit reasons using a death_row() function -%%% * Verification (and rejection) of other end's crypto parameters -%%% * OTP:ification (proc_lib?) -%%% * An application to belong to (crypto|kernel?) -%%% * Secret on file (cookie as default?), parameter handling -%%% * Restart and/or code reload policy +%%% * Cryptoanalysis by experts, this is crypto amateur work. +%%% * Is it useful over inet_tls_dist; i.e to not have to bother +%%% with certificates but instead manage a secret cluster cookie? +%%% * An application to belong to (kernel) +%%% * Restart and/or code reload policy (not needed in kernel) +%%% * Fitting into the epmd/Erlang distro protocol version framework +%%% (something needs to be created for multiple protocols, epmd, +%%% multiple address families, fallback to previous version, etc) -%% Debug client and server -test_config() -> - #{secret => <<"Secret Cluster Password 123456">>}. +%% Debug client and server test_server() -> - {ok, Listen} = gen_tcp:listen(0, [{packet, 2}, {active, false}, binary]), + {ok, Listen} = gen_tcp:listen(0, socket_options()), {ok, Port} = inet:port(Listen), io:format(?MODULE_STRING":test_client(~w).~n", [Port]), {ok, Socket} = gen_tcp:accept(Listen), test(Socket). test_client(Port) -> - {ok, Socket} = - gen_tcp:connect( - localhost, Port, [{packet, 2}, {active, false}, binary]), + {ok, Socket} = gen_tcp:connect(localhost, Port, socket_options()), test(Socket). test(Socket) -> - start_dist_ctrl(Socket, test_config(), 10000). + start_dist_ctrl(Socket, 10000). %% ------------------------------------------------------------------------- -start_dist_ctrl(Socket, Config, Timeout) -> - Protocol = ?PROTOCOL, +start_dist_ctrl(Socket, Timeout) -> + Secret = atom_to_binary(auth:get_cookie(), latin1), Controller = self(), Server = monitor_dist_proc( @@ -667,9 +781,9 @@ start_dist_ctrl(Socket, Config, Timeout) -> receive {?MODULE, From, start} -> {SendParams, RecvParams} = - init(Socket, Config, Protocol), + init(Socket, Secret), reply(From, self()), - handshake(SendParams, 0, RecvParams, 0, Controller) + handshake(SendParams, 1, RecvParams, 1, Controller) end end, [link, @@ -691,12 +805,13 @@ call_dist_ctrl(Server, Msg, Timeout) -> erlang:demonitor(Ref, [flush]), Res; {'DOWN', Ref, process, Server, Reason} -> - exit({?PROTOCOL, Reason}) - after Timeout -> - exit(Server, timeout), + error({dist_ctrl, Reason}) + after Timeout -> % Timeout < infinity is only used by start_dist_ctrl/2 receive {'DOWN', Ref, process, Server, _} -> - exit({?PROTOCOL, timeout}) + receive {Ref, _} -> ok after 0 -> ok end, + error({dist_ctrl, timeout}) + %% Server will be killed by link end end. @@ -706,21 +821,9 @@ reply({Ref, Pid}, Msg) -> %% ------------------------------------------------------------------------- --record(params, - {protocol, % Encryption protocol tag - socket, - dist_handle, - hash_algorithm, - block_crypto, - rekey_interval, - iv, - key, - tag_len}). - --define(TCP_ACTIVE, 64). --define(CHUNK_SIZE, (65536 - 512)). -%% The start chunk starts with zeros, so it seems logical to not have -%% a chunk type with value 0 +-define(TCP_ACTIVE, 16). +-define(CHUNK_SIZE, (?PACKET_SIZE - 512)). + -define(HANDSHAKE_CHUNK, 1). -define(DATA_CHUNK, 2). -define(TICK_CHUNK, 3). @@ -739,170 +842,189 @@ reply({Ref, Pid}, Msg) -> %% and waits for the other end's start message. So if the send %% blocks we have a deadlock. %% -%% The init message is unencrypted and contains the block cipher and hash -%% algorithms the sender will use, the IV and a key salt. Both sides' -%% key salt is used with the mutual secret as input to the hash algorithm -%% to create different encryption/decryption keys for both directions. +%% The init + start sequence tries to implement Password Encrypted +%% Key Exchange using a node public/private key pair and the +%% shared secret (the Cookie) to create session encryption keys +%% that can not be re-created if the shared secret is compromized, +%% which should create forward secrecy. You need both nodes' +%% key pairs and the shared secret to decrypt the traffic +%% between the nodes. +%% +%% All exchanged messages uses {packet, 2} i.e 16 bit size header. +%% +%% The init message contains a random number and encrypted: the public key +%% and two random numbers. The encryption is done with Key and IV hashed +%% from the unencrypted random number and the shared secret. +%% +%% The other node's public key is used with the own node's private +%% key to create a shared key that is hashed with one of the encrypted +%% random numbers from each side to create Key and IV for the session. %% -%% The start message is the first encrypted message and contains just -%% encrypted zeros the width of the key, with the header of the init -%% message as AAD data. Successfully decrypting this message -%% verifies that we have an encrypted channel. +%% The start message contains the two encrypted random numbers +%% this time encrypted with the session keys for verification +%% by the other side, plus the rekey interval. The rekey interval +%% is just there to get an early check for if the other side's +%% maximum rekey interal is acceptable, it is just an embryo +%% of some better check. Any side may rekey earlier but if the +%% rekey interval is exceeded the connection fails. %% %% Subsequent encrypted messages has the sequence number and the length -%% of the message as AAD data. These messages has got a message type -%% differentiating data from ticks. Ticks have a random size in an -%% attempt to make them less obvious to spot. +%% of the message as AAD data, and an incrementing IV. These messages +%% has got a message type that differentiates data from ticks and rekeys. +%% Ticks have a random size in an attempt to make them less obvious to spot. %% -%% The only reaction to errors is to crash noisily wich will bring +%% Rekeying is done by the sender that creates a new key pair and +%% a new shared secret from the other end's public key and with +%% this and the current key and iv hashes a new key and iv. +%% The new public key is sent to the other end that uses it +%% and its old private key to create the same new shared +%% secret and from that a new key and iv. +%% So the receiver keeps its private key, and the sender keeps +%% the receivers public key for the connection's life time. +%% While the sender generates a new key pair at every rekey, +%% which changes the shared secret at every rekey. +%% +%% The only reaction to errors is to crash noisily (?) wich will bring %% down the connection and hopefully produce something useful %% in the local log, but all the other end sees is a closed connection. %% ------------------------------------------------------------------------- -init(Socket, Config, Protocol) -> - Secret = maps:get(secret, Config), - HashAlgorithm = - maps:get(hash_algorithm, Config, ?DEFAULT_HASH_ALGORITHM), - BlockCrypto = - maps:get(block_crypto, Config, ?DEFAULT_BLOCK_CRYPTO), - RekeyInterval = - maps:get(rekey_interval, Config, ?DEFAULT_REKEY_INTERVAL), +init(Socket, Secret) -> + #key_pair{public = PubKey} = KeyPair = get_key_pair(), + Params = params(Socket), + {R2, R3, Msg} = init_msg(Params, PubKey, Secret), + ok = gen_tcp:send(Socket, Msg), + init_recv(Params, Secret, KeyPair, R2, R3). + +init_recv( + #params{socket = Socket, iv = IVLen} = Params, Secret, KeyPair, R2, R3) -> %% - SendParams = - init_params( - Socket, Protocol, HashAlgorithm, BlockCrypto, RekeyInterval), - send_init(SendParams, Secret). + {ok, InitMsg} = gen_tcp:recv(Socket, 0), + IVSaltLen = IVLen - 6, + try + case init_msg(Params, Secret, KeyPair, R2, R3, InitMsg) of + {#params{iv = <<IV2ASalt:IVSaltLen/binary, IV2ANo:48>>} = + SendParams, + RecvParams, SendStartMsg} -> + ok = gen_tcp:send(Socket, SendStartMsg), + {ok, RecvStartMsg} = gen_tcp:recv(Socket, 0), + #params{ + iv = <<IV2BSalt:IVSaltLen/binary, IV2BNo:48>>} = + RecvParams_1 = + start_msg(RecvParams, R2, R3, RecvStartMsg), + {SendParams#params{iv = {IV2ASalt, IV2ANo}}, + RecvParams_1#params{iv = {IV2BSalt, IV2BNo}}} + end + catch + error : Reason : Stacktrace-> + _ = trace({Reason, Stacktrace}), + exit(connection_closed) + end. -send_init( + + +init_msg( #params{ - protocol = Protocol, - socket = Socket, - block_crypto = BlockCrypto, - iv = IVLen, + hmac_algorithm = HmacAlgo, + aead_cipher = AeadCipher, key = KeyLen, - hash_algorithm = HashAlgorithm} = SendParams, - Secret) -> + iv = IVLen, + tag_len = TagLen}, PubKeyA, Secret) -> %% - ProtocolString = atom_to_binary(Protocol, utf8), - BlockCryptoString = atom_to_binary(BlockCrypto, utf8), - HashAlgorithmString = atom_to_binary(HashAlgorithm, utf8), - SendHeader = - <<ProtocolString/binary, 0, - HashAlgorithmString/binary, 0, - BlockCryptoString/binary, 0>>, - <<IV:IVLen/binary, KeySalt:KeyLen/binary>> = IV_KeySalt = - crypto:strong_rand_bytes(IVLen + KeyLen), - InitPacket = [SendHeader, IV_KeySalt], - ok = gen_tcp:send(Socket, InitPacket), - recv_init(SendParams#params{iv = IV, key = KeySalt}, Secret, SendHeader). - -recv_init( + RLen = KeyLen + IVLen, + <<R1A:RLen/binary, R2A:RLen/binary, R3A:RLen/binary>> = + crypto:strong_rand_bytes(3 * RLen), + {Key1A, IV1A} = hmac_key_iv(HmacAlgo, R1A, Secret, KeyLen, IVLen), + Plaintext = [R2A, R3A, PubKeyA], + MsgLen = byte_size(R1A) + TagLen + iolist_size(Plaintext), + AAD = [<<MsgLen:32>>, R1A], + {Ciphertext, Tag} = + crypto:block_encrypt(AeadCipher, Key1A, IV1A, {AAD, Plaintext, TagLen}), + Msg = [R1A, Tag, Ciphertext], + {R2A, R3A, Msg}. +%% +init_msg( #params{ - socket = Socket, - hash_algorithm = SendHashAlgorithm, - key = SendKeySalt} = SendParams, Secret, SendHeader) -> + hmac_algorithm = HmacAlgo, + aead_cipher = AeadCipher, + key = KeyLen, + iv = IVLen, + tag_len = TagLen, + rekey_interval = RekeyInterval} = Params, + Secret, KeyPair, R2A, R3A, Msg) -> %% - {ok, InitPacket} = gen_tcp:recv(Socket, 0), - [ProtocolString, Rest_1] = binary:split(InitPacket, <<0>>), - Protocol = binary_to_existing_atom(ProtocolString, utf8), - case Protocol of - ?PROTOCOL -> - [HashAlgorithmString, Rest_2] = binary:split(Rest_1, <<0>>), - HashAlgorithm = binary_to_existing_atom(HashAlgorithmString, utf8), - [BlockCryptoString, Rest_3] = binary:split(Rest_2, <<0>>), - BlockCrypto = binary_to_existing_atom(BlockCryptoString, utf8), - #params{ - hash_algorithm = RecvHashAlgorithm, - iv = RecvIVLen, - key = RecvKeyLen} = RecvParams = - init_params( - Socket, Protocol, HashAlgorithm, BlockCrypto, undefined), - <<RecvIV:RecvIVLen/binary, - RecvKeySalt:RecvKeyLen/binary>> = Rest_3, - SendKey = - hash_key(SendHashAlgorithm, SendKeySalt, [RecvKeySalt, Secret]), - RecvKey = - hash_key(RecvHashAlgorithm, RecvKeySalt, [SendKeySalt, Secret]), - SendParams_1 = SendParams#params{key = SendKey}, - RecvParams_1 = RecvParams#params{iv = RecvIV, key = RecvKey}, - RecvHeaderLen = byte_size(InitPacket) - RecvIVLen - RecvKeyLen, - <<RecvHeader:RecvHeaderLen/binary, _/binary>> = InitPacket, - send_start(SendParams_1, SendHeader), - RecvRekeyInterval = recv_start(RecvParams_1, RecvHeader), - {SendParams_1, - RecvParams_1#params{rekey_interval = RecvRekeyInterval}} + RLen = KeyLen + IVLen, + case Msg of + <<R1B:RLen/binary, Tag:TagLen/binary, Ciphertext/binary>> -> + {Key1B, IV1B} = hmac_key_iv(HmacAlgo, R1B, Secret, KeyLen, IVLen), + MsgLen = byte_size(Msg), + AAD = [<<MsgLen:32>>, R1B], + case + crypto:block_decrypt( + AeadCipher, Key1B, IV1B, {AAD, Ciphertext, Tag}) + of + <<R2B:RLen/binary, R3B:RLen/binary, PubKeyB/binary>> -> + SharedSecret = compute_shared_secret(KeyPair, PubKeyB), + %% + {Key2A, IV2A} = + hmac_key_iv( + HmacAlgo, SharedSecret, [R2A, R3B], KeyLen, IVLen), + SendParams = + Params#params{ + rekey_key = PubKeyB, + key = Key2A, iv = IV2A}, + %% + StartCleartext = [R2B, R3B, <<RekeyInterval:32>>], + StartMsgLen = TagLen + iolist_size(StartCleartext), + StartAAD = <<StartMsgLen:32>>, + {StartCiphertext, StartTag} = + crypto:block_encrypt( + AeadCipher, Key2A, IV2A, + {StartAAD, StartCleartext, TagLen}), + StartMsg = [StartTag, StartCiphertext], + %% + {Key2B, IV2B} = + hmac_key_iv( + HmacAlgo, SharedSecret, [R2B, R3A], KeyLen, IVLen), + RecvParams = + Params#params{ + rekey_key = KeyPair, + key = Key2B, iv = IV2B}, + %% + {SendParams, RecvParams, StartMsg} + end end. -send_start( +start_msg( #params{ - socket = Socket, - block_crypto = BlockCrypto, - rekey_interval= RekeyInterval, - iv = IV, - key = Key, - tag_len = TagLen}, AAD) -> + aead_cipher = AeadCipher, + key = Key2B, + iv = IV2B, + tag_len = TagLen, + rekey_interval = RekeyIntervalA} = RecvParams, R2A, R3A, Msg) -> %% - KeyLen = byte_size(Key), - Zeros = binary:copy(<<0>>, KeyLen), - {Ciphertext, CipherTag} = - crypto:block_encrypt( - crypto_cipher_name(BlockCrypto), - Key, IV, {AAD, [Zeros, <<RekeyInterval:32>>], TagLen}), - ok = gen_tcp:send(Socket, [Ciphertext, CipherTag]). - -recv_start( - #params{ - socket = Socket, - block_crypto = BlockCrypto, - iv = IV, - key = Key, - tag_len = TagLen}, AAD) -> - {ok, Packet} = gen_tcp:recv(Socket, 0), - KeyLen = byte_size(Key), - PacketLen = KeyLen + 4, - <<Ciphertext:PacketLen/binary, CipherTag:TagLen/binary>> = Packet, - Zeros = binary:copy(<<0>>, KeyLen), - case - crypto:block_decrypt( - crypto_cipher_name(BlockCrypto), - Key, IV, {AAD, Ciphertext, CipherTag}) - of - <<Zeros:KeyLen/binary, RekeyInterval:32>> - when 1 =< RekeyInterval -> - RekeyInterval; - _ -> - error(decrypt_error) - end. - -init_params(Socket, Protocol, HashAlgorithm, BlockCrypto, RekeyInterval) -> - #{block_size := 1, - iv_length := IVLen, - key_length := KeyLen} = crypto:cipher_info(BlockCrypto), - case crypto:hash_info(HashAlgorithm) of - #{size := HashSize} when HashSize >= KeyLen -> - #params{ - socket = Socket, - protocol = Protocol, - hash_algorithm = HashAlgorithm, - block_crypto = BlockCrypto, - rekey_interval = RekeyInterval, - iv = IVLen, - key = KeyLen, - tag_len = 16} - end. - -crypto_cipher_name(BlockCrypto) -> - case BlockCrypto of - aes_128_gcm -> aes_gcm; - aes_192_gcm -> aes_gcm; - aes_256_gcm -> aes_gcm + case Msg of + <<Tag:TagLen/binary, Ciphertext/binary>> -> + KeyLen = byte_size(Key2B), + IVLen = byte_size(IV2B), + RLen = KeyLen + IVLen, + MsgLen = byte_size(Msg), + AAD = <<MsgLen:32>>, + case + crypto:block_decrypt( + AeadCipher, Key2B, IV2B, {AAD, Ciphertext, Tag}) + of + <<R2A:RLen/binary, R3A:RLen/binary, RekeyIntervalB:32>> + when RekeyIntervalA =< (RekeyIntervalB bsl 2), + RekeyIntervalB =< (RekeyIntervalA bsl 2) -> + RecvParams#params{rekey_interval = RekeyIntervalB} + end end. -hash_key(HashAlgorithm, KeySalt, OtherSalt) -> - KeyLen = byte_size(KeySalt), - <<Key:KeyLen/binary, _/binary>> = - crypto:hash(HashAlgorithm, [KeySalt, OtherSalt]), - Key. +hmac_key_iv(HmacAlgo, MacKey, Data, KeyLen, IVLen) -> + <<Key:KeyLen/binary, IV:IVLen/binary>> = + crypto:hmac(HmacAlgo, MacKey, Data, KeyLen + IVLen), + {Key, IV}. %% ------------------------------------------------------------------------- %% net_kernel distribution handshake in progress @@ -918,7 +1040,6 @@ handshake( reply(From, Result), handshake(SendParams, SendSeq, RecvParams, RecvSeq, Controller_1); {?MODULE, From, {handshake_complete, DistHandle}} -> - reply(From, ok), InputHandler = monitor_dist_proc( spawn_opt( @@ -945,25 +1066,38 @@ handshake( ok = gen_tcp:controlling_process(Socket, InputHandler), ok = erlang:dist_ctrl_input_handler(DistHandle, InputHandler), InputHandler ! DistHandle, + crypto:rand_seed_alg(crypto_cache), + reply(From, ok), process_flag(priority, normal), erlang:dist_ctrl_get_data_notification(DistHandle), - crypto:rand_seed_alg(crypto_cache), output_handler( SendParams#params{dist_handle = DistHandle}, SendSeq); %% {?MODULE, From, {send, Data}} -> - {SendParams_1, SendSeq_1} = + case encrypt_and_send_chunk( - SendParams, SendSeq, [?HANDSHAKE_CHUNK, Data]), - reply(From, ok), - handshake( - SendParams_1, SendSeq_1, RecvParams, RecvSeq, Controller); + SendParams, SendSeq, [?HANDSHAKE_CHUNK, Data]) + of + {SendParams_1, SendSeq_1, ok} -> + reply(From, ok), + handshake( + SendParams_1, SendSeq_1, RecvParams, RecvSeq, + Controller); + {_, _, Error} -> + reply(From, {error, closed}), + death_row({send, trace(Error)}) + end; {?MODULE, From, recv} -> - {RecvParams_1, RecvSeq_1, Reply} = - recv_and_decrypt_chunk(RecvParams, RecvSeq), - reply(From, Reply), - handshake( - SendParams, SendSeq, RecvParams_1, RecvSeq_1, Controller); + case recv_and_decrypt_chunk(RecvParams, RecvSeq) of + {RecvParams_1, RecvSeq_1, {ok, _} = Reply} -> + reply(From, Reply), + handshake( + SendParams, SendSeq, RecvParams_1, RecvSeq_1, + Controller); + {_, _, Error} -> + reply(From, Error), + death_row({recv, trace(Error)}) + end; {?MODULE, From, peername} -> reply(From, inet:peername(Socket)), handshake(SendParams, SendSeq, RecvParams, RecvSeq, Controller); @@ -978,10 +1112,12 @@ recv_and_decrypt_chunk(#params{socket = Socket} = RecvParams, RecvSeq) -> case decrypt_chunk(RecvParams, RecvSeq, Chunk) of <<?HANDSHAKE_CHUNK, Cleartext/binary>> -> {RecvParams, RecvSeq + 1, {ok, Cleartext}}; + OtherChunk when is_binary(OtherChunk) -> + {RecvParams, RecvSeq + 1, {error, decrypt_error}}; #params{} = RecvParams_1 -> recv_and_decrypt_chunk(RecvParams_1, 0); - _ -> - error(decrypt_error) + error -> + {RecvParams, RecvSeq, {error, decrypt_error}} end; Error -> {RecvParams, RecvSeq, Error} @@ -1001,8 +1137,9 @@ output_handler(Params, Seq) -> output_handler_data(Params, Seq); dist_tick -> output_handler_tick(Params, Seq); - _Other -> + Other -> %% Ignore + _ = trace(Other), output_handler(Params, Seq) end end. @@ -1015,14 +1152,15 @@ output_handler_data(Params, Seq) -> output_handler_data(Params, Seq); dist_tick -> output_handler_data(Params, Seq); - _Other -> + Other -> %% Ignore + _ = trace(Other), output_handler_data(Params, Seq) end after 0 -> DistHandle = Params#params.dist_handle, Q = get_data(DistHandle, empty_q()), - {Params_1, Seq_1} = output_handler_send(Params, Seq, Q, true), + {Params_1, Seq_1} = output_handler_send(Params, Seq, Q), erlang:dist_ctrl_get_data_notification(DistHandle), output_handler(Params_1, Seq_1) end. @@ -1035,39 +1173,56 @@ output_handler_tick(Params, Seq) -> output_handler_data(Params, Seq); dist_tick -> output_handler_tick(Params, Seq); - _Other -> + Other -> %% Ignore + _ = trace(Other), output_handler_tick(Params, Seq) end after 0 -> - TickSize = 8 + rand:uniform(56), + TickSize = 7 + rand:uniform(56), TickData = binary:copy(<<0>>, TickSize), - {Params_1, Seq_1} = - encrypt_and_send_chunk(Params, Seq, [?TICK_CHUNK, TickData]), - output_handler(Params_1, Seq_1) + case + encrypt_and_send_chunk(Params, Seq, [?TICK_CHUNK, TickData]) + of + {Params_1, Seq_1, ok} -> + output_handler(Params_1, Seq_1); + {_, _, Error} -> + _ = trace(Error), + death_row() + end end. -output_handler_send( - #params{dist_handle = DistHandle} = Params, Seq, {_, Size, _} = Q, Retry) -> - %% +output_handler_send(Params, Seq, {_, Size, _} = Q) -> if ?CHUNK_SIZE < Size -> - {Cleartext, Q_1} = deq_iovec(?CHUNK_SIZE, Q), - {Params_1, Seq_1} = - encrypt_and_send_chunk(Params, Seq, [?DATA_CHUNK, Cleartext]), - output_handler_send(Params_1, Seq_1, Q_1, Retry); - Retry -> - Q_1 = get_data(DistHandle, Q), - output_handler_send(Params, Seq, Q_1, false); + output_handler_send(Params, Seq, Q, ?CHUNK_SIZE); true -> - {Cleartext, _} = deq_iovec(Size, Q), - encrypt_and_send_chunk(Params, Seq, [?DATA_CHUNK, Cleartext]) + case get_data(Params#params.dist_handle, Q) of + {_, 0, _} -> + {Params, Seq}; + {_, Size, _} = Q_1 -> % Got no more + output_handler_send(Params, Seq, Q_1, Size); + Q_1 -> + output_handler_send(Params, Seq, Q_1) + end + end. + +output_handler_send(Params, Seq, Q, Size) -> + {Cleartext, Q_1} = deq_iovec(Size, Q), + case + encrypt_and_send_chunk(Params, Seq, [?DATA_CHUNK, Cleartext]) + of + {Params_1, Seq_1, ok} -> + output_handler_send(Params_1, Seq_1, Q_1); + {_, _, Error} -> + _ = trace(Error), + death_row() end. %% ------------------------------------------------------------------------- %% Input handler process %% -%% Here is T 0 or infinity to steer if we should try to receive +%% Here is T = 0|infinity to steer if we should try to receive %% more data or not; start with infinity, and when we get some %% data try with 0 to see if more is waiting @@ -1086,11 +1241,12 @@ input_handler(#params{socket = Socket} = Params, Seq, Q, T) -> end, input_handler(Params, Seq, Q_1, infinity); {tcp, Socket, Chunk} -> - input_chunk(Params, Seq, Q, Chunk); + input_chunk(Params, Seq, Q, T, Chunk); {tcp_closed, Socket} -> - error(connection_closed); - _Other -> + exit(connection_closed); + Other -> %% Ignore... + _ = trace(Other), input_handler(Params, Seq, Q, T) end after T -> @@ -1098,16 +1254,20 @@ input_handler(#params{socket = Socket} = Params, Seq, Q, T) -> input_handler(Params, Seq, Q_1, infinity) end. -input_chunk(Params, Seq, Q, Chunk) -> +input_chunk(Params, Seq, Q, T, Chunk) -> case decrypt_chunk(Params, Seq, Chunk) of <<?DATA_CHUNK, Cleartext/binary>> -> input_handler(Params, Seq + 1, enq_binary(Cleartext, Q), 0); <<?TICK_CHUNK, _/binary>> -> - input_handler(Params, Seq + 1, Q, 0); + input_handler(Params, Seq + 1, Q, T); + OtherChunk when is_binary(OtherChunk) -> + _ = trace(invalid_chunk), + exit(connection_closed); #params{} = Params_1 -> - input_handler(Params_1, 0, Q, 0); - _ -> - error(decrypt_error) + input_handler(Params_1, 0, Q, T); + error -> + _ = trace(decrypt_error), + exit(connection_closed) end. %% ------------------------------------------------------------------------- @@ -1198,64 +1358,110 @@ deliver_data(DistHandle, Front, Size, Rear, Bin) -> encrypt_and_send_chunk( #params{ - socket = Socket, rekey_interval = Seq, - key = Key, iv = IV, hash_algorithm = HashAlgorithm} = Params, + socket = Socket, + rekey_interval = Seq, + rekey_key = PubKeyB, + key = Key, + iv = {IVSalt, IVNo}, + hmac_algorithm = HmacAlgo} = Params, Seq, Cleartext) -> %% KeyLen = byte_size(Key), - IVLen = byte_size(IV), - Chunk = <<IV_1:IVLen/binary, KeySalt:KeyLen/binary>> = - crypto:strong_rand_bytes(IVLen + KeyLen), - ok = gen_tcp:send(Socket, encrypt_chunk(Params, Seq, [?REKEY_CHUNK, Chunk])), - Key_1 = hash_key(HashAlgorithm, Key, KeySalt), - Params_1 = Params#params{key = Key_1, iv = IV_1}, - ok = gen_tcp:send(Socket, encrypt_chunk(Params_1, 0, Cleartext)), - {Params_1, 1}; + IVSaltLen = byte_size(IVSalt), + #key_pair{public = PubKeyA} = KeyPair = get_new_key_pair(), + case + gen_tcp:send( + Socket, encrypt_chunk(Params, Seq, [?REKEY_CHUNK, PubKeyA])) + of + ok -> + SharedSecret = compute_shared_secret(KeyPair, PubKeyB), + IV = <<(IVNo + Seq):48>>, + {Key_1, <<IVSalt_1:IVSaltLen/binary, IVNo_1:48>>} = + hmac_key_iv( + HmacAlgo, SharedSecret, [Key, IVSalt, IV], + KeyLen, IVSaltLen + 6), + Params_1 = Params#params{key = Key_1, iv = {IVSalt_1, IVNo_1}}, + Result = + gen_tcp:send(Socket, encrypt_chunk(Params_1, 0, Cleartext)), + {Params_1, 1, Result}; + SendError -> + {Params, Seq + 1, SendError} + end; encrypt_and_send_chunk(#params{socket = Socket} = Params, Seq, Cleartext) -> - ok = gen_tcp:send(Socket, encrypt_chunk(Params, Seq, Cleartext)), - {Params, Seq + 1}. + Result = gen_tcp:send(Socket, encrypt_chunk(Params, Seq, Cleartext)), + {Params, Seq + 1, Result}. encrypt_chunk( #params{ - block_crypto = BlockCrypto, - iv = IV, key = Key, tag_len = TagLen}, Seq, Cleartext) -> + aead_cipher = AeadCipher, + iv = {IVSalt, IVNo}, key = Key, tag_len = TagLen}, Seq, Cleartext) -> %% ChunkLen = iolist_size(Cleartext) + TagLen, AAD = <<Seq:32, ChunkLen:32>>, + IVBin = <<IVSalt/binary, (IVNo + Seq):48>>, {Ciphertext, CipherTag} = - crypto:block_encrypt( - crypto_cipher_name(BlockCrypto), Key, IV, {AAD, Cleartext, TagLen}), + crypto:block_encrypt(AeadCipher, Key, IVBin, {AAD, Cleartext, TagLen}), Chunk = [Ciphertext,CipherTag], Chunk. decrypt_chunk( #params{ - block_crypto = BlockCrypto, - iv = IV, key = Key, tag_len = TagLen} = Params, Seq, Chunk) -> + aead_cipher = AeadCipher, + iv = {IVSalt, IVNo}, key = Key, tag_len = TagLen} = Params, Seq, Chunk) -> %% ChunkLen = byte_size(Chunk), - true = TagLen =< ChunkLen, % Assert - AAD = <<Seq:32, ChunkLen:32>>, - CiphertextLen = ChunkLen - TagLen, - <<Ciphertext:CiphertextLen/binary, CipherTag:TagLen/binary>> = Chunk, - block_decrypt( - Params, Seq, crypto_cipher_name(BlockCrypto), - Key, IV, {AAD, Ciphertext, CipherTag}). + if + ChunkLen < TagLen -> + error; + true -> + AAD = <<Seq:32, ChunkLen:32>>, + IVBin = <<IVSalt/binary, (IVNo + Seq):48>>, + CiphertextLen = ChunkLen - TagLen, + case Chunk of + <<Ciphertext:CiphertextLen/binary, + CipherTag:TagLen/binary>> -> + block_decrypt( + Params, Seq, AeadCipher, Key, IVBin, + {AAD, Ciphertext, CipherTag}); + _ -> + error + end + end. block_decrypt( - #params{rekey_interval = Seq} = Params, Seq, CipherName, Key, IV, Data) -> + #params{ + rekey_key = #key_pair{public = PubKeyA} = KeyPair, + rekey_interval = RekeyInterval} = Params, + Seq, AeadCipher, Key, IV, Data) -> %% - KeyLen = byte_size(Key), - IVLen = byte_size(IV), - case crypto:block_decrypt(CipherName, Key, IV, Data) of - <<?REKEY_CHUNK, IV_1:IVLen/binary, KeySalt:KeyLen/binary>> -> - Key_1 = hash_key(Params#params.hash_algorithm, Key, KeySalt), - Params#params{iv = IV_1, key = Key_1}; - _ -> - error(decrypt_error) - end; -block_decrypt(_Params, _Seq, CipherName, Key, IV, Data) -> - crypto:block_decrypt(CipherName, Key, IV, Data). + case crypto:block_decrypt(AeadCipher, Key, IV, Data) of + <<?REKEY_CHUNK, Rest/binary>> -> + PubKeyLen = byte_size(PubKeyA), + case Rest of + <<PubKeyB:PubKeyLen/binary>> -> + SharedSecret = compute_shared_secret(KeyPair, PubKeyB), + KeyLen = byte_size(Key), + IVLen = byte_size(IV), + IVSaltLen = IVLen - 6, + {Key_1, <<IVSalt:IVSaltLen/binary, IVNo:48>>} = + hmac_key_iv( + Params#params.hmac_algorithm, + SharedSecret, [Key, IV], KeyLen, IVLen), + Params#params{iv = {IVSalt, IVNo}, key = Key_1}; + _ -> + error + end; + Chunk when is_binary(Chunk) -> + case Seq of + RekeyInterval -> + %% This was one chunk too many without rekeying + error; + _ -> + Chunk + end; + error -> + error + end. %% ------------------------------------------------------------------------- %% Queue of binaries i.e an iovec queue @@ -1289,34 +1495,46 @@ deq_iovec(GetSize, [Bin|Front], Size, Rear, Acc) -> %% ------------------------------------------------------------------------- +death_row() -> death_row(connection_closed). +%% +death_row(normal) -> death_row(connection_closed); +death_row(Reason) -> receive after 5000 -> exit(Reason) end. + +%% ------------------------------------------------------------------------- + %% Trace point trace(Term) -> Term. %% Keep an eye on this Pid (debug) +-ifndef(undefined). +monitor_dist_proc(Pid) -> + Pid. +-else. monitor_dist_proc(Pid) -> -%%% spawn( -%%% fun () -> -%%% MRef = erlang:monitor(process, Pid), -%%% receive -%%% {'DOWN', MRef, _, _, normal} -> -%%% error_logger:error_report( -%%% [dist_proc_died, -%%% {reason, normal}, -%%% {pid, Pid}]); -%%% {'DOWN', MRef, _, _, Reason} -> -%%% error_logger:info_report( -%%% [dist_proc_died, -%%% {reason, Reason}, -%%% {pid, Pid}]) -%%% end -%%% end), + spawn( + fun () -> + MRef = erlang:monitor(process, Pid), + receive + {'DOWN', MRef, _, _, normal} -> + error_logger:error_report( + [dist_proc_died, + {reason, normal}, + {pid, Pid}]); + {'DOWN', MRef, _, _, Reason} -> + error_logger:info_report( + [dist_proc_died, + {reason, Reason}, + {pid, Pid}]) + end + end), Pid. +-endif. dbg() -> dbg:stop(), dbg:tracer(), dbg:p(all, c), - dbg:tpl(?MODULE, cx), + dbg:tpl(?MODULE, trace, cx), dbg:tpl(erlang, dist_ctrl_get_data_notification, cx), dbg:tpl(erlang, dist_ctrl_get_data, cx), dbg:tpl(erlang, dist_ctrl_put_data, cx), diff --git a/lib/ssl/test/ssl_dist_bench_SUITE.erl b/lib/ssl/test/ssl_dist_bench_SUITE.erl index 1fea6f6f72..67944c74d2 100644 --- a/lib/ssl/test/ssl_dist_bench_SUITE.erl +++ b/lib/ssl/test/ssl_dist_bench_SUITE.erl @@ -169,16 +169,10 @@ end_per_suite(Config) -> init_per_group(ssl, Config) -> [{ssl_dist, true}, {ssl_dist_prefix, "SSL"}|Config]; init_per_group(crypto, Config) -> - case inet_crypto_dist:is_supported() of - true -> - [{ssl_dist, false}, {ssl_dist_prefix, "Crypto"}, - {ssl_dist_args, - "-proto_dist inet_crypto " - "-inet_crypto '#{secret => \"123456\"}'"} - |Config]; - false -> - {skip, "Not supported on this OTP version"} - end; + [{ssl_dist, false}, {ssl_dist_prefix, "Crypto"}, + {ssl_dist_args, + "-proto_dist inet_crypto"} + |Config]; init_per_group(plain, Config) -> [{ssl_dist, false}, {ssl_dist_prefix, "Plain"}|Config]; init_per_group(_GroupName, Config) -> diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index 98070f794c..b5545b71f7 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 9.2.2 +SSL_VSN = 9.2.3 diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index 65650a25c7..ef12e525e3 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -31,6 +31,24 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 3.8.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A bug in gen_statem has been fixed where the internal + timeout message could arrive as an info to the callback + module during high load due to incorrect use of + asynchronous timer cancel.</p> + <p> + Own Id: OTP-15295</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.8.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/src/stdlib.appup.src b/lib/stdlib/src/stdlib.appup.src index 08612ed17f..9a1b92a87c 100644 --- a/lib/stdlib/src/stdlib.appup.src +++ b/lib/stdlib/src/stdlib.appup.src @@ -43,7 +43,8 @@ {<<"^3\\.7\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.7\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.8$">>,[restart_new_emulator]}, - {<<"^3\\.8\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}], + {<<"^3\\.8\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.8\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], [{<<"^3\\.4$">>,[restart_new_emulator]}, {<<"^3\\.4\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -60,4 +61,5 @@ {<<"^3\\.7\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.7\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.8$">>,[restart_new_emulator]}, - {<<"^3\\.8\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}]}. + {<<"^3\\.8\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.8\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index 6471dc70e0..80ec81b832 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -1 +1 @@ -STDLIB_VSN = 3.8.1 +STDLIB_VSN = 3.8.2 diff --git a/make/otp_patch_solve_forward_merge_version b/make/otp_patch_solve_forward_merge_version index ec635144f6..f599e28b8a 100644 --- a/make/otp_patch_solve_forward_merge_version +++ b/make/otp_patch_solve_forward_merge_version @@ -1 +1 @@ -9 +10 diff --git a/otp_versions.table b/otp_versions.table index 566aa3cb4c..28063bb4dd 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,4 @@ +OTP-21.3.8 : common_test-1.17.2 eldap-1.2.7 erl_interface-3.11.3 erts-10.3.5 public_key-1.6.6 ssl-9.2.3 stdlib-3.8.2 # asn1-5.0.8 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 erl_docgen-0.9 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 inets-7.0.7 jinterface-1.9.1 kernel-6.3.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6 syntax_tools-2.1.7 tftp-1.0.1 tools-3.1 wx-1.8.7 xmerl-1.3.20 : OTP-21.3.7 : ssh-4.7.6 # asn1-5.0.8 common_test-1.17.1 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.6 erl_docgen-0.9 erl_interface-3.11.2 erts-10.3.4 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 inets-7.0.7 jinterface-1.9.1 kernel-6.3.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.5 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssl-9.2.2 stdlib-3.8.1 syntax_tools-2.1.7 tftp-1.0.1 tools-3.1 wx-1.8.7 xmerl-1.3.20 : OTP-21.3.6 : ssl-9.2.2 # asn1-5.0.8 common_test-1.17.1 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.6 erl_docgen-0.9 erl_interface-3.11.2 erts-10.3.4 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 inets-7.0.7 jinterface-1.9.1 kernel-6.3.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.5 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.5 stdlib-3.8.1 syntax_tools-2.1.7 tftp-1.0.1 tools-3.1 wx-1.8.7 xmerl-1.3.20 : OTP-21.3.5 : diameter-2.2.1 erts-10.3.4 inets-7.0.7 # asn1-5.0.8 common_test-1.17.1 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 edoc-0.10 eldap-1.2.6 erl_docgen-0.9 erl_interface-3.11.2 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 jinterface-1.9.1 kernel-6.3.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.5 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.5 ssl-9.2.1 stdlib-3.8.1 syntax_tools-2.1.7 tftp-1.0.1 tools-3.1 wx-1.8.7 xmerl-1.3.20 : |