diff options
148 files changed, 10456 insertions, 1829 deletions
diff --git a/OTP_VERSION b/OTP_VERSION index 95f8a283f0..758704512b 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -22.0.2 +22.0.4 diff --git a/erts/doc/src/alt_dist.xml b/erts/doc/src/alt_dist.xml index 7c997cae20..f72e8acd2c 100644 --- a/erts/doc/src/alt_dist.xml +++ b/erts/doc/src/alt_dist.xml @@ -680,7 +680,10 @@ of the distribution controller, <c>Received</c> is received packets, <c>Sent</c> is sent packets, and <c>PendSend</c> is - amount of packets in queue to be sent + amount of data in queue to be sent + (typically in bytes, but <c>dist_util</c> + only checks whether the value is non-zero + to know there is data in queue) or a <c>boolean()</c> indicating whether there are packets in queue to be sent. </p> diff --git a/erts/doc/src/crash_dump.xml b/erts/doc/src/crash_dump.xml index 33d0903622..876834307a 100644 --- a/erts/doc/src/crash_dump.xml +++ b/erts/doc/src/crash_dump.xml @@ -289,7 +289,7 @@ Slogan: <reason></pre> <marker id="memory"></marker> <title>Memory Information</title> <p>Under the tag <em>=memory</em> is shown information similar - to what can be obtainted on a living node with + to what can be obtained on a living node with <seealso marker="erts:erlang#memory/0"> <c>erlang:memory()</c></seealso>.</p> </section> diff --git a/erts/doc/src/driver.xml b/erts/doc/src/driver.xml index 8f31df4cad..a873bf9931 100644 --- a/erts/doc/src/driver.xml +++ b/erts/doc/src/driver.xml @@ -391,7 +391,7 @@ void encode_result(ei_x_buff* x, PGresult* res, PGconn* conn) <p>Before a driver can be called from Erlang, it must be loaded and opened. Loading is done using the <c><![CDATA[erl_ddll]]></c> module (the <c><![CDATA[erl_ddll]]></c> driver that loads dynamic - driver is actually a driver itself). If loading is successfull, + driver is actually a driver itself). If loading is successful, the port can be opened with <c><![CDATA[open_port/2]]></c>. The port name must match the name of the shared library and the name in the driver entry structure.</p> diff --git a/erts/doc/src/erl_ext_dist.xml b/erts/doc/src/erl_ext_dist.xml index 3730f0e8ac..2ba5994557 100644 --- a/erts/doc/src/erl_ext_dist.xml +++ b/erts/doc/src/erl_ext_dist.xml @@ -128,7 +128,7 @@ are deprecated and are only kept for backward compatibility when decoding terms encoded by older nodes.</p> <p>Support for UTF-8 encoded atoms in the external format has been - available since ERTS 5.10 (OTP R16). This abillity allows such old nodes + available since ERTS 5.10 (OTP R16). This ability allows such old nodes to decode, store and encode any Unicode atoms received from a new OTP 20 node.</p> <p>The maximum number of allowed characters in an atom is 255. In the @@ -913,7 +913,7 @@ <tcaption>SMALL_BIG_EXT</tcaption></table> <p> Bignums are stored in unary form with a <c>Sign</c> byte, - that is, 0 if the binum is positive and 1 if it is negative. The + that is, 0 if the bignum is positive and 1 if it is negative. The digits are stored with the least significant byte stored first. To calculate the integer, the following formula can be used: </p> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index f41ed87048..0e82ceba7d 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -10129,6 +10129,11 @@ timestamp() -> time stamp flags are remembered, so if two are passed and the one with highest precedence later is disabled, the other one becomes active.</p> + <p>If a match specification (applicable only for <c>call</c>, <c>send</c> + and <c>'receive'</c> tracing) contains a <c>{message}</c> action + function with a non-boolean value, that value is added as an extra + element to the message tuple either in the last position or before + the timestamp (if it is present).</p> <p>Trace messages:</p> <marker id="trace_3_trace_messages"></marker> <taglist> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index c631ea01ce..cfa952f01c 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -31,6 +31,67 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 10.4.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a buffer overflow when + <c>binary_to_existing_atom/2</c> and + <c>list_to_existing_atom/2</c> was used with the + <c>latin1</c> encoding.</p> + <p> + Own Id: OTP-15819 Aux Id: ERL-944 </p> + </item> + <item> + <p> + The runtime system disconnected a connection if it + received an <c>exit/2</c> signal where the recipient was + a process on an old incarnation of the current node. That + is, the receiving node had the same node name, but + another "creation" number. The signal will now just be + dropped since the receiving process no longer exists.</p> + <p> + Own Id: OTP-15867 Aux Id: ERIERL-373 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.4.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed <c>process_info(Pid,reductions)</c> to not + categorically increase reduction count of the measured + process <c>Pid</c>. Repeated reduction measure of an idle + process will most often (but not guaranteed) return the + same value, like it behaved before OTP 21.3.8.</p> + <p> + Own Id: OTP-15865 Aux Id: ERL-964 </p> + </item> + <item> + <p>Fixed an incorrect load-time optimization that could + cause a crash when extracting deeply nested tuple + elements.</p> + <p> + Own Id: OTP-15871 Aux Id: ERIERL-374 </p> + </item> + <item> + <p> + Fix bug causing VM crash when pressing P for "proc info" + in Erlang shell break menu. Bug exists since OTP 22.0.</p> + <p> + Own Id: OTP-15873 Aux Id: ERL-965 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.4.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -531,6 +592,95 @@ </section> +<section><title>Erts 10.3.5.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If you set <c>{linger,{true,0}}</c> on a <c>gen_tcp</c> + listen socket, accept a connection on that socket, and + then close the accepted socket, now the linger zero + setting is transferred to the accepted socket. Before + this correction that information was lost and the close + behaviour on the accepted socket incorrect.</p> + <p> + Own Id: OTP-15370 Aux Id: ERIERL-353 </p> + </item> + <item> + <p> + Fixed <c>process_info(Pid,reductions)</c> to not + categorically increase reduction count of the measured + process <c>Pid</c>. Repeated reduction measure of an idle + process will most often (but not guaranteed) return the + same value, like it behaved before OTP 21.3.8.</p> + <p> + Own Id: OTP-15865 Aux Id: ERL-964 </p> + </item> + <item> + <p> + The runtime system disconnected a connection if it + received an <c>exit/2</c> signal where the recipient was + a process on an old incarnation of the current node. That + is, the receiving node had the same node name, but + another "creation" number. The signal will now just be + dropped since the receiving process no longer exists.</p> + <p> + Own Id: OTP-15867 Aux Id: ERIERL-373 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The possibility to send ancillary data, in particular the + TOS field, has been added to <c>gen_udp:send/4,5</c>.</p> + <p> + Own Id: OTP-15747 Aux Id: ERIERL-294 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.3.5.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>In nested use of <c>try</c>/<c>catch</c>, rethrowing + an exception using <c>erlang:raise/3</c> with a different + class would not always be able to change the class of the + exception.</p> + <p> + Own Id: OTP-15834 Aux Id: ERIERL-367 </p> + </item> + <item> + <p> + Fixed bug in <c>seq_trace:set_token(label,Term)</c> which + could cause VM crash if <c>Term</c> was heap allocated + (not an atom, small integer, local pid or port). Bug + exists since OTP 21.0 when terms other than small + integers were first allowed as labels.</p> + <p> + Own Id: OTP-15849 Aux Id: ERL-700 </p> + </item> + <item> + <p> + Fix <c>seq_trace:print/2</c> not to raise <c>badarg</c> + exception if label is not a small integer. Bug exists + since OTP 21.0.</p> + <p> + Own Id: OTP-15859 Aux Id: ERL-700 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.3.5.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -2557,6 +2707,23 @@ </section> +<section><title>Erts 9.3.3.11</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a buffer overflow when + <c>binary_to_existing_atom/2</c> and + <c>list_to_existing_atom/2</c> was used with the + <c>latin1</c> encoding.</p> + <p> + Own Id: OTP-15819 Aux Id: ERL-944 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 9.3.3.10</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/doc/src/socket.xml b/erts/doc/src/socket.xml index 9b34bf1df1..b4e22e86a8 100644 --- a/erts/doc/src/socket.xml +++ b/erts/doc/src/socket.xml @@ -41,6 +41,38 @@ e.g. <seealso marker="#recv/3"><c>recv/3</c></seealso>, has a timeout argument. </p> <note> + <p>Some functions allow for an <i>asynchronous</i> call. + This is achieved by setting the <c>Timeout</c> argument to + <c>nowait</c>. For instance, if calling the + <seealso marker="#recv_async"><c>recv/3</c></seealso> + function with Timeout set to <c>nowait</c> (<c>recv(Sock, 0, nowait)</c>) + when there is actually nothing to read, it will return with + <c>{ok, </c> + <seealso marker="#type-select_info"><c>SelectInfo</c></seealso><c>}</c>. + When data eventually arrives a 'select' message + will be sent to the caller: </p> + <taglist> + <!-- NOTE THAT THE EMPTY TAG IS INTENTIONAL --> + <tag></tag> + <item><c>{'$socket', socket(), select, select_ref()}</c></item> + </taglist> + <p>The caller can now make another + call to the recv function and now expect data.</p> + <p>Note that all other users are <em>locked out</em> until the + 'current user' has called the function (recv in this case).</p> + <p>Another message the user must be prepared for (when making asynchronous + calls) is the <c>abort</c> message:</p> + <taglist> + <!-- NOTE THAT THE EMPTY TAG IS INTENTIONAL --> + <tag></tag> + <item><c>{'$socket', socket(), abort, Info}</c></item> + </taglist> + <p>This message indicates + that the (asynchronous) operation has been aborted. + If, for instance, the socket has been closed (by another process), + <c>Info</c> will be <c>{select_ref(), closed}</c>. </p> + </note> + <note> <p>There is currently <em>no</em> support for Windows. </p> <p>Support for IPv6 has been implemented but <em>not</em> tested. </p> <p>SCTP has only been partly implemented (and not tested). </p> @@ -65,6 +97,21 @@ </desc> </datatype> <datatype> + <name name="select_tag"/> + <desc> + <p>A tag that describes the (select) operation.</p> + </desc> + </datatype> + <datatype> + <name name="select_ref"/> + <desc> + <p>A reference that uniquely identifies the (select) operation.</p> + </desc> + </datatype> + <datatype> + <name name="select_info"/> + </datatype> + <datatype> <name name="ip4_address"/> </datatype> <datatype> @@ -257,7 +304,7 @@ <funcs> <func> <name name="accept" arity="1" since="OTP 22.0"/> - <name name="accept" arity="2" since="OTP 22.0"/> + <name name="accept" arity="2" clause_i="2" since="OTP 22.0"/> <fsummary>Accept a connection on a socket.</fsummary> <desc> <p>Accept a connection on a socket.</p> @@ -269,6 +316,26 @@ </func> <func> + <name name="accept" arity="2" clause_i="1" anchor="accept_async" since="OTP @OTP-15731@"/> + <fsummary>Accept a connection on a socket.</fsummary> + <desc> + <p>Accept a connection on a socket.</p> + + <p>This call is used with connection-based socket types + (<c>stream</c> or <c>seqpacket</c>). It extracs the first pending + connection request for the listen socket and returns the (newly) + connected socket.</p> + + <p>In the case when there is no connections waiting, the function + will return with the <c>SelectInfo</c>. The caller can then await a + select message, <c>{'$socket', Socket, select, Info}</c> (where + <c>Info</c> is the <c>select_ref()</c> from the <c>SelectInfo</c>), + when a client connects (a subsequent call to accept will then return + the socket). </p> + </desc> + </func> + + <func> <name name="bind" arity="2" since="OTP 22.0"/> <fsummary>Bind a name to a socket.</fsummary> <desc> @@ -282,6 +349,18 @@ </func> <func> + <name name="cancel" arity="2" since="OTP @OTP-15731@"/> + <fsummary>Cancel an asynchronous request.</fsummary> + <desc> + <p>Cancel an asynchronous request.</p> + + <p>Call this function in order to cancel a previous + asynchronous call to, e.g. + <seealso marker="#recv/3"><c>recv/3</c></seealso>. </p> + </desc> + </func> + + <func> <name name="close" arity="1" since="OTP 22.0"/> <fsummary>Close a socket.</fsummary> <desc> @@ -302,7 +381,7 @@ <func> <name name="connect" arity="2" since="OTP 22.0"/> - <name name="connect" arity="3" since="OTP 22.0"/> + <name name="connect" arity="3" clause_i="2" since="OTP 22.0"/> <fsummary>Initiate a connection on a socket.</fsummary> <desc> <p>This function connects the socket to the address @@ -311,6 +390,24 @@ </func> <func> + <name name="connect" arity="3" clause_i="1" anchor="connect_async" since="OTP @OTP-15731@"/> + <fsummary>Initiate a connection on a socket.</fsummary> + <desc> + <p>This function connects the socket to the address + specied by the <c>SockAddr</c> argument.</p> + + <p>In the case when its not possible to immediately establish a + connection, the function will return with the + <seealso marker="#type-select_info"><c>SelectInfo</c></seealso>. + The caller can then await a + select message, <c>{'$socket', Socket, select, Info}</c> (where + <c>Info</c> is the <c>select_ref()</c> from the <c>SelectInfo</c>, + a subsequent call to connect will then + establish the connection). </p> + </desc> + </func> + + <func> <name name="getopt" arity="3" clause_i="1" since="OTP 22.0"/> <name name="getopt" arity="3" clause_i="2" since="OTP 22.0"/> <name name="getopt" arity="3" clause_i="3" since="OTP 22.0"/> @@ -406,8 +503,8 @@ <name name="recv" arity="1" since="OTP 22.0"/> <name name="recv" arity="2" since="OTP 22.0"/> <name name="recv" arity="3" clause_i="1" since="OTP 22.0"/> - <name name="recv" arity="3" clause_i="2" since="OTP 22.0"/> - <name name="recv" arity="4" since="OTP 22.0"/> + <name name="recv" arity="3" clause_i="3" since="OTP 22.0"/> + <name name="recv" arity="4" clause_i="2" since="OTP 22.0"/> <fsummary>Receive a message from a socket.</fsummary> <desc> <p>Receive a message from a socket.</p> @@ -418,36 +515,91 @@ </func> <func> + <name name="recv" arity="3" clause_i="2" anchor="recv_async" since="OTP @OTP-15731@"/> + <name name="recv" arity="4" clause_i="1" since="OTP @OTP-15731@"/> + <fsummary>Receive a message from a socket.</fsummary> + <desc> + <p>Receive a message from a socket.</p> + + <p>There is a special case for the argument <c>Length</c>. + If it is set to zero (0), it means "give me everything you + currently have".</p> + + <p>In the case when there is no data waiting, the function + will return with the <c>SelectInfo</c>. The caller can then await a + select message, <c>{'$socket', Socket, select, Info}</c> (where + <c>Info</c> is the <c>select_ref()</c> from the <c>SelectInfo</c>), + when data has arrived (a subsequent call to recv will then return + the data). </p> + <p>Note that if a length (<c>> 0</c>) is specified, and only part + of that amount of data is available, the function will return with + that data <em>and</em> the <c>SelectInfo</c> (if the caller don't + want to wait for the remaining data, it must immediately call + the <seealso marker="#cancel/2"><c>cancel/2</c></seealso> function.)</p> + </desc> + </func> + + <func> <name name="recvfrom" arity="1" since="OTP 22.0"/> <name name="recvfrom" arity="2" since="OTP 22.0"/> - <name name="recvfrom" arity="3" clause_i="1" since="OTP 22.0"/> <name name="recvfrom" arity="3" clause_i="2" since="OTP 22.0"/> <name name="recvfrom" arity="3" clause_i="3" since="OTP 22.0"/> - <name name="recvfrom" arity="4" since="OTP 22.0"/> + <name name="recvfrom" arity="3" clause_i="5" since="OTP 22.0"/> + <name name="recvfrom" arity="4" clause_i="2" since="OTP 22.0"/> <fsummary>Receive a message from a socket.</fsummary> <desc> <p>Receive a message from a socket.</p> <p>This function reads "messages", which means that regardless of - how much we want to read, it returns when we get a message.</p> + how much we want to read, it returns when we get a message + (if the buffer size is to small, the message will be truncated).</p> + <p>The <c>BufSz</c> argument basically defines the size of the + receive buffer. By setting the value to zero (0), the configured + size (setopt with <c>Level</c> = <c>otp</c> and + <c>Key</c> = <c>rcvbuf</c>) is used.</p> + <p>It may be impossible to know what (buffer) size is appropriate + "in advance", and in those cases it may be convenient to use the + (recv) 'peek' flag. When this flag is provided, the message is *not* + "consumed" from the underlying buffers, so another recvfrom call + is needed, possibly with a then adjusted buffer size.</p> + </desc> + </func> + + <func> + <name name="recvfrom" arity="3" clause_i="1" anchor="recvfrom_async" since="OTP @OTP-15731@"/> + <name name="recvfrom" arity="3" clause_i="4" since="OTP @OTP-15731@"/> + <name name="recvfrom" arity="4" clause_i="1" since="OTP @OTP-15731@"/> + <fsummary>Receive a message from a socket.</fsummary> + <desc> + <p>Receive a message from a socket.</p> + <p>This function reads "messages", which means that regardless of + how much we want to read, it returns when we get a message + (if the buffer size is to small, the message will be truncated).</p> <p>The <c>BufSz</c> argument basically defines the size of the receive buffer. By setting the value to zero (0), the configured - size (setopt with <c>Level</c> = <c>otp</c> and <c>Key</c> = <c>rcvbuf</c>) - is used.</p> + size (setopt with <c>Level</c> = <c>otp</c> and + <c>Key</c> = <c>rcvbuf</c>) is used.</p> <p>It may be impossible to know what (buffer) size is appropriate "in advance", and in those cases it may be convenient to use the (recv) 'peek' flag. When this flag is provided, the message is *not* "consumed" from the underlying buffers, so another recvfrom call is needed, possibly with a then adjusted buffer size.</p> + + <p>In the case when there is no data waiting, the function + will return with the <c>SelectInfo</c>. The caller can then await a + select message, <c>{'$socket', Socket, select, Info}</c> (where + <c>Info</c> is the <c>select_ref()</c> from the <c>SelectInfo</c>), + when data has arrived (a subsequent call to recvfrom will then return + the data). </p> </desc> </func> <func> <name name="recvmsg" arity="1" since="OTP 22.0"/> <name name="recvmsg" arity="2" clause_i="1" since="OTP 22.0"/> - <name name="recvmsg" arity="2" clause_i="2" since="OTP 22.0"/> - <name name="recvmsg" arity="3" clause_i="1" since="OTP 22.0"/> + <name name="recvmsg" arity="2" clause_i="3" since="OTP 22.0"/> <name name="recvmsg" arity="3" clause_i="2" since="OTP 22.0"/> - <name name="recvmsg" arity="5" since="OTP 22.0"/> + <name name="recvmsg" arity="3" clause_i="3" since="OTP 22.0"/> + <name name="recvmsg" arity="5" clause_i="2" since="OTP 22.0"/> <fsummary>Receive a message from a socket.</fsummary> <desc> <p>Receive a message from a socket.</p> @@ -461,8 +613,8 @@ <p>The <c>BufSz</c> argument basically defines the size of the receive buffer. By setting the value to zero (0), the configured - size (setopt with <c>Level</c> = <c>otp</c> and <c>Key</c> = <c>rcvbuf</c>) - is used.</p> + size (setopt with <c>Level</c> = <c>otp</c> and + <c>Key</c> = <c>rcvbuf</c>) is used.</p> <p>The <c>CtrlSz</c> argument basically defines the size of the receive buffer for the control messages. @@ -478,21 +630,83 @@ </func> <func> + <name name="recvmsg" arity="2" clause_i="2" anchor="recvmsg_async" since="OTP @OTP-15731@"/> + <name name="recvmsg" arity="3" clause_i="1" since="OTP @OTP-15731@"/> + <name name="recvmsg" arity="5" clause_i="1" since="OTP @OTP-15731@"/> + <fsummary>Receive a message from a socket.</fsummary> + <desc> + <p>Receive a message from a socket.</p> + <p>This function reads "messages", which means that regardless of + how much we want to read, it returns when we get a message.</p> + <p>The message will be delivered in the form of a <c>msghdr()</c>, + which may contain the source address (if socket not connected), + a list of <c>cmsghdr_recv()</c> (depends on what socket options have + been set and what the protocol and platform supports) and + also a set of flags, providing further info about the read. </p> + + <p>The <c>BufSz</c> argument basically defines the size of the + receive buffer. By setting the value to zero (0), the configured + size (setopt with <c>Level</c> = <c>otp</c> and + <c>Key</c> = <c>rcvbuf</c>) is used.</p> + + <p>The <c>CtrlSz</c> argument basically defines the size of the + receive buffer for the control messages. + By setting the value to zero (0), the configured size (setopt + with <c>Level</c> = <c>otp</c>) is used.</p> + + <p>It may be impossible to know what (buffer) size is appropriate + "in advance", and in those cases it may be convenient to use the + (recv) 'peek' flag. When this flag is provided, the message is *not* + "consumed" from the underlying buffers, so another recvmsg call + is needed, possibly with a then adjusted buffer size.</p> + + <p>In the case when there is no data waiting, the function + will return with the <c>SelectInfo</c>. The caller can then await a + select message, <c>{'$socket', Socket, select, Info}</c> (where + <c>Info</c> is the <c>select_ref()</c> from the <c>SelectInfo</c>), + when data has arrived (a subsequent call to recvmsg will then return + the data). </p> + </desc> + </func> + + <func> <name name="send" arity="2" since="OTP 22.0"/> <name name="send" arity="3" clause_i="1" since="OTP 22.0"/> - <name name="send" arity="3" clause_i="2" since="OTP 22.0"/> - <name name="send" arity="4" since="OTP 22.0"/> + <name name="send" arity="3" clause_i="3" since="OTP 22.0"/> + <name name="send" arity="4" clause_i="2" since="OTP 22.0"/> + <fsummary>Send a message on a socket.</fsummary> + <desc> + <p>Send a message on a connected socket.</p> + </desc> + </func> + + <func> + <name name="send" arity="3" clause_i="2" anchor="send_async" since="OTP @OTP-15731@"/> + <name name="send" arity="4" clause_i="1" since="OTP @OTP-15731@"/> <fsummary>Send a message on a socket.</fsummary> <desc> <p>Send a message on a connected socket.</p> + + <p>In the case when there is no room in the (system-) buffers, + the function will return with the <c>SelectInfo</c>. The caller + can then await a select message, + <c>{'$socket', Socket, select, Info}</c> + (where <c>Info</c> is the <c>select_ref()</c> from the + <c>SelectInfo</c>), when there is room for more data (a subsequent + call to send will then send the data). </p> + <p>Note that if not all the data was sent, the function will return + with the remaining data <em>and</em> the <c>SelectInfo</c> + (if the caller don't + want to wait to be able to send the rest, it should immediately call + the <seealso marker="#cancel/2"><c>cancel/2</c></seealso> function.)</p> </desc> </func> <func> <name name="sendmsg" arity="2" since="OTP 22.0"/> <name name="sendmsg" arity="3" clause_i="1" since="OTP 22.0"/> - <name name="sendmsg" arity="3" clause_i="2" since="OTP 22.0"/> - <name name="sendmsg" arity="4" since="OTP 22.0"/> + <name name="sendmsg" arity="3" clause_i="3" since="OTP 22.0"/> + <name name="sendmsg" arity="4" clause_i="2" since="OTP 22.0"/> <fsummary>Send a message on a socket.</fsummary> <desc> <p>Send a message on a socket. The destination, if needed @@ -513,10 +727,39 @@ </func> <func> + <name name="sendmsg" arity="3" clause_i="2" anchor="sendmsg_async" since="OTP @OTP-15731@"/> + <name name="sendmsg" arity="4" clause_i="1" since="OTP @OTP-15731@"/> + <fsummary>Send a message on a socket.</fsummary> + <desc> + <p>Send a message on a socket. The destination, if needed + (socket <em>not</em> connected) is provided in the <c>MsgHdr</c>, + which also contains the message to send, + The <c>MsgHdr</c> may also contain an list of optional <c>cmsghdr_send()</c> + (depends on what the protocol and platform supports).</p> + + <p>Unlike the <seealso marker="#send/2"><c>send</c></seealso> function, + this one sends <em>one message</em>. + This means that if, for whatever reason, its not possible to send the + message in one go, the function will instead return with the + <em>remaining</em> data (<c>{ok, Remaining}</c>). Thereby leaving it + up to the caller to decide what to do (retry with the remaining data + of give up). </p> + + <p>In the case when there is no room in the (system-) buffers, + the function will return with the <c>SelectInfo</c>. The caller + can then await a select message, + <c>{'$socket', Socket, select, Info}</c> + (where <c>Info</c> is the <c>select_ref()</c> from the + <c>SelectInfo</c>), when there is room for more data (a subsequent + call to sendmsg will then send the data). </p> + </desc> + </func> + + <func> <name name="sendto" arity="3" since="OTP 22.0"/> <name name="sendto" arity="4" clause_i="1" since="OTP 22.0"/> - <name name="sendto" arity="4" clause_i="2" since="OTP 22.0"/> - <name name="sendto" arity="5" since="OTP 22.0"/> + <name name="sendto" arity="4" clause_i="3" since="OTP 22.0"/> + <name name="sendto" arity="5" clause_i="2" since="OTP 22.0"/> <fsummary>Send a message on a socket.</fsummary> <desc> <p>Send a message on a socket, to the specified destination.</p> @@ -524,6 +767,23 @@ </func> <func> + <name name="sendto" arity="4" clause_i="2" anchor="sendto_async" since="OTP @OTP-15731@"/> + <name name="sendto" arity="5" clause_i="1" since="OTP @OTP-15731@"/> + <fsummary>Send a message on a socket.</fsummary> + <desc> + <p>Send a message on a socket, to the specified destination.</p> + + <p>In the case when there is no room in the (system-) buffers, + the function will return with the <c>SelectInfo</c>. The caller + can then await a select message, + <c>{'$socket', Socket, select, Info}</c> + (where <c>Info</c> is the <c>select_ref()</c> from the + <c>SelectInfo</c>), when there is room for more data (a subsequent + call to sendto will then send the data). </p> + </desc> + </func> + + <func> <name name="setopt" arity="4" clause_i="1" since="OTP 22.0"/> <name name="setopt" arity="4" clause_i="2" since="OTP 22.0"/> <name name="setopt" arity="4" clause_i="3" since="OTP 22.0"/> diff --git a/erts/doc/src/socket_usage.xml b/erts/doc/src/socket_usage.xml index 4b3872d7e3..7e65bcbf70 100644 --- a/erts/doc/src/socket_usage.xml +++ b/erts/doc/src/socket_usage.xml @@ -34,11 +34,84 @@ <title>Introduction</title> <p>The socket interface (module) is basically an "thin" layer on top of the OS socket interface. It is assumed that, unless you have special needs, - gen_[tcp|udp|sctp] should be sufficent. </p> + gen_[tcp|udp|sctp] should be sufficent (when they become available). </p> <p>Note that just because we have a documented and described option, it does <em>not</em> mean that the OS supports it. So its recommended that the user reads the platform specific documentation for the option used. </p> + <section> + <title>Asynchronous calls</title> + <p>Some functions allow for an <i>asynchronous</i> call + (<seealso marker="socket#accept_async"><c>accept/2</c></seealso>, + <seealso marker="socket#connect_async"><c>connect/3</c></seealso>, + <seealso marker="socket#recv_async"><c>recv/3,4</c></seealso>, + <seealso marker="socket#recvfrom_async"><c>recvfrom/3,4</c></seealso>, + <seealso marker="socket#recvmsg_async"><c>recvmsg/2,3,5</c></seealso>, + <seealso marker="socket#send_async"><c>send/3,4</c></seealso>, + <seealso marker="socket#sendmsg_async"><c>sendmsg/3,4</c></seealso> and + <seealso marker="socket#sendto_async"><c>sendto/4,5</c></seealso>). + This is achieved by setting the <c>Timeout</c> argument to + <c>nowait</c>. For instance, if calling the + <seealso marker="socket#recv_async"><c>recv/3</c></seealso> + function with Timeout set to <c>nowait</c> (i.e. + <c>recv(Sock, 0, nowait)</c>) + when there is actually nothing to read, it will return with + <c>{ok, </c> + <seealso marker="socket#type-select_info"><c>SelectInfo</c></seealso><c>}</c>. + When data eventually arrives a 'select message' + will be sent to the caller:</p> + <taglist> + <!-- NOTE THAT THE EMPTY TAG IS INTENTIONAL --> + <tag></tag> + <item><c>{'$socket', socket(), select, select_ref()}</c></item> + </taglist> + <p>The caller can then make another + call to the recv function and now expect data.</p> + <p>The user must also be prepared to receive an abort message: </p> + <!-- + <list type="ordered"> + <item><c>{'$socket', Sock, abort, Info}</c></item> + </list> + --> + <taglist> + <!-- NOTE THAT THE EMPTY TAG IS INTENTIONAL --> + <tag></tag> + <item><c>{'$socket', socket(), abort, Info}</c></item> + </taglist> + <p>If the operation is aborted + for whatever reason (e.g. if the socket is closed "by someone else"). + The <c>Info</c> part contains the abort reason (in this case that + the socket has been closed <c>Info = {select_ref(), closed}</c>). </p> + <p>Note that all other users are <em>locked out</em> until the + 'current user' has called the function (recv in this case). So either + immediately call the function or + <seealso marker="socket#cancel/2"><c>cancel</c></seealso>. </p> + + <p>The general form of the 'socket' message is: </p> + <taglist> + <!-- NOTE THAT THE EMPTY TAG IS INTENTIONAL --> + <tag></tag> + <item><c>{'$socket', Sock :: socket(), Tag :: atom(), Info :: term()}</c></item> + </taglist> + <p>Where the format of <c>Info</c> is a function of <c>Tag</c>:</p> + <table> + <row> + <cell><em>Tag</em></cell> + <cell><em>Info value type</em></cell> + </row> + <row> + <cell>select</cell> + <cell>select_ref()</cell> + </row> + <row> + <cell>abort</cell> + <cell>{select_ref(), Reason :: term()}</cell> + </row> + <tcaption>socket message info value type</tcaption> + </table> + <p>The <c>select_ref()</c> is the same as was received in the + <seealso marker="socket#type-select_info"><c>SelectInfo</c></seealso>. </p> + </section> </section> <section> diff --git a/erts/doc/src/zlib.xml b/erts/doc/src/zlib.xml index 38229456c9..8a580c8d7b 100644 --- a/erts/doc/src/zlib.xml +++ b/erts/doc/src/zlib.xml @@ -411,7 +411,7 @@ list_to_binary([B1,B2])</pre> <func> <name name="deflateParams" arity="3" since=""/> - <fsummary>Dynamicly update deflate parameters.</fsummary> + <fsummary>Dynamically update deflate parameters.</fsummary> <desc> <p>Dynamically updates the compression level and compression strategy. The interpretation of <c><anno>Level</anno></c> and diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index 80e871aaf6..6379e4e04d 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -84,8 +84,9 @@ process_info(fmtfn_t to, void *to_arg) * they are most likely just created and has invalid data */ if (p->heap != NULL) { - ErtsProcLocks locks = (p == esdp->current_process || - p == esdp->free_process) ? ERTS_PROC_LOCK_MAIN : 0; + ErtsProcLocks locks = ((esdp && (p == esdp->current_process || + p == esdp->free_process)) + ? ERTS_PROC_LOCK_MAIN : 0); print_process_info(to, to_arg, p, locks); } } diff --git a/erts/emulator/beam/dist.c b/erts/emulator/beam/dist.c index ff19ef018e..d8501ea6ac 100644 --- a/erts/emulator/beam/dist.c +++ b/erts/emulator/beam/dist.c @@ -2055,10 +2055,17 @@ int erts_net_message(Port *prt, token = tuple[4]; } if (is_not_pid(from) - || dep != external_pid_dist_entry(from) - || is_not_internal_pid(to)) { + || dep != external_pid_dist_entry(from)) { goto invalid_message; } + if (is_not_internal_pid(to)) { + if (is_external_pid(to)) { + DistEntry *dep = external_pid_dist_entry(to); + if (dep == erts_this_dist_entry) + break; /* Old incarnation of this node... */ + } + goto invalid_message; + } if (!erts_proc_lookup(to)) { if (ede_hfrag != NULL) { @@ -3368,7 +3375,7 @@ dist_get_stat_1(BIF_ALIST_1) am_ok, erts_bld_sint64(hpp, szp, read), erts_bld_sint64(hpp, szp, write), - pend ? am_true : am_false); + erts_bld_sint64(hpp, szp, pend)); if (hpp) break; hp = HAlloc(BIF_P, sz); diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 2704b99aa4..0339589b79 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -768,7 +768,7 @@ static ErtsProcessInfoArgs pi_args[] = { {am_memory, 0, ERTS_PI_FLAG_NEED_MSGQ_LEN|ERTS_PI_FLAG_FORCE_SIG_SEND, ERTS_PROC_LOCK_MAIN}, {am_garbage_collection, 3+2 + 3+2 + 3+2 + 3+2 + 3+2 + ERTS_MAX_HEAP_SIZE_MAP_SZ, 0, ERTS_PROC_LOCK_MAIN}, {am_group_leader, 0, 0, ERTS_PROC_LOCK_MAIN}, - {am_reductions, 0, ERTS_PI_FLAG_FORCE_SIG_SEND, ERTS_PROC_LOCK_MAIN}, + {am_reductions, 0, 0, ERTS_PROC_LOCK_MAIN}, {am_priority, 0, 0, 0}, {am_trace, 0, 0, ERTS_PROC_LOCK_MAIN}, {am_binary, 0, ERTS_PI_FLAG_FORCE_SIG_SEND, ERTS_PROC_LOCK_MAIN}, diff --git a/erts/emulator/beam/erl_db_catree.c b/erts/emulator/beam/erl_db_catree.c index e0d5e44f58..fed4b44a9b 100644 --- a/erts/emulator/beam/erl_db_catree.c +++ b/erts/emulator/beam/erl_db_catree.c @@ -166,8 +166,17 @@ static void split_catree(DbTableCATree *tb, static void join_catree(DbTableCATree *tb, DbTableCATreeNode *thiz, DbTableCATreeNode *parent); - - +static ERTS_INLINE +int try_wlock_base_node(DbTableCATreeBaseNode *base_node); +static ERTS_INLINE +void wunlock_base_node(DbTableCATreeNode *base_node); +static ERTS_INLINE +void wlock_base_node_no_stats(DbTableCATreeNode *base_node); +static ERTS_INLINE +void wunlock_adapt_base_node(DbTableCATree* tb, + DbTableCATreeNode* node, + DbTableCATreeNode* parent, + int current_level); /* ** External interface */ @@ -210,12 +219,16 @@ DbTableMethod db_catree = * Constants */ -#define ERL_DB_CATREE_LOCK_FAILURE_CONTRIBUTION 200 +#define ERL_DB_CATREE_LOCK_FAILURE_CONTRIBUTION 250 #define ERL_DB_CATREE_LOCK_SUCCESS_CONTRIBUTION (-1) +#define ERL_DB_CATREE_LOCK_GRAVITY_CONTRIBUTION (-500) +#define ERL_DB_CATREE_LOCK_GRAVITY_PATTERN (0xFF800000) #define ERL_DB_CATREE_LOCK_MORE_THAN_ONE_CONTRIBUTION (-10) #define ERL_DB_CATREE_HIGH_CONTENTION_LIMIT 1000 #define ERL_DB_CATREE_LOW_CONTENTION_LIMIT (-1000) -#define ERL_DB_CATREE_MAX_ROUTE_NODE_LAYER_HEIGHT 14 +#define ERL_DB_CATREE_MAX_ROUTE_NODE_LAYER_HEIGHT 16 +#define ERL_DB_CATREE_LOCK_LOW_NO_CONTRIBUTION_LIMIT (-20000) +#define ERL_DB_CATREE_LOCK_HIGH_NO_CONTRIBUTION_LIMIT (20000) /* * Internal CA tree related helper functions and macros @@ -245,6 +258,27 @@ DbTableMethod db_catree = #define SET_LEFT_RELB(ca_tree_route_node, v) erts_atomic_set_relb(&(ca_tree_route_node->u.route.left), (erts_aint_t)(v)); #define SET_RIGHT_RELB(ca_tree_route_node, v) erts_atomic_set_relb(&(ca_tree_route_node->u.route.right), (erts_aint_t)(v)); +/* Change base node lock statistics */ +#define BASE_NODE_STAT_SET(NODE, VALUE) erts_atomic_set_nob(&(NODE)->u.base.lock_statistics, VALUE) +#define BASE_NODE_STAT_READ(NODE) erts_atomic_read_nob(&(NODE)->u.base.lock_statistics) +#define BASE_NODE_STAT_ADD(NODE, VALUE) \ + do { \ + Sint v = erts_atomic_read_nob(&((NODE)->u.base.lock_statistics)); \ + ASSERT(VALUE > 0); \ + if(v < ERL_DB_CATREE_LOCK_HIGH_NO_CONTRIBUTION_LIMIT) { \ + erts_atomic_set_nob(&(NODE->u.base.lock_statistics), v + VALUE); \ + } \ + }while(0); +#define BASE_NODE_STAT_SUB(NODE, VALUE) \ + do { \ + Sint v = erts_atomic_read_nob(&((NODE)->u.base.lock_statistics)); \ + ASSERT(VALUE < 0); \ + if(v > ERL_DB_CATREE_LOCK_LOW_NO_CONTRIBUTION_LIMIT) { \ + erts_atomic_set_nob(&(NODE->u.base.lock_statistics), v + VALUE); \ + } \ + }while(0); + + /* Compares a key to the key in a route node */ static ERTS_INLINE Sint cmp_key_route(Eterm key, DbTableCATreeNode *obj) @@ -653,10 +687,10 @@ static void dbg_provoke_random_splitjoin(DbTableCATree* tb, switch (dbg_fastrand() % 8) { case 1: - base_node->u.base.lock_statistics = 1+ERL_DB_CATREE_HIGH_CONTENTION_LIMIT; + BASE_NODE_STAT_ADD(base_node, 1+ERL_DB_CATREE_HIGH_CONTENTION_LIMIT); break; case 2: - base_node->u.base.lock_statistics = -1+ERL_DB_CATREE_LOW_CONTENTION_LIMIT; + BASE_NODE_STAT_SUB(base_node, -1+ERL_DB_CATREE_LOW_CONTENTION_LIMIT); break; } } @@ -664,6 +698,48 @@ static void dbg_provoke_random_splitjoin(DbTableCATree* tb, # define dbg_provoke_random_splitjoin(T,N) #endif /* PROVOKE_RANDOM_SPLIT_JOIN */ +static ERTS_NOINLINE +void do_random_join(DbTableCATree* tb, Uint rand) +{ + DbTableCATreeNode* node = GET_ROOT_ACQB(tb); + DbTableCATreeNode* parent = NULL; + int level = 0; + Sint stat; + while (!node->is_base_node) { + parent = node; + if ((rand & (1 << level)) == 0) { + node = GET_LEFT_ACQB(node); + } else { + node = GET_RIGHT_ACQB(node); + } + level++; + } + BASE_NODE_STAT_SUB(node, ERL_DB_CATREE_LOCK_GRAVITY_CONTRIBUTION); + stat = BASE_NODE_STAT_READ(node); + if (stat >= ERL_DB_CATREE_LOW_CONTENTION_LIMIT && + stat <= ERL_DB_CATREE_HIGH_CONTENTION_LIMIT) { + return; /* No adaptation */ + } + if (parent != NULL && !try_wlock_base_node(&node->u.base)) { + if (!node->u.base.is_valid) { + wunlock_base_node(node); + return; + } + wunlock_adapt_base_node(tb, node, parent, level); + } +} + +static ERTS_INLINE +void do_random_join_with_low_probability(DbTableCATree* tb, Uint seed) +{ +#ifndef ERTS_DB_CA_TREE_NO_RANDOM_JOIN_WITH_LOW_PROBABILITY + Uint32 rand = erts_sched_local_random(seed); + if (((rand & ERL_DB_CATREE_LOCK_GRAVITY_PATTERN)) == 0) { + do_random_join(tb, rand); + } +#endif +} + static ERTS_INLINE int try_wlock_base_node(DbTableCATreeBaseNode *base_node) { @@ -691,9 +767,9 @@ void wlock_base_node(DbTableCATreeNode *base_node) if (try_wlock_base_node(&base_node->u.base)) { /* The lock is contended */ wlock_base_node_no_stats(base_node); - base_node->u.base.lock_statistics += ERL_DB_CATREE_LOCK_FAILURE_CONTRIBUTION; + BASE_NODE_STAT_ADD(base_node, ERL_DB_CATREE_LOCK_FAILURE_CONTRIBUTION); } else { - base_node->u.base.lock_statistics += ERL_DB_CATREE_LOCK_SUCCESS_CONTRIBUTION; + BASE_NODE_STAT_SUB(base_node, ERL_DB_CATREE_LOCK_SUCCESS_CONTRIBUTION); } } @@ -709,13 +785,14 @@ void wunlock_adapt_base_node(DbTableCATree* tb, DbTableCATreeNode* parent, int current_level) { + Sint base_node_lock_stat = BASE_NODE_STAT_READ(node); dbg_provoke_random_splitjoin(tb,node); if ((!node->u.base.root && parent && !(tb->common.status & DB_CATREE_FORCE_SPLIT)) - || node->u.base.lock_statistics < ERL_DB_CATREE_LOW_CONTENTION_LIMIT) { + || base_node_lock_stat < ERL_DB_CATREE_LOW_CONTENTION_LIMIT) { join_catree(tb, node, parent); } - else if (node->u.base.lock_statistics > ERL_DB_CATREE_HIGH_CONTENTION_LIMIT + else if (base_node_lock_stat > ERL_DB_CATREE_HIGH_CONTENTION_LIMIT && current_level < ERL_DB_CATREE_MAX_ROUTE_NODE_LAYER_HEIGHT) { split_catree(tb, node, parent); } @@ -728,11 +805,23 @@ static ERTS_INLINE void rlock_base_node(DbTableCATreeNode *base_node) { ASSERT(base_node->is_base_node); - erts_rwmtx_rlock(&base_node->u.base.lock); + if (EBUSY == erts_rwmtx_tryrlock(&base_node->u.base.lock)) { + /* The lock is contended */ + BASE_NODE_STAT_ADD(base_node, ERL_DB_CATREE_LOCK_FAILURE_CONTRIBUTION); + erts_rwmtx_rlock(&base_node->u.base.lock); + } +} + +static ERTS_INLINE +void runlock_base_node(DbTableCATreeNode *base_node, DbTableCATree* tb) +{ + ASSERT(base_node->is_base_node); + erts_rwmtx_runlock(&base_node->u.base.lock); + do_random_join_with_low_probability(tb, (Uint)base_node); } static ERTS_INLINE -void runlock_base_node(DbTableCATreeNode *base_node) +void runlock_base_node_no_rand(DbTableCATreeNode *base_node) { ASSERT(base_node->is_base_node); erts_rwmtx_runlock(&base_node->u.base.lock); @@ -814,7 +903,7 @@ void unlock_iter_base_node(CATreeRootIterator* iter) { ASSERT(iter->locked_bnode); if (iter->read_only) - runlock_base_node(iter->locked_bnode); + runlock_base_node(iter->locked_bnode, iter->tb); else if (iter->locked_bnode->u.base.is_valid) { wunlock_adapt_base_node(iter->tb, iter->locked_bnode, iter->bnode_parent, iter->bnode_level); @@ -874,7 +963,7 @@ DbTableCATreeNode* find_rlock_valid_base_node(DbTableCATree* tb, Eterm key) rlock_base_node(base_node); if (base_node->u.base.is_valid) break; - runlock_base_node(base_node); + runlock_base_node_no_rand(base_node); } return base_node; } @@ -923,8 +1012,8 @@ static DbTableCATreeNode *create_base_node(DbTableCATree *tb, "erl_db_catree_base_node", NIL, ERTS_LOCK_FLAGS_CATEGORY_DB); - p->u.base.lock_statistics = ((tb->common.status & DB_CATREE_FORCE_SPLIT) - ? INT_MAX : 0); + BASE_NODE_STAT_SET(p, ((tb->common.status & DB_CATREE_FORCE_SPLIT) + ? INT_MAX : 0)); p->u.base.is_valid = 1; return p; } @@ -1094,7 +1183,7 @@ static void join_catree(DbTableCATree *tb, ASSERT(thiz->is_base_node); if (parent == NULL) { - thiz->u.base.lock_statistics = 0; + BASE_NODE_STAT_SET(thiz, 0); wunlock_base_node(thiz); return; } @@ -1103,11 +1192,11 @@ static void join_catree(DbTableCATree *tb, neighbor = leftmost_base_node(GET_RIGHT_ACQB(parent)); if (try_wlock_base_node(&neighbor->u.base)) { /* Failed to acquire lock */ - thiz->u.base.lock_statistics = 0; + BASE_NODE_STAT_SET(thiz, 0); wunlock_base_node(thiz); return; } else if (!neighbor->u.base.is_valid) { - thiz->u.base.lock_statistics = 0; + BASE_NODE_STAT_SET(thiz, 0); wunlock_base_node(thiz); wunlock_base_node(neighbor); return; @@ -1153,11 +1242,11 @@ static void join_catree(DbTableCATree *tb, neighbor = rightmost_base_node(GET_LEFT_ACQB(parent)); if (try_wlock_base_node(&neighbor->u.base)) { /* Failed to acquire lock */ - thiz->u.base.lock_statistics = 0; + BASE_NODE_STAT_SET(thiz, 0); wunlock_base_node(thiz); return; } else if (!neighbor->u.base.is_valid) { - thiz->u.base.lock_statistics = 0; + BASE_NODE_STAT_SET(thiz, 0); wunlock_base_node(thiz); wunlock_base_node(neighbor); return; @@ -1241,7 +1330,7 @@ static void split_catree(DbTableCATree *tb, if (less_than_two_elements(base->u.base.root)) { if (!(tb->common.status & DB_CATREE_FORCE_SPLIT)) - base->u.base.lock_statistics = 0; + BASE_NODE_STAT_SET(base, 0); wunlock_base_node(base); return; } else { @@ -1521,7 +1610,7 @@ static int db_get_catree(Process *p, DbTable *tbl, Eterm key, Eterm *ret) int result = db_get_tree_common(p, &tb->common, node->u.base.root, key, ret, NULL); - runlock_base_node(node); + runlock_base_node(node, tb); return result; } @@ -1804,7 +1893,7 @@ static int db_member_catree(DbTable *tbl, Eterm key, Eterm *ret) int result = db_member_tree_common(&tb->common, node->u.base.root, key, ret, NULL); - runlock_base_node(node); + runlock_base_node(node, tb); return result; } @@ -1816,7 +1905,7 @@ static int db_get_element_catree(Process *p, DbTable *tbl, int result = db_get_element_tree_common(p, &tb->common, node->u.base.root, key, ndex, ret, NULL); - runlock_base_node(node); + runlock_base_node(node, tb); return result; } @@ -2250,7 +2339,7 @@ void db_catree_force_split(DbTableCATree* tb, int on) init_root_iterator(tb, &iter, 1); root = catree_find_first_root(&iter); do { - iter.locked_bnode->u.base.lock_statistics = (on ? INT_MAX : 0); + BASE_NODE_STAT_SET(iter.locked_bnode, (on ? INT_MAX : 0)); root = catree_find_next_root(&iter, NULL); } while (root); destroy_root_iterator(&iter); diff --git a/erts/emulator/beam/erl_db_catree.h b/erts/emulator/beam/erl_db_catree.h index cf3498dabb..c2c884eee3 100644 --- a/erts/emulator/beam/erl_db_catree.h +++ b/erts/emulator/beam/erl_db_catree.h @@ -42,7 +42,7 @@ typedef struct { typedef struct { erts_rwmtx_t lock; /* The lock for this base node */ - Sint lock_statistics; + erts_atomic_t lock_statistics; int is_valid; /* If this base node is still valid */ TreeDbTerm *root; /* The root of the sequential tree */ ErtsThrPrgrLaterOp free_item; /* Used when freeing using thread progress */ diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index bbf50b4189..0d6b512f78 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -2632,6 +2632,9 @@ void erts_notify_inc_runq(ErtsRunQueue *runq); void erts_sched_finish_poke(ErtsSchedulerSleepInfo *, erts_aint32_t); ERTS_GLB_INLINE void erts_sched_poke(ErtsSchedulerSleepInfo *ssi); void erts_aux_thread_poke(void); +ERTS_GLB_INLINE Uint32 erts_sched_local_random_hash_64_to_32_shift(Uint64 key); +ERTS_GLB_INLINE Uint32 erts_sched_local_random(Uint additional_seed); + #if ERTS_GLB_INLINE_INCL_FUNC_DEF @@ -2648,6 +2651,39 @@ erts_sched_poke(ErtsSchedulerSleepInfo *ssi) } +/* + * Source: https://gist.github.com/badboy/6267743 + * http://web.archive.org/web/20071223173210/http://www.concentric.net/~Ttwang/tech/inthash.htm + */ +ERTS_GLB_INLINE +Uint32 erts_sched_local_random_hash_64_to_32_shift(Uint64 key) +{ + key = (~key) + (key << 18); /* key = (key << 18) - key - 1; */ + key = key ^ (key >> 31); + key = (key + (key << 2)) + (key << 4); + key = key ^ (key >> 11); + key = key + (key << 6); + key = key ^ (key >> 22); + return (Uint32) key; +} + +/* + * This function attempts to return a random number based on the state + * of the scheduler, the current process and the additional_seed + * parameter. + */ +ERTS_GLB_INLINE +Uint32 erts_sched_local_random(Uint additional_seed) +{ + ErtsSchedulerData *esdp = erts_get_scheduler_data(); + Uint64 seed = + additional_seed + + esdp->reductions + + esdp->current_process->fcalls + + (((Uint64)esdp->no) << 32); + return erts_sched_local_random_hash_64_to_32_shift(seed); +} + #endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ diff --git a/erts/emulator/beam/erl_trace.c b/erts/emulator/beam/erl_trace.c index c85a7df5ec..9c835ac357 100644 --- a/erts/emulator/beam/erl_trace.c +++ b/erts/emulator/beam/erl_trace.c @@ -713,7 +713,9 @@ trace_sched(Process *p, ErtsProcLocks locks, Eterm what) trace_sched_aux(p, locks, what); } -/* Send {trace_ts, Pid, Send, Msg, DestPid, Timestamp} +/* Send {trace_ts, Pid, Send, Msg, DestPid, PamResult, Timestamp} + * or {trace_ts, Pid, Send, Msg, DestPid, Timestamp} + * or {trace, Pid, Send, Msg, DestPid, PamResult} * or {trace, Pid, Send, Msg, DestPid} * * where 'Send' is 'send' or 'send_to_non_existing_process'. @@ -773,7 +775,9 @@ trace_send(Process *p, Eterm to, Eterm msg) erts_match_set_release_result_trace(p, pam_result); } -/* Send {trace_ts, Pid, receive, Msg, Timestamp} +/* Send {trace_ts, Pid, receive, Msg, PamResult, Timestamp} + * or {trace_ts, Pid, receive, Msg, Timestamp} + * or {trace, Pid, receive, Msg, PamResult} * or {trace, Pid, receive, Msg} */ void diff --git a/erts/emulator/beam/ops.tab b/erts/emulator/beam/ops.tab index 10ca74cd60..9d68a4a68b 100644 --- a/erts/emulator/beam/ops.tab +++ b/erts/emulator/beam/ops.tab @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2018. All Rights Reserved. +# Copyright Ericsson AB 1997-2019. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -699,13 +699,18 @@ is_tuple NotTupleFail Tuple=x | is_tagged_tuple WrongRecordFail Tuple Arity Atom is_tagged_tuple_ff f? f? rx A a -get_tuple_element Reg=x P1 D1=x | get_tuple_element Reg=x P2 D2=x | \ +get_tuple_element Reg=x P1 D1=x | \ + get_tuple_element Reg=x P2 D2=x | \ get_tuple_element Reg=x P3 D3=x | \ - succ(P1, P2) | succ(P2, P3) | \ - succ(D1, D2) | succ(D2, D3) => i_get_tuple_element3 Reg P1 D1 - -get_tuple_element Reg=x P1 D1=x | get_tuple_element Reg=x P2 D2=x | \ - succ(P1, P2) | succ(D1, D2) => i_get_tuple_element2 Reg P1 D1 + succ(P1, P2) | succ(P2, P3) | succ(D1, D2) | succ(D2, D3) | \ + distinct(D1, Reg) | distinct(D2, Reg) => \ + i_get_tuple_element3 Reg P1 D1 + +get_tuple_element Reg=x P1 D1=x | \ + get_tuple_element Reg=x P2 D2=x | \ + succ(P1, P2) | succ(D1, D2) | \ + distinct(D1, Reg) => \ + i_get_tuple_element2 Reg P1 D1 get_tuple_element Reg=x P1 D1=x | get_tuple_element Reg=x P2 D2=x | \ succ(P1, P2) | distinct(D1, Reg) => i_get_tuple_element2_dst Reg P1 D1 D2 diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index c93966d24f..311c5fdd6a 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2018. All Rights Reserved. + * Copyright Ericsson AB 1997-2019. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -1077,6 +1077,7 @@ typedef struct { long forced_events; /* Mask of events that are forcefully signalled on windows see winsock_event_select for details */ + int err; /* Keeps error code for closed socket */ int send_would_block; /* Last send attempt failed with "WOULDBLOCK" */ #endif ErlDrvPort port; /* the port identifier */ @@ -3379,6 +3380,71 @@ static int udp_parse_ancillary_data(ErlDrvTermData *spec, int i, i = LOAD_NIL(spec, i); return LOAD_LIST(spec, i, n+1); } + +static int compile_ancillary_data(struct msghdr *mhdr, + char *ptr, ErlDrvSizeT anc_len) { + struct cmsghdr *cmsg; + size_t controllen = 0; + cmsg = CMSG_FIRSTHDR(mhdr); + for (;;) { + if (anc_len == 0) { + /* End of options to compile */ + mhdr->msg_controllen = controllen; + return 0; + } + if (cmsg == NULL) { + /* End of destination before end of options */ + return 1; + } + +#define COMPILE_ANCILLARY_DATA_ITEM(Level, Opt, Type, Get, Size) \ + do { \ + if (anc_len < (Size)) return 1; \ + sys_memset(cmsg, '\0', CMSG_SPACE(sizeof(Type))); \ + cmsg->cmsg_level = Level; \ + cmsg->cmsg_type = Opt; \ + cmsg->cmsg_len = CMSG_LEN(sizeof(Type)); \ + *((Type *) CMSG_DATA(cmsg)) = Get(ptr); \ + controllen += CMSG_SPACE(sizeof(Type)); \ + cmsg = CMSG_NXTHDR(mhdr, cmsg); \ + ptr += 4; \ + anc_len -= 4; \ + } while (0) +#define SIZEOF_ANCILLARY_DATA (2 * CMSG_SPACE(sizeof(int))) + /* (IP_TOS | IPV6_TCLASS) + IP_TTL */ + + switch (anc_len--, *ptr++) { + case INET_OPT_TOS: { +#if defined(IPPROTO_IP) && defined(IP_TOS) + COMPILE_ANCILLARY_DATA_ITEM(IPPROTO_IP, IP_TOS, int, get_int32, 4); +#else + return 1; /* Socket option not implemented */ +#endif + break; + } + case INET_OPT_TTL: { +#if defined(IPPROTO_IP) && defined(IP_TTL) + COMPILE_ANCILLARY_DATA_ITEM(IPPROTO_IP, IP_TTL, int, get_int32, 4); +#else + return 1; /* Socket option not implemented */ +#endif + break; + } + case INET_OPT_TCLASS: { +#if defined(IPPROTO_IPV6) && defined(IPV6_TCLASS) + COMPILE_ANCILLARY_DATA_ITEM(IPPROTO_IPV6, IPV6_TCLASS, int, get_int32, 4); +#else + return 1; /* Socket option not implemented */ +#endif + break; + } + default: + /* Unknow socket option */ + return 1; + } +#undef COMPILE_ANCILLARY_DATA_ITEM + } +} #endif /* ifndef __WIN32__ */ /* @@ -9063,6 +9129,7 @@ static ErlDrvData inet_start(ErlDrvPort port, int size, int protocol) desc->event_mask = 0; #ifdef __WIN32__ desc->forced_events = 0; + desc->err = 0; desc->send_would_block = 0; #endif desc->port = port; @@ -9846,10 +9913,8 @@ static tcp_descriptor* tcp_inet_copy(tcp_descriptor* desc,SOCKET s, copy_desc->send_timeout = desc->send_timeout; copy_desc->send_timeout_close = desc->send_timeout_close; - if (desc->tcp_add_flags & TCP_ADDF_SHOW_ECONNRESET) - copy_desc->tcp_add_flags |= TCP_ADDF_SHOW_ECONNRESET; - else - copy_desc->tcp_add_flags &= ~TCP_ADDF_SHOW_ECONNRESET; + copy_desc->tcp_add_flags = desc->tcp_add_flags + & (TCP_ADDF_SHOW_ECONNRESET | TCP_ADDF_LINGER_ZERO); /* The new port will be linked and connected to the original caller */ port = driver_create_port(port, owner, "tcp_inet", (ErlDrvData) copy_desc); @@ -10914,7 +10979,7 @@ static int winsock_event_select(inet_descriptor *desc, int flags, int on) { int save_event_mask = desc->event_mask; - desc->forced_events = 0; + desc->forced_events &= FD_CLOSE; if (on) desc->event_mask |= flags; else @@ -10966,7 +11031,7 @@ static int winsock_event_select(inet_descriptor *desc, int flags, int on) TIMEVAL tmo = {0,0}; FD_SET fds; int ret; - + FD_ZERO(&fds); FD_SET(desc->s,&fds); do_force = (select(desc->s+1,0,&fds,0,&tmo) > 0); @@ -10983,7 +11048,7 @@ static int winsock_event_select(inet_descriptor *desc, int flags, int on) FD_SET fds; int ret; unsigned long arg; - + FD_ZERO(&fds); FD_SET(desc->s,&fds); ret = select(desc->s+1,&fds,0,0,&tmo); @@ -11022,13 +11087,16 @@ static void tcp_inet_event(ErlDrvData e, ErlDrvEvent event) goto error; } - DEBUGF((" => event=%02X, mask=%02X\r\n", - netEv.lNetworkEvents, desc->inet.event_mask)); + DEBUGF((" => event=%02X, mask=%02X, forced=%02X\r\n", + netEv.lNetworkEvents, desc->inet.event_mask, + desc->inet.forced_events)); /* Add the forced events. */ - netEv.lNetworkEvents |= desc->inet.forced_events; + if (desc->inet.forced_events & FD_CLOSE) + netEv.iErrorCode[FD_CLOSE_BIT] = desc->inet.err; + /* * Calling WSAEventSelect() with a mask of 0 doesn't always turn off * all events. To avoid acting on events we don't want, we mask @@ -11048,16 +11116,29 @@ static void tcp_inet_event(ErlDrvData e, ErlDrvEvent event) goto error; } if (netEv.lNetworkEvents & FD_CLOSE) { - /* + + /* We may not get any more FD_CLOSE events so we + keep this event and always signal it from + this moment on. */ + if ((desc->inet.forced_events & FD_CLOSE) == 0) { + desc->inet.forced_events |= FD_CLOSE; + desc->inet.err = netEv.iErrorCode[FD_CLOSE_BIT]; + } + + /* * We must loop to read out the remaining packets (if any). */ for (;;) { - DEBUGF(("Retrying read due to closed port\r\n")); - /* XXX The buffer will be thrown away on error (empty que). - Possible SMP FIXME. */ - if (!desc->inet.active && (desc->inet.opt) == NULL) { - goto error; - } + + /* if passive and no subscribers, break loop */ + if (!desc->inet.active && desc->inet.opt == NULL) { + /* do not trigger close event when socket is + transitioned to passive */ + netEv.lNetworkEvents &= ~FD_CLOSE; + break; + } + + DEBUGF(("Retrying read due to FD_CLOSE\r\n")); if (tcp_inet_input(desc, event) < 0) { goto error; } @@ -11386,7 +11467,7 @@ static int tcp_shutdown_error(tcp_descriptor* desc, int err) static void tcp_inet_delay_send(ErlDrvData data, ErlDrvTermData dummy) { tcp_descriptor *desc = (tcp_descriptor*)data; - (void)tcp_inet_output(desc, INETP(desc)->s); + (void)tcp_inet_output(desc, (HANDLE) INETP(desc)->s); } /* @@ -12553,7 +12634,7 @@ static void packet_inet_timeout(ErlDrvData e) sock_select(desc, FD_READ, 0); async_error_am (desc, am_timeout); } else { - (void)packet_inet_input(udesc, desc->s); + (void)packet_inet_input(udesc, (HANDLE) desc->s); } } @@ -12597,11 +12678,8 @@ static void packet_inet_command(ErlDrvData e, char* buf, ErlDrvSizeT len) char ancd[CMSG_SPACE(sizeof(*sri))]; } cmsg; - if (len < SCTP_GET_SENDPARAMS_LEN) { - inet_reply_error(desc, EINVAL); - return; - } - + if (len < SCTP_GET_SENDPARAMS_LEN) goto return_einval; + /* The ancillary data */ sri = (struct sctp_sndrcvinfo *) (CMSG_DATA(&cmsg.hdr)); /* Get the "sndrcvinfo" from the buffer, advancing the "ptr": */ @@ -12634,28 +12712,85 @@ static void packet_inet_command(ErlDrvData e, char* buf, ErlDrvSizeT len) goto check_result_code; } #endif - /* UDP socket. Even if it is connected, there is an address prefix - here -- ignored for connected sockets: */ - sz = len; - qtr = ptr; - xerror = inet_set_faddress(desc->sfamily, &other, &qtr, &sz); - if (xerror != NULL) { - inet_reply_error_am(desc, driver_mk_atom(xerror)); - return; - } - len -= (qtr - ptr); - ptr = qtr; - /* Now "ptr" is the user data ptr, "len" is data length: */ - inet_output_count(desc, len); - - if (desc->state & INET_F_ACTIVE) { /* connected (ignore address) */ - code = sock_send(desc->s, ptr, len, 0); - } - else { - code = sock_sendto(desc->s, ptr, len, 0, &other.sa, sz); + { + ErlDrvSizeT anc_len; + + /* UDP socket. Even if it is connected, there is an address prefix + here -- ignored for connected sockets: */ + sz = len; + qtr = ptr; + xerror = inet_set_faddress(desc->sfamily, &other, &qtr, &sz); + if (xerror != NULL) { + inet_reply_error_am(desc, driver_mk_atom(xerror)); + return; + } + len -= (qtr - ptr); + ptr = qtr; + + /* Here comes ancillary data */ + if (len < 4) goto return_einval; + anc_len = get_int32(ptr); + len -= 4; ptr += 4; + if (len < anc_len) goto return_einval; + + if (anc_len == 0 && !!0/*XXX-short-circuit-for-testing*/) { + /* Empty ancillary data */ + /* Now "ptr" is the user data ptr, "len" is data length: */ + inet_output_count(desc, len); + if (desc->state & INET_F_ACTIVE) { + /* connected (ignore address) */ + code = sock_send(desc->s, ptr, len, 0); + } + else { + code = sock_sendto(desc->s, ptr, len, 0, &other.sa, sz); + } + } + else { +#ifdef __WIN32__ + goto return_einval; /* Can not send ancillary data on Windows */ +#else + struct iovec iov[1]; + struct msghdr mhdr; + union { /* For ancillary data */ + struct cmsghdr hdr; + char ancd[SIZEOF_ANCILLARY_DATA]; + } cmsg; + sys_memset(&iov, '\0', sizeof(iov)); + sys_memset(&mhdr, '\0', sizeof(mhdr)); + sys_memset(&cmsg, '\0', sizeof(cmsg)); + if (desc->state & INET_F_ACTIVE) { + /* connected (ignore address) */ + mhdr.msg_name = NULL; + mhdr.msg_namelen = 0; + } + else { + mhdr.msg_name = &other; + mhdr.msg_namelen = sz; + } + mhdr.msg_control = cmsg.ancd; + mhdr.msg_controllen = sizeof(cmsg.ancd); + if (compile_ancillary_data(&mhdr, ptr, anc_len) != 0) { + goto return_einval; + } + if (mhdr.msg_controllen == 0) { + /* XXX Testing - only possible for anc_len == 0 */ + mhdr.msg_control = NULL; + } + len -= anc_len; + ptr += anc_len; + /* Now "ptr" is the user data ptr, "len" is data length: */ + iov[0].iov_len = len; + iov[0].iov_base = ptr; + mhdr.msg_iov = iov; + mhdr.msg_iovlen = 1; + mhdr.msg_flags = 0; + inet_output_count(desc, len); + code = sock_sendmsg(desc->s, &mhdr, 0); +#endif + } } -#ifdef HAVE_SCTP +#ifdef HAVE_SCTP check_result_code: /* "code" analysis is the same for both SCTP and UDP cases above: */ #endif @@ -12665,8 +12800,15 @@ static void packet_inet_command(ErlDrvData e, char* buf, ErlDrvSizeT len) } else inet_reply_ok(desc); + return; + + return_einval: + inet_reply_error(desc, EINVAL); + return; } -#endif + +#endif /* HAVE_UDP */ + #ifdef __WIN32__ static void packet_inet_event(ErlDrvData e, ErlDrvEvent event) diff --git a/erts/emulator/hipe/hipe_arm.c b/erts/emulator/hipe/hipe_arm.c index b61939724c..3e1d7e4d5e 100644 --- a/erts/emulator/hipe/hipe_arm.c +++ b/erts/emulator/hipe/hipe_arm.c @@ -18,6 +18,7 @@ * %CopyrightEnd% */ +#ifdef __arm__ #include <stddef.h> /* offsetof() */ #ifdef HAVE_CONFIG_H @@ -30,24 +31,39 @@ #include "hipe_native_bif.h" /* nbif_callemu() */ #include "hipe_bif0.h" +#ifndef __has_builtin +# define __has_builtin(x) 0 +#endif + /* Flush dcache and invalidate icache for a range of addresses. */ void hipe_flush_icache_range(void *address, unsigned int nbytes) { -#if defined(__ARM_EABI__) + void* end = (char*)address + nbytes; + +#if ERTS_AT_LEAST_GCC_VSN__(4, 3, 0) || __has_builtin(__builtin___clear_cache) + __builtin___clear_cache(address, end); +#elif defined(__clang__) + void __clear_cache(void *start, void *end); + __clear_cache(address, end); +#elif defined(__linux__) +# if defined(__ARM_EABI__) register unsigned long beg __asm__("r0") = (unsigned long)address; - register unsigned long end __asm__("r1") = (unsigned long)address + nbytes; + register unsigned long end __asm__("r1") = (unsigned long)end; register unsigned long flg __asm__("r2") = 0; register unsigned long scno __asm__("r7") = 0xf0002; __asm__ __volatile__("swi 0" /* sys_cacheflush() */ : "=r"(beg) : "0"(beg), "r"(end), "r"(flg), "r"(scno)); -#else +# else register unsigned long beg __asm__("r0") = (unsigned long)address; - register unsigned long end __asm__("r1") = (unsigned long)address + nbytes; + register unsigned long end __asm__("r1") = (unsigned long)end; register unsigned long flg __asm__("r2") = 0; __asm__ __volatile__("swi 0x9f0002" /* sys_cacheflush() */ : "=r"(beg) : "0"(beg), "r"(end), "r"(flg)); +# endif +#else +# error "Don't know how to flush instruction cache" #endif } @@ -270,3 +286,5 @@ void hipe_arch_print_pcb(struct hipe_process_state *p) U("narity ", narity); #undef U } + +#endif /*__arm__*/ diff --git a/erts/emulator/hipe/hipe_x86.h b/erts/emulator/hipe/hipe_x86.h index 8967793171..a1fe75e792 100644 --- a/erts/emulator/hipe/hipe_x86.h +++ b/erts/emulator/hipe/hipe_x86.h @@ -22,17 +22,28 @@ #ifndef HIPE_X86_H #define HIPE_X86_H -static __inline__ void hipe_flush_icache_word(void *address) -{ - /* Do nothing. This works as long as compiled code is - executed by a single CPU thread. */ -} +#ifndef __has_builtin +# define __has_builtin(x) 0 +#endif static __inline__ void hipe_flush_icache_range(void *address, unsigned int nbytes) { - /* Do nothing. This works as long as compiled code is - executed by a single CPU thread. */ + void* end = (char*)address + nbytes; + +#if ERTS_AT_LEAST_GCC_VSN__(4, 3, 0) || __has_builtin(__builtin___clear_cache) + __builtin___clear_cache(address, end); +#elif defined(__clang__) + void __clear_cache(void *start, void *end); + __clear_cache(address, end); +#else +# warning "Don't know how to flush instruction cache" +#endif +} + +static __inline__ void hipe_flush_icache_word(void *address) +{ + hipe_flush_icache_range(address, sizeof(void*)); } /* for stack descriptor hash lookup */ diff --git a/erts/emulator/nifs/common/socket_nif.c b/erts/emulator/nifs/common/socket_nif.c index 25bc712949..adecbb3b6e 100644 --- a/erts/emulator/nifs/common/socket_nif.c +++ b/erts/emulator/nifs/common/socket_nif.c @@ -429,6 +429,7 @@ static void (*esock_sctp_freepaddrs)(struct sockaddr *addrs) = NULL; #define SOCKET_STATE_CONNECTING (SOCKET_STATE_OPEN | SOCKET_FLAG_CON) #define SOCKET_STATE_ACCEPTING (SOCKET_STATE_LISTENING | SOCKET_FLAG_ACC) #define SOCKET_STATE_CLOSING (SOCKET_FLAG_CLOSE) +#define SOCKET_STATE_DTOR (0xFFFF) #define IS_CLOSED(d) \ ((d)->state == SOCKET_STATE_CLOSED) @@ -485,6 +486,9 @@ static void (*esock_sctp_freepaddrs)(struct sockaddr *addrs) = NULL; #define SOCKET_OPT_VALUE_TYPE_INT 1 #define SOCKET_OPT_VALUE_TYPE_BOOL 2 +#define ESOCK_DESC_PATTERN_CREATED 0x03030303 +#define ESOCK_DESC_PATTERN_DTOR 0xC0C0C0C0 + typedef union { struct { // 0 = not open, 1 = open @@ -808,6 +812,14 @@ typedef struct { typedef struct { + /* + * +++ This is a way to, possibly, detect memory overrides "and stuff" +++ + * + * We have two patterns. One is set when the descriptor is created (allocated) + * and one is set when the descriptor is dtor'ed. + */ + Uint32 pattern; + /* +++ The actual socket +++ */ SOCKET sock; HANDLE event; @@ -825,6 +837,10 @@ typedef struct { ErlNifPid ctrlPid; ESockMonitor ctrlMon; + /* +++ Connector process +++ */ + ErlNifPid connPid; + ESockMonitor connMon; + /* +++ Write stuff +++ */ ErlNifMutex* writeMtx; ESockRequestor currentWriter; @@ -2377,6 +2393,8 @@ static int socket_setopt(int sock, const void* optVal, const socklen_t optLen); +static BOOLEAN_T is_connector(ErlNifEnv* env, + ESockDescriptor* descP); static BOOLEAN_T verify_is_connected(ESockDescriptor* descP, int* err); static ESockDescriptor* alloc_descriptor(SOCKET sock, HANDLE event); @@ -4895,17 +4913,17 @@ ERL_NIF_TERM nconnect(ErlNifEnv* env, return esock_make_error(env, atom_closed); if (!IS_OPEN(descP)) { - SSDBG( descP, ("SOCKET", "nif_connect -> not open\r\n") ); + SSDBG( descP, ("SOCKET", "nconnect -> not open\r\n") ); return esock_make_error(env, atom_exbadstate); } if (IS_CONNECTED(descP)) { - SSDBG( descP, ("SOCKET", "nif_connect -> already connected\r\n") ); + SSDBG( descP, ("SOCKET", "nconnect -> already connected\r\n") ); return esock_make_error(env, atom_eisconn); } - if (IS_CONNECTING(descP)) { - SSDBG( descP, ("SOCKET", "nif_connect -> already connecting\r\n") ); + if (IS_CONNECTING(descP) && !is_connector(env, descP)) { + SSDBG( descP, ("SOCKET", "nconnect -> already connecting\r\n") ); return esock_make_error(env, esock_atom_einval); } @@ -4919,31 +4937,93 @@ ERL_NIF_TERM nconnect(ErlNifEnv* env, descP->addrLen); save_errno = sock_errno(); - SSDBG( descP, ("SOCKET", "nif_connect -> connect result: %d, %d\r\n", + SSDBG( descP, ("SOCKET", "nconnect -> connect result: %d, %d\r\n", code, save_errno) ); - if (IS_SOCKET_ERROR(code) && - ((save_errno == ERRNO_BLOCK) || /* Winsock2 */ - (save_errno == EINPROGRESS))) { /* Unix & OSE!! */ - ref = MKREF(env); - descP->state = SOCKET_STATE_CONNECTING; - if ((sres = esock_select_write(env, descP->sock, descP, NULL, - sockRef, ref)) < 0) { - res = esock_make_error(env, - MKT2(env, - esock_atom_select_failed, - MKI(env, sres))); - } else { - res = esock_make_ok2(env, ref); + if (IS_SOCKET_ERROR(code)) { + switch (save_errno) { + case ERRNO_BLOCK: /* Winsock2 */ + case EINPROGRESS: /* Unix & OSE!! */ + SSDBG( descP, ("SOCKET", "nconnect -> would block => select\r\n") ); + + ref = MKREF(env); + + if (IS_CONNECTING(descP)) { + /* Glitch */ + res = esock_make_ok2(env, ref); + } else { + + /* First time here */ + + if (enif_self(env, &descP->connPid) == NULL) + return esock_make_error(env, atom_exself); + + if (MONP("nconnect -> conn", + env, descP, + &descP->connPid, + &descP->connMon) != 0) + return esock_make_error(env, atom_exmon); + + descP->state = SOCKET_STATE_CONNECTING; + + if ((sres = esock_select_write(env, descP->sock, descP, NULL, + sockRef, ref)) < 0) { + res = esock_make_error(env, + MKT2(env, + esock_atom_select_failed, + MKI(env, sres))); + } else { + res = esock_make_ok2(env, ref); + } + } + break; + + case EISCONN: + SSDBG( descP, ("SOCKET", "nconnect -> *already* connected\r\n") ); + { + /* This is ***strange*** so make sure */ + int err = 0; + if (!verify_is_connected(descP, &err)) { + descP->state = SOCKET_STATE_OPEN; /* restore state */ + res = esock_make_error_errno(env, err); + } else { + descP->state = SOCKET_STATE_CONNECTED; + /* And just to be on the safe side, reset these */ + enif_set_pid_undefined(&descP->connPid); + DEMONP("nconnect -> connected", + env, descP, &descP->connMon); + descP->isReadable = TRUE; + descP->isWritable = TRUE; + res = esock_atom_ok; + } + } + break; + + default: + SSDBG( descP, ("SOCKET", "nconnect -> other error(1): %d\r\n", + save_errno) ); + res = esock_make_error_errno(env, save_errno); + break; } + } else if (code == 0) { /* ok we are connected */ + SSDBG( descP, ("SOCKET", "nconnect -> connected\r\n") ); + descP->state = SOCKET_STATE_CONNECTED; + enif_set_pid_undefined(&descP->connPid); + DEMONP("nconnect -> connected", env, descP, &descP->connMon); descP->isReadable = TRUE; descP->isWritable = TRUE; res = esock_atom_ok; + } else { + /* Do we really need this case? */ + + SSDBG( descP, ("SOCKET", "nconnect -> other error(2): %d\r\n", + save_errno) ); + res = esock_make_error_errno(env, save_errno); } @@ -4998,7 +5078,7 @@ ERL_NIF_TERM nfinalize_connection(ErlNifEnv* env, { int error; - if (descP->state != SOCKET_STATE_CONNECTING) + if (!IS_CONNECTING(descP)) return esock_make_error(env, atom_enotconn); if (!verify_is_connected(descP, &error)) { @@ -5007,6 +5087,8 @@ ERL_NIF_TERM nfinalize_connection(ErlNifEnv* env, } descP->state = SOCKET_STATE_CONNECTED; + enif_set_pid_undefined(&descP->connPid); + DEMONP("nfinalize_connection -> connected", env, descP, &descP->connMon); descP->isReadable = TRUE; descP->isWritable = TRUE; @@ -5069,6 +5151,29 @@ BOOLEAN_T verify_is_connected(ESockDescriptor* descP, int* err) +/* *** is_connector *** + * Check if the current process is the connector process. + */ +#if !defined(__WIN32__) +static +BOOLEAN_T is_connector(ErlNifEnv* env, + ESockDescriptor* descP) +{ + ErlNifPid caller; + + if (enif_self(env, &caller) == NULL) + return FALSE; + + if (COMPARE_PIDS(&descP->connPid, &caller) == 0) + return TRUE; + else + return FALSE; + +} +#endif + + + /* ---------------------------------------------------------------------- * nif_listen * @@ -16872,8 +16977,10 @@ ESockDescriptor* alloc_descriptor(SOCKET sock, HANDLE event) if ((descP = enif_alloc_resource(sockets, sizeof(ESockDescriptor))) != NULL) { char buf[64]; /* Buffer used for building the mutex name */ - // This needs to be released when the socket is closed! - // descP->env = enif_alloc_env(); + descP->pattern = ESOCK_DESC_PATTERN_CREATED; + + enif_set_pid_undefined(&descP->connPid); + MON_INIT(&descP->connMon); sprintf(buf, "esock[w,%d]", sock); descP->writeMtx = MCREATE(buf); @@ -18214,21 +18321,30 @@ void socket_dtor(ErlNifEnv* env, void* obj) #if !defined(__WIN32__) ESockDescriptor* descP = (ESockDescriptor*) obj; - MDESTROY(descP->writeMtx); - MDESTROY(descP->readMtx); - MDESTROY(descP->accMtx); - MDESTROY(descP->closeMtx); - MDESTROY(descP->cfgMtx); + MDESTROY(descP->writeMtx); descP->writeMtx = NULL; + MDESTROY(descP->readMtx); descP->readMtx = NULL; + MDESTROY(descP->accMtx); descP->accMtx = NULL; + MDESTROY(descP->closeMtx); descP->closeMtx = NULL; + MDESTROY(descP->cfgMtx); descP->cfgMtx = NULL; - if (descP->currentReader.env) + if (descP->currentReader.env) { esock_free_env("dtor reader", descP->currentReader.env); - if (descP->currentWriter.env) + descP->currentReader.env = NULL; + } + if (descP->currentWriter.env) { esock_free_env("dtor writer", descP->currentWriter.env); - if (descP->currentAcceptor.env) + descP->currentWriter.env = NULL; + } + if (descP->currentAcceptor.env) { esock_free_env("dtor acceptor", descP->currentAcceptor.env); + descP->currentAcceptor.env = NULL; + } free_request_queue(&descP->readersQ); free_request_queue(&descP->writersQ); free_request_queue(&descP->acceptorsQ); + + descP->state = SOCKET_STATE_DTOR; + descP->pattern = ESOCK_DESC_PATTERN_DTOR; #endif } @@ -18652,6 +18768,17 @@ void socket_down(ErlNifEnv* env, MON2T(env, mon)); } + } else if (COMPARE_PIDS(&descP->connPid, pid) == 0) { + + /* The connPid is only set during the connection. + * The same goes for the monitor (connMon). + */ + + descP->state = SOCKET_STATE_OPEN; /* restore state */ + enif_set_pid_undefined(&descP->connPid); + DEMONP("socket_down -> connector", + env, descP, &descP->connMon); + } else { /* check all operation queue(s): acceptor, writer and reader. diff --git a/erts/emulator/test/esock_ttest/esock-ttest b/erts/emulator/test/esock_ttest/esock-ttest index cf1d9cd9ab..9adc51fc8b 100755 --- a/erts/emulator/test/esock_ttest/esock-ttest +++ b/erts/emulator/test/esock_ttest/esock-ttest @@ -60,6 +60,9 @@ usage() -> "~n Which domain to use." "~n Only valid for server." "~n Defaults to: inet" + "~n --async Asynchronous mode (Timeout = nowait)" + "~n This option is only valid for transport = sock." + "~n Also, its only used when active =/= false." "~n --active <active> boolean() | once." "~n Valid for both client and server." "~n Defaults to: false" @@ -111,6 +114,7 @@ process_args(Args) -> process_server_args(Args) -> Defaults = #{role => server, domain => inet, + async => false, active => false, transport => {sock, plain}}, process_server_args(Args, Defaults). @@ -124,6 +128,9 @@ process_server_args(["--domain", Domain|Args], State) (Domain =:= "inet6")) -> process_server_args(Args, State#{domain => list_to_atom(Domain)}); +process_server_args(["--async"|Args], State) -> + process_server_args(Args, State#{async => true}); + process_server_args(["--active", Active|Args], State) when ((Active =:= "false") orelse (Active =:= "once") orelse @@ -145,6 +152,7 @@ process_server_args([Arg|_], _State) -> process_client_args(Args) -> Defaults = #{role => client, + async => false, active => false, transport => {sock, plain}, %% Will cause error if not provided @@ -159,10 +167,13 @@ process_client_args(Args) -> process_client_args([], State) -> process_client_args_ensure_max_outstanding(State); +process_client_args(["--async"|Args], State) -> + process_client_args(Args, State#{async => true}); + process_client_args(["--active", Active|Args], State) - when ((Active =:= "false") orelse - (Active =:= "once") orelse - (Active =:= "true")) -> + when (Active =:= "false") orelse + (Active =:= "once") orelse + (Active =:= "true") -> process_client_args(Args, State#{active => list_to_atom(Active)}); process_client_args(["--transport", "gen" | Args], State) -> @@ -280,9 +291,10 @@ exec(#{role := server, end; exec(#{role := server, domain := Domain, + async := Async, active := Active, transport := {sock, Method}}) -> - case socket_test_ttest_tcp_server_socket:start(Method, Domain, Active) of + case socket_test_ttest_tcp_server_socket:start(Method, Domain, Async, Active) of {ok, {Pid, _}} -> MRef = erlang:monitor(process, Pid), receive @@ -323,12 +335,14 @@ exec(#{role := client, end; exec(#{role := client, server := ServerInfo, + async := Async, active := Active, transport := {sock, Method}, msg_id := MsgID, max_outstanding := MaxOutstanding, runtime := RunTime}) -> case socket_test_ttest_tcp_client_socket:start(true, + Async, Method, ServerInfo, Active, diff --git a/erts/emulator/test/esock_ttest/esock-ttest-server-sock b/erts/emulator/test/esock_ttest/esock-ttest-server-sock index 4ec0d335d9..fc87499f09 100755 --- a/erts/emulator/test/esock_ttest/esock-ttest-server-sock +++ b/erts/emulator/test/esock_ttest/esock-ttest-server-sock @@ -24,9 +24,27 @@ EMU=$ERL_TOP/erts/emulator EMU_TEST=$EMU/test ESOCK_TTEST=$EMU_TEST/esock_ttest -if [ $# = 1 ]; then - ACTIVE="--active $1" +# $1 - async - boolean() +# $2 - active - once | boolean() +if [ $# = 2 ]; then + + async=$1 + active=$2 + + if [ $async = true ]; then + ASYNC="--async" + else + ASYNC= + fi + + ACTIVE="--active $active" + +else + echo "<ERROR> Missing args: async and active" + echo "" + exit 1 fi -$ESOCK_TTEST/esock-ttest --server --transport sock $ACTIVE + +$ESOCK_TTEST/esock-ttest --server $ASYNC --transport sock $ACTIVE diff --git a/erts/emulator/test/process_SUITE.erl b/erts/emulator/test/process_SUITE.erl index b530ced566..3684cde8d4 100644 --- a/erts/emulator/test/process_SUITE.erl +++ b/erts/emulator/test/process_SUITE.erl @@ -1098,42 +1098,86 @@ process_info_status_handled_signal(Config) when is_list(Config) -> %% OTP-15709 %% Provoke a bug where process_info(reductions) returned wrong result %% because REDS_IN (def_arg_reg[5]) is read when the process in not running. +%% +%% And a bug where process_info(reductions) on a process which was releasing its +%% main lock during execution could result in negative reduction diffs. process_info_reductions(Config) when is_list(Config) -> - pi_reductions_tester(spawn_link(fun() -> pi_reductions_spinnloop() end)), - pi_reductions_tester(spawn_link(fun() -> pi_reductions_recvloop() end)), + {S1, S2} = case erlang:system_info(schedulers) of + 1 -> {1,1}; + _ -> {1,2} + end, + io:format("Run on schedulers ~p and ~p\n", [S1,S2]), + Boss = self(), + Doer = spawn_opt(fun () -> + pi_reductions_tester(true, 10, fun pi_reductions_spinnloop/0, S2), + pi_reductions_tester(true, 10, fun pi_reductions_recvloop/0, S2), + pi_reductions_tester(false, 100, fun pi_reductions_main_unlocker/0, S2), + Boss ! {self(), done} + end, + [link, {scheduler, S1}]), + + {Doer, done} = receive M -> M end, ok. -pi_reductions_tester(Pid) -> - {_, DiffList} = - lists:foldl(fun(_, {Prev, Acc}) -> - %% Add another item that force sending the request - %% as a signal, like 'current_function'. - PI = process_info(Pid, [reductions, current_function]), - [{reductions,Reds}, {current_function,_}] = PI, - Diff = Reds - Prev, - {Diff, true} = {Diff, (Diff >= 0)}, - {Diff, true} = {Diff, (Diff =< 1000*1000)}, - {Reds, [Diff | Acc]} - end, - {0, []}, - lists:seq(1,10)), +pi_reductions_tester(ForceSignal, MaxCalls, Fun, S2) -> + Pid = spawn_opt(Fun, [link, {scheduler,S2}]), + Extra = case ForceSignal of + true -> + %% Add another item that force sending the request + %% as a signal, like 'current_function'. + [current_function]; + false -> + [] + end, + LoopFun = fun Me(Calls, Prev, Acc0) -> + PI = process_info(Pid, [reductions | Extra]), + [{reductions,Reds} | _] = PI, + Diff = Reds - Prev, + %% Verify we get sane non-negative reduction diffs + {Diff, true} = {Diff, (Diff >= 0)}, + {Diff, true} = {Diff, (Diff =< 1000*1000)}, + Acc1 = [Diff | Acc0], + case Calls >= MaxCalls of + true -> Acc1; + false -> Me(Calls+1, Reds, Acc1) + end + end, + DiffList = LoopFun(0, 0, []), unlink(Pid), exit(Pid,kill), - io:format("Reduction diffs: ~p\n", [DiffList]), + io:format("Reduction diffs: ~p\n", [lists:reverse(DiffList)]), ok. pi_reductions_spinnloop() -> %% 6 args to make use of def_arg_reg[5] which is also used as REDS_IN - pi_reductions_spinnloop(1, atom, "hej", self(), make_ref(), 3.14). + pi_reductions_spinnloop(999*1000, atom, "hej", self(), make_ref(), 3.14). -pi_reductions_spinnloop(A,B,C,D,E,F) -> - pi_reductions_spinnloop(B,C,D,E,F,A). +pi_reductions_spinnloop(N,A,B,C,D,E) when N > 0 -> + pi_reductions_spinnloop(N-1,B,C,D,E,A); +pi_reductions_spinnloop(0,_,_,_,_,_) -> + %% Stop to limit max number of reductions consumed + pi_reductions_recvloop(). pi_reductions_recvloop() -> receive "a free lunch" -> false end. +pi_reductions_main_unlocker() -> + Other = spawn_link(fun() -> receive die -> ok end end), + pi_reductions_main_unlocker_loop(Other). + +pi_reductions_main_unlocker_loop(Other) -> + %% Assumption: register(OtherPid, Name) will unlock main lock of calling + %% process during execution. + register(pi_reductions_main_unlocker, Other), + unregister(pi_reductions_main_unlocker), + + %% Yield in order to increase probability of process_info sometimes probing + %% this process when it's not RUNNING. + erlang:yield(), + pi_reductions_main_unlocker_loop(Other). + %% Tests erlang:bump_reductions/1. bump_reductions(Config) when is_list(Config) -> diff --git a/erts/emulator/test/socket_SUITE.erl b/erts/emulator/test/socket_SUITE.erl index 49b0fcccc2..d8cb1013e9 100644 --- a/erts/emulator/test/socket_SUITE.erl +++ b/erts/emulator/test/socket_SUITE.erl @@ -81,6 +81,24 @@ api_b_sendmsg_and_recvmsg_tcp4/1, api_b_sendmsg_and_recvmsg_tcpL/1, + %% *** API async *** + api_a_connect_tcp4/1, + api_a_sendto_and_recvfrom_udp4/1, + api_a_sendmsg_and_recvmsg_udp4/1, + api_a_send_and_recv_tcp4/1, + api_a_sendmsg_and_recvmsg_tcp4/1, + api_a_recvfrom_cancel_udp4/1, + api_a_recvmsg_cancel_udp4/1, + api_a_accept_cancel_tcp4/1, + api_a_recv_cancel_tcp4/1, + api_a_recvmsg_cancel_tcp4/1, + api_a_mrecvfrom_cancel_udp4/1, + api_a_mrecvmsg_cancel_udp4/1, + api_a_maccept_cancel_tcp4/1, + api_a_mrecv_cancel_tcp4/1, + api_a_mrecvmsg_cancel_tcp4/1, + + %% *** API Options *** api_opt_simple_otp_options/1, api_opt_simple_otp_rcvbuf_option/1, @@ -565,6 +583,7 @@ use_group(Group, Env, Default) -> groups() -> [{api, [], api_cases()}, {api_basic, [], api_basic_cases()}, + {api_async, [], api_async_cases()}, {api_options, [], api_options_cases()}, {api_op_with_timeout, [], api_op_with_timeout_cases()}, {socket_closure, [], socket_closure_cases()}, @@ -639,6 +658,7 @@ groups() -> api_cases() -> [ {group, api_basic}, + {group, api_async}, {group, api_options}, {group, api_op_with_timeout} ]. @@ -659,6 +679,25 @@ api_basic_cases() -> api_b_sendmsg_and_recvmsg_tcpL ]. +api_async_cases() -> + [ + api_a_connect_tcp4, + api_a_sendto_and_recvfrom_udp4, + api_a_sendmsg_and_recvmsg_udp4, + api_a_send_and_recv_tcp4, + api_a_sendmsg_and_recvmsg_tcp4, + api_a_recvfrom_cancel_udp4, + api_a_recvmsg_cancel_udp4, + api_a_accept_cancel_tcp4, + api_a_recv_cancel_tcp4, + api_a_recvmsg_cancel_tcp4, + api_a_mrecvfrom_cancel_udp4, + api_a_mrecvmsg_cancel_udp4, + api_a_maccept_cancel_tcp4, + api_a_mrecv_cancel_tcp4, + api_a_mrecvmsg_cancel_tcp4 + ]. + api_options_cases() -> [ api_opt_simple_otp_options, @@ -1543,11 +1582,12 @@ init_per_group(ttest = _GroupName, Config) -> io:format("init_per_group(~w) -> entry with" "~n Config: ~p" "~n", [_GroupName, Config]), + ttest_manager_start(), case lists:keysearch(esock_test_ttest_runtime, 1, Config) of {value, _} -> Config; false -> - [{esock_test_ttest_runtime, which_ttest_runtime_env()}|Config] + [{esock_test_ttest_runtime, which_ttest_runtime_env()} | Config] end; init_per_group(_GroupName, Config) -> Config. @@ -1556,6 +1596,7 @@ end_per_group(ttest = _GroupName, Config) -> io:format("init_per_group(~w) -> entry with" "~n Config: ~p" "~n", [_GroupName, Config]), + ttest_manager_stop(), lists:keydelete(esock_test_ttest_runtime, 1, Config); end_per_group(_GroupName, Config) -> Config. @@ -1565,16 +1606,16 @@ init_per_testcase(_TC, Config) -> io:format("init_per_testcase(~w) -> entry with" "~n Config: ~p" "~n", [_TC, Config]), - case quiet_mode(Config) of - default -> - ?LOGGER:start(); - Quiet -> - ?LOGGER:start(Quiet) - end, + %% case quiet_mode(Config) of + %% default -> + %% ?LOGGER:start(); + %% Quiet -> + %% ?LOGGER:start(Quiet) + %% end, Config. end_per_testcase(_TC, Config) -> - ?LOGGER:stop(), + %% ?LOGGER:stop(), Config. @@ -2526,11 +2567,7 @@ api_b_send_and_recv_tcp(InitState) -> ?SEV_ANNOUNCE_CONTINUE(Server, accept), ok end}, - #{desc => "sleep", - cmd => fun(_) -> - ?SLEEP(?SECS(1)), - ok - end}, + ?SEV_SLEEP(?SECS(1)), #{desc => "order client to continue (with connect)", cmd => fun(#{client := Client} = _State) -> ?SEV_ANNOUNCE_CONTINUE(Client, connect), @@ -2617,6 +2654,3852 @@ api_b_send_and_recv_tcp(InitState) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically establish a TCP connection via an async connect. + +api_a_connect_tcp4(suite) -> + []; +api_a_connect_tcp4(doc) -> + []; +api_a_connect_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(api_a_connect_tcp4, + fun() -> + Connect = fun(Sock, SockAddr) -> + socket:connect(Sock, SockAddr, nowait) + end, + Send = fun(Sock, Data) -> + socket:send(Sock, Data) + end, + Recv = fun(Sock) -> + socket:recv(Sock) + end, + InitState = #{domain => inet, + connect => Connect, + send => Send, + recv => Recv}, + ok = api_a_connect_tcp(InitState) + end). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_connect_tcp(InitState) -> + process_flag(trap_exit, true), + ServerSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{lsa => LSA}} + end}, + #{desc => "create listen socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{lsock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{lsock := LSock, lsa := LSA} = State) -> + case socket:bind(LSock, LSA) of + {ok, Port} -> + {ok, State#{lport => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "make listen socket", + cmd => fun(#{lsock := LSock}) -> + socket:listen(LSock) + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, lport := Port}) -> + ?SEV_ANNOUNCE_READY(Tester, init, Port), + ok + end}, + + %% The actual test + #{desc => "await continue (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, accept) + end}, + #{desc => "await connection", + cmd => fun(#{lsock := LSock} = State) -> + case socket:accept(LSock) of + {ok, Sock} -> + ?SEV_IPRINT("accepted: ~n ~p", [Sock]), + {ok, State#{csock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, accept), + ok + end}, + + #{desc => "await continue (recv_req)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv_req) + end}, + #{desc => "recv req", + cmd => fun(#{csock := Sock, recv := Recv}) -> + case Recv(Sock) of + {ok, ?BASIC_REQ} -> + ok; + {ok, UnexpData} -> + {error, {unexpected_data, UnexpData}}; + {error, _} = ERROR -> + %% At the moment there is no way to get + %% status or state for the socket... + ERROR + end + end}, + #{desc => "announce ready (recv_req)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_req), + ok + end}, + #{desc => "await continue (send_rep)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, send_rep) + end}, + #{desc => "send rep", + cmd => fun(#{csock := Sock, send := Send}) -> + Send(Sock, ?BASIC_REP) + end}, + #{desc => "announce ready (send_rep)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, send_rep), + ok + end}, + + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close connection socket", + cmd => fun(#{csock := Sock} = State) -> + ok = socket:close(Sock), + {ok, maps:remove(csock, State)} + end}, + #{desc => "close listen socket", + cmd => fun(#{lsock := Sock} = State) -> + ok = socket:close(Sock), + {ok, maps:remove(lsock, State)} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + ClientSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + {Tester, Port} = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester, server_port => Port}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** The init part *** + #{desc => "which server (local) address", + cmd => fun(#{domain := Domain, server_port := Port} = State) -> + LSA = which_local_socket_addr(Domain), + SSA = LSA#{port => Port}, + {ok, State#{local_sa => LSA, server_sa => SSA}} + end}, + #{desc => "create socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{sock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{sock := Sock, local_sa := LSA} = _State) -> + case socket:bind(Sock, LSA) of + {ok, _Port} -> + ok; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + + %% *** The actual test *** + #{desc => "await continue (async connect)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, async_connect) + end}, + #{desc => "connect (async) to server", + cmd => fun(#{sock := Sock, + server_sa := SSA, + connect := Connect} = State) -> + case Connect(Sock, SSA) of + ok -> + ?SEV_IPRINT("ok -> " + "unexpected success => SKIP", + []), + {skip, unexpected_success}; + {select, {select_info, ST, SR}} -> + ?SEV_IPRINT("select ->" + "~n tag: ~p" + "~n ref: ~p", [ST, SR]), + {ok, State#{connect_stag => ST, + connect_sref => SR}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (connect select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, connect_select), + ok + end}, + #{desc => "await select message", + cmd => fun(#{sock := Sock, connect_sref := Ref}) -> + receive + {'$socket', Sock, select, Ref} -> + ?SEV_IPRINT("select message ->" + "~n ref: ~p", [Ref]), + ok + after 5000 -> + ?SEV_EPRINT("timeout: " + "~n message queue: ~p", + [mq()]), + {error, timeout} + end + end}, + #{desc => "announce ready (select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, select), + ok + end}, + #{desc => "connect (async) to server", + cmd => fun(#{sock := Sock, server_sa := SSA, connect := Connect}) -> + case Connect(Sock, SSA) of + ok -> + ok; + {select, SelectInfo} -> + {error, {unexpected_select, SelectInfo}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (connect)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, connect), + ok + end}, + #{desc => "get peername", + cmd => fun(#{sock := Sock} = _State) -> + case socket:peername(Sock) of + {ok, SockAddr} -> + ?SEV_IPRINT("Peer Name: ~p", [SockAddr]), + ok; + {error, _} = ERROR -> + ERROR + end + end}, + + #{desc => "await continue (send_req)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, send_req) + end}, + #{desc => "send req", + cmd => fun(#{sock := Sock, send := Send}) -> + Send(Sock, ?BASIC_REQ) + end}, + #{desc => "announce ready (send_req)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, send_req), + ok + end}, + #{desc => "await continue (recv_rep)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv_rep) + end}, + #{desc => "recv rep", + cmd => fun(#{sock := Sock, recv := Recv}) -> + case Recv(Sock) of + {ok, ?BASIC_REP} -> + ok; + {ok, UnexpData} -> + {error, {unexpected_data, UnexpData}}; + {error, _} = ERROR -> + %% At the moment there is no way to get + %% status or state for the socket... + ERROR + end + end}, + #{desc => "announce ready (recv_rep)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_rep), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close socket", + cmd => fun(#{sock := Sock} = State) -> + ok = socket:close(Sock), + State2 = maps:remove(sock, State), + State3 = maps:remove(connect_stag, State2), + State4 = maps:remove(connect_sref, State3), + {ok, State4} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor client", + cmd => fun(#{client := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, Port} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{server_port => Port}} + end}, + + %% Start the client + #{desc => "order client start", + cmd => fun(#{client := Pid, server_port := Port} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Port), + ok + end}, + #{desc => "await client ready (init)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, init) + end}, + + + %% *** The actual test *** + #{desc => "order client to continue (async connect)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Client, async_connect), + ok + end}, + #{desc => "await client ready (connect select)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_AWAIT_READY(Client, client, connect_select) + end}, + + ?SEV_SLEEP(?SECS(1)), + + #{desc => "order server to continue (with accept)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Server, accept), + ok + end}, + #{desc => "await client ready (select)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_AWAIT_READY(Client, client, select) + end}, + #{desc => "await client ready (connect)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_AWAIT_READY(Client, client, connect) + end}, + #{desc => "await server ready (accept)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_AWAIT_READY(Server, server, accept) + end}, + + ?SEV_SLEEP(?SECS(1)), + + #{desc => "order server to recv test req (recv req)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Server, recv_req), + ok + end}, + #{desc => "order client to send test req (send req)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Client, send_req), + ok + end}, + #{desc => "await client ready (send_req)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_AWAIT_READY(Client, client, send_req) + end}, + #{desc => "await server ready (recv_req)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_AWAIT_READY(Server, server, recv_req) + end}, + #{desc => "order client to recv test rep (send rep)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Client, recv_rep), + ok + end}, + #{desc => "order server to send test rep (send rep)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Server, send_rep), + ok + end}, + #{desc => "await server ready (send_rep)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_AWAIT_READY(Server, server, send_rep) + end}, + #{desc => "await client ready (recv_rep)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_AWAIT_READY(Client, client, recv_rep) + end}, + + + %% *** Termination *** + #{desc => "order client to terminate", + cmd => fun(#{client := Client} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Client), + ok + end}, + #{desc => "await client termination", + cmd => fun(#{client := Client} = State) -> + ?SEV_AWAIT_TERMINATION(Client), + State1 = maps:remove(client, State), + {ok, State1} + end}, + #{desc => "order server to terminate", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Server), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Server} = State) -> + ?SEV_AWAIT_TERMINATION(Server), + State1 = maps:remove(server, State), + {ok, State1} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + Server = ?SEV_START("server", ServerSeq, InitState), + + i("start client evaluator"), + Client = ?SEV_START("client", ClientSeq, InitState), + i("await evaluator(s)"), + + i("start tester evaluator"), + TesterInitState = #{server => Server#ev.pid, + client => Client#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + ok = ?SEV_AWAIT_FINISH([Server, Client, Tester]). + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically send and receive on an IPv4 UDP (dgram) socket using +%% sendto and recvfrom. But we try to be async. That is, we use +%% the 'nowait' value for the Timeout argument (and await the eventual +%% select message). Note that we only do this for the recvfrom, +%% since its much more difficult to "arrange" for sendto. +%% +api_a_sendto_and_recvfrom_udp4(suite) -> + []; +api_a_sendto_and_recvfrom_udp4(doc) -> + []; +api_a_sendto_and_recvfrom_udp4(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(api_a_sendto_and_recvfrom_udp4, + fun() -> + Send = fun(Sock, Data, Dest) -> + socket:sendto(Sock, Data, Dest) + end, + Recv = fun(Sock) -> + socket:recvfrom(Sock, 0, nowait) + end, + InitState = #{domain => inet, + send => Send, + recv => Recv}, + ok = api_a_send_and_recv_udp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically send and receive on an IPv4 UDP (dgram) socket using +%% sendto and recvfrom. But we try to be async. That is, we use +%% the 'nowait' value for the Timeout argument (and await the eventual +%% select message). Note that we only do this for the recvmsg, +%% since its much more difficult to "arrange" for sendmsg. +%% +api_a_sendmsg_and_recvmsg_udp4(suite) -> + []; +api_a_sendmsg_and_recvmsg_udp4(doc) -> + []; +api_a_sendmsg_and_recvmsg_udp4(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(api_a_sendmsg_and_recvmsg_udp4, + fun() -> + Send = fun(Sock, Data, Dest) -> + MsgHdr = #{addr => Dest, + %% ctrl => CMsgHdrs, + iov => [Data]}, + socket:sendmsg(Sock, MsgHdr) + end, + Recv = fun(Sock) -> + case socket:recvmsg(Sock, nowait) of + {ok, #{addr := Source, + iov := [Data]}} -> + {ok, {Source, Data}}; + {ok, _} = OK -> + OK; + {select, _} = SELECT -> + SELECT; + {error, _} = ERROR -> + ERROR + end + end, + InitState = #{domain => inet, + send => Send, + recv => Recv}, + ok = api_a_send_and_recv_udp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_send_and_recv_udp(InitState) -> + ServerSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{local_sa => LSA}} + end}, + #{desc => "create socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, dgram, udp) of + {ok, Sock} -> + {ok, State#{sock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind socket (to local address)", + cmd => fun(#{sock := Sock, local_sa := LSA} = State) -> + case socket:bind(Sock, LSA) of + {ok, Port} -> + {ok, State#{port => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, local_sa := LSA, port := Port}) -> + ServerSA = LSA#{port => Port}, + ?SEV_ANNOUNCE_READY(Tester, init, ServerSA), + ok + end}, + + %% The actual test + #{desc => "await continue (recv)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv) + end}, + #{desc => "try recv request (with nowait, expect select)", + cmd => fun(#{sock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, {select_info, Tag, RecvRef}} -> + ?SEV_IPRINT("expected select: " + "~n Tag: ~p" + "~n Ref: ~p", [Tag, RecvRef]), + {ok, State#{recv_stag => Tag, + recv_sref => RecvRef}}; + {ok, X} -> + {error, {unexpected_succes, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await select message", + cmd => fun(#{sock := Sock, recv_sref := RecvRef}) -> + receive + {'$socket', Sock, select, RecvRef} -> + ok + after 5000 -> + ?SEV_EPRINT("message queue: ~p", [mq()]), + {error, timeout} + end + end}, + #{desc => "announce ready (select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, select), + ok + end}, + #{desc => "now read the data (request)", + cmd => fun(#{sock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {ok, {Src, ?BASIC_REQ}} -> + {ok, State#{req_src => Src}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv request)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_req), + ok + end}, + + #{desc => "await continue (send reply)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, send_reply) + end}, + #{desc => "send reply", + cmd => fun(#{sock := Sock, req_src := Src, send := Send}) -> + Send(Sock, ?BASIC_REP, Src) + end}, + #{desc => "announce ready (send)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, send), + ok + end}, + + %% Termination + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + State2 = maps:remove(tester, State), + State3 = maps:remove(recv_stag, State2), + State4 = maps:remove(recv_sref, State3), + State5 = maps:remove(req_src, State4), + {ok, State5}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close socket", + cmd => fun(#{sock := Sock} = State) -> + ok = socket:close(Sock), + {ok, maps:remove(sock, State)} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + ClientSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start", + cmd => fun(State) -> + {Tester, ServerSA} = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester, + server_sa => ServerSA}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{lsa => LSA}} + end}, + #{desc => "open socket", + cmd => fun(#{domain := Domain} = State) -> + Sock = sock_open(Domain, dgram, udp), + SA = sock_sockname(Sock), + {ok, State#{sock => Sock, sa => SA}} + end}, + #{desc => "bind socket (to local address)", + cmd => fun(#{sock := Sock, lsa := LSA}) -> + case socket:bind(Sock, LSA) of + {ok, _Port} -> + ok; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + + %% The actual test + #{desc => "await continue (send request)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, send_req) + end}, + #{desc => "send request", + cmd => fun(#{sock := Sock, server_sa := Server, send := Send}) -> + Send(Sock, ?BASIC_REQ, Server) + end}, + #{desc => "announce ready (send request)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, send_req), + ok + end}, + #{desc => "await continue (recv)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv) + end}, + #{desc => "try recv reply (with nowait)", + cmd => fun(#{sock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, {select_info, Tag, RecvRef}} -> + {ok, State#{recv_stag => Tag, + recv_sref => RecvRef}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await select message", + cmd => fun(#{sock := Sock, recv_sref := RecvRef}) -> + receive + {'$socket', Sock, select, RecvRef} -> + ok + end + end}, + #{desc => "announce ready (select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, select), + ok + end}, + #{desc => "now read the data (reply)", + cmd => fun(#{sock := Sock, recv := Recv}) -> + case Recv(Sock) of + {ok, {_Src, ?BASIC_REP}} -> + ok; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv reply)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_rep), + ok + end}, + + %% Termination + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + State2 = maps:remove(tester, State), + State3 = maps:remove(recv_stag, State2), + State4 = maps:remove(recv_sref, State3), + {ok, State4}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close socket", + cmd => fun(#{sock := Sock} = State) -> + ok = socket:close(Sock), + {ok, maps:remove(sock, State)} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor client", + cmd => fun(#{client := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, ServerSA} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{server_sa => ServerSA}} + end}, + + %% Start the client + #{desc => "order client start", + cmd => fun(#{client := Pid, + server_sa := ServerSA} = _State) -> + ?SEV_ANNOUNCE_START(Pid, ServerSA), + ok + end}, + #{desc => "await client ready (init)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, init) + end}, + + %% The actual test + #{desc => "order server continue (recv)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await server ready (recv_select)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, recv_select) + end}, + + #{desc => "order client continue (send request)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, send_req), + ok + end}, + #{desc => "await client ready (send request)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, send_req) + end}, + #{desc => "await server ready (select)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, select) + end}, + #{desc => "await server ready (recv request)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, recv_req) + end}, + + #{desc => "order client continue (recv)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await client ready (recv_select)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, recv_select) + end}, + #{desc => "order server continue (send reply)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, send_reply), + ok + end}, + #{desc => "await server ready (send)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, send) + end}, + #{desc => "await client ready (select)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, select) + end}, + #{desc => "await client ready (recv reply)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, recv_rep) + end}, + + %% Terminations + #{desc => "order client to terminate", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await client termination", + cmd => fun(#{client := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(client, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "order server to terminate", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(server, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + ServerInitState = InitState, + Server = ?SEV_START("server", ServerSeq, ServerInitState), + + i("start client evaluator(s)"), + ClientInitState = InitState, + Client = ?SEV_START("client", ClientSeq, ClientInitState), + + i("start 'tester' evaluator"), + TesterInitState = #{server => Server#ev.pid, + client => Client#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator"), + ok = ?SEV_AWAIT_FINISH([Server, Client, Tester]). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically send and receive using the "common" functions (send and recv) +%% on an IPv4 TCP (stream) socket. But we try to be async. That is, we use +%% the 'nowait' value for the Timeout argument (and await the eventual +%% select message). Note that we only do this for the recv, +%% since its much more difficult to "arrange" for send. +%% We *also* test async for accept. +api_a_send_and_recv_tcp4(suite) -> + []; +api_a_send_and_recv_tcp4(doc) -> + []; +api_a_send_and_recv_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(api_a_send_and_recv_tcp4, + fun() -> + Send = fun(Sock, Data) -> + socket:send(Sock, Data) + end, + Recv = fun(Sock) -> + socket:recv(Sock, 0, nowait) + end, + InitState = #{domain => inet, + send => Send, + recv => Recv}, + ok = api_a_send_and_recv_tcp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically send and receive using the msg functions (sendmsg and recvmsg) +%% on an IPv4 TCP (stream) socket. But we try to be async. That is, we use +%% the 'nowait' value for the Timeout argument (and await the eventual +%% select message). Note that we only do this for the recvmsg, +%% since its much more difficult to "arrange" for sendmsg. +%% We *also* test async for accept. +api_a_sendmsg_and_recvmsg_tcp4(suite) -> + []; +api_a_sendmsg_and_recvmsg_tcp4(doc) -> + []; +api_a_sendmsg_and_recvmsg_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(api_a_sendmsg_and_recvmsg_tcp4, + fun() -> + Send = fun(Sock, Data) -> + MsgHdr = #{iov => [Data]}, + socket:sendmsg(Sock, MsgHdr) + end, + Recv = fun(Sock) -> + case socket:recvmsg(Sock, nowait) of + {ok, #{addr := undefined, + iov := [Data]}} -> + {ok, Data}; + {ok, _} = OK -> + OK; + {select, _} = SELECT -> + SELECT; + {error, _} = ERROR -> + ERROR + end + end, + InitState = #{domain => inet, + send => Send, + recv => Recv}, + ok = api_a_send_and_recv_tcp(InitState) + end). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_send_and_recv_tcp(InitState) -> + process_flag(trap_exit, true), + ServerSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{lsa => LSA}} + end}, + #{desc => "create listen socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{lsock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{lsock := LSock, lsa := LSA} = State) -> + case socket:bind(LSock, LSA) of + {ok, Port} -> + {ok, State#{lport => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "make listen socket", + cmd => fun(#{lsock := LSock}) -> + socket:listen(LSock) + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, lport := Port}) -> + ?SEV_ANNOUNCE_READY(Tester, init, Port), + ok + end}, + + %% The actual test + #{desc => "await continue (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, accept) + end}, + #{desc => "await connection (nowait)", + cmd => fun(#{lsock := LSock} = State) -> + case socket:accept(LSock, nowait) of + {select, {select_info, Tag, Ref}} -> + ?SEV_IPRINT("accept select: " + "~n Tag: ~p" + "~n Ref: ~p", [Tag, Ref]), + {ok, State#{accept_stag => Tag, + accept_sref => Ref}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (accept_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, accept_select), + ok + end}, + #{desc => "await select message", + cmd => fun(#{lsock := Sock, accept_sref := Ref}) -> + receive + {'$socket', Sock, select, Ref} -> + ok + end + end}, + #{desc => "announce ready (select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, select), + ok + end}, + #{desc => "await connection (again)", + cmd => fun(#{lsock := LSock} = State) -> + case socket:accept(LSock, nowait) of + {ok, Sock} -> + ?SEV_IPRINT("accepted: " + "~n Sock: ~p", [Sock]), + {ok, State#{csock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, accept), + ok + end}, + + #{desc => "await continue (recv request)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv_req) + end}, + #{desc => "try recv request (with nowait, expect select)", + cmd => fun(#{csock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, {select_info, Tag, Ref}} -> + ?SEV_IPRINT("recv select: " + "~n Tag: ~p" + "~n Ref: ~p", [Tag, Ref]), + {ok, State#{recv_stag => Tag, + recv_sref => Ref}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await select message", + cmd => fun(#{csock := Sock, recv_sref := RecvRef}) -> + receive + {'$socket', Sock, select, RecvRef} -> + ok + end + end}, + #{desc => "announce ready (select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, select), + ok + end}, + #{desc => "now read the data (request)", + cmd => fun(#{csock := Sock, recv := Recv} = _State) -> + case Recv(Sock) of + {ok, ?BASIC_REQ} -> + ok; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv request)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_req), + ok + end}, + + #{desc => "await continue (send reply)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, send_rep) + end}, + #{desc => "send reply", + cmd => fun(#{csock := Sock, send := Send}) -> + Send(Sock, ?BASIC_REP) + end}, + #{desc => "announce ready (send reply)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, send_rep), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close connection socket", + cmd => fun(#{csock := Sock}) -> + socket:close(Sock) + end}, + #{desc => "close listen socket", + cmd => fun(#{lsock := Sock}) -> + socket:close(Sock) + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + ClientSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + {Tester, Port} = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester, server_port => Port}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** The init part *** + #{desc => "which server (local) address", + cmd => fun(#{domain := Domain, server_port := Port} = State) -> + LSA = which_local_socket_addr(Domain), + SSA = LSA#{port => Port}, + {ok, State#{local_sa => LSA, server_sa => SSA}} + end}, + #{desc => "create socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{sock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{sock := Sock, local_sa := LSA} = _State) -> + case socket:bind(Sock, LSA) of + {ok, _Port} -> + ok; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + + %% *** The actual test *** + #{desc => "await continue (connect)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, connect) + end}, + #{desc => "connect to server", + cmd => fun(#{sock := Sock, server_sa := SSA}) -> + socket:connect(Sock, SSA) + end}, + #{desc => "announce ready (connect)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, connect), + ok + end}, + + #{desc => "await continue (send request)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, send_req) + end}, + #{desc => "send request (to server)", + cmd => fun(#{sock := Sock, send := Send}) -> + ok = Send(Sock, ?BASIC_REQ) + end}, + #{desc => "announce ready (send request)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, send_req), + ok + end}, + + #{desc => "try recv reply (with nowait, expect select)", + cmd => fun(#{sock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, {select_info, Tag, Ref}} -> + ?SEV_IPRINT("recv select: " + "~n Tag: ~p" + "~n Ref: ~p", [Tag, Ref]), + {ok, State#{recv_stag => Tag, + recv_sref => Ref}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await select message", + cmd => fun(#{sock := Sock, recv_sref := RecvRef}) -> + receive + {'$socket', Sock, select, RecvRef} -> + ok + end + end}, + #{desc => "announce ready (select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, select), + ok + end}, + #{desc => "now read the data (reply)", + cmd => fun(#{sock := Sock, recv := Recv}) -> + {ok, ?BASIC_REP} = Recv(Sock), + ok + end}, + #{desc => "announce ready (recv reply)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_rep), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close socket", + cmd => fun(#{sock := Sock}) -> + socket:close(Sock) + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor client", + cmd => fun(#{client := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, Port} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{server_port => Port}} + end}, + + %% Start the client + #{desc => "order client start", + cmd => fun(#{client := Pid, server_port := Port} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Port), + ok + end}, + #{desc => "await client ready (init)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, init) + end}, + + %% *** The actual test *** + #{desc => "order server to continue (with accept)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Server, accept), + ok + end}, + #{desc => "await server ready (accept select)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, accept_select) + end}, + #{desc => "order client to continue (with connect)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, connect), + ok + end}, + #{desc => "await server ready (select)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, select) + end}, + #{desc => "await server ready (accept)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, accept) + end}, + #{desc => "await client ready (connect)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, client, connect) + end}, + + #{desc => "order server to continue (recv request)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv_req), + ok + end}, + #{desc => "await server ready (recv select)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, recv_select) + end}, + #{desc => "order client to continue (send request)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, send_req), + ok + end}, + #{desc => "await client ready (send request)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_AWAIT_READY(Client, client, send_req) + end}, + #{desc => "await server ready (select)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, select) + end}, + #{desc => "await server ready (recv request)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, recv_req) + end}, + + #{desc => "order client to continue (recv reply)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv_rep), + ok + end}, + #{desc => "await client ready (recv select)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, client, recv_select) + end}, + #{desc => "order server to continue (send reply)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, send_rep), + ok + end}, + #{desc => "await server ready (send reply)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, send_rep) + end}, + #{desc => "await client ready (select)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, client, select) + end}, + #{desc => "await client ready (reply recv)", + cmd => fun(#{client := Client} = _State) -> + ?SEV_AWAIT_READY(Client, client, recv_rep) + end}, + + + %% *** Termination *** + #{desc => "order client to terminate", + cmd => fun(#{client := Client} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Client), + ok + end}, + #{desc => "await client termination", + cmd => fun(#{client := Client} = State) -> + ?SEV_AWAIT_TERMINATION(Client), + State1 = maps:remove(client, State), + {ok, State1} + end}, + #{desc => "order server to terminate", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Server), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Server} = State) -> + ?SEV_AWAIT_TERMINATION(Server), + State1 = maps:remove(server, State), + {ok, State1} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + Server = ?SEV_START("server", ServerSeq, InitState), + + i("start client evaluator"), + Client = ?SEV_START("client", ClientSeq, InitState), + + i("start tester evaluator"), + TesterInitState = #{server => Server#ev.pid, + client => Client#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator(s)"), + ok = ?SEV_AWAIT_FINISH([Server, Client, Tester]). + + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make an async (Timeout = nowait) call to recvfrom, +%% wait some time and then cancel. +%% +api_a_recvfrom_cancel_udp4(suite) -> + []; +api_a_recvfrom_cancel_udp4(doc) -> + []; +api_a_recvfrom_cancel_udp4(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(api_a_recvfrom_cancel_udp4, + fun() -> + Recv = fun(Sock) -> + case socket:recvfrom(Sock, 0, nowait) of + {ok, _} = OK -> + OK; + {select, _} = SELECT -> + SELECT; + {error, _} = ERROR -> + ERROR + end + end, + InitState = #{domain => inet, + recv => Recv}, + ok = api_a_recv_cancel_udp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make an async (Timeout = nowait) call to recvmsg, +%% wait some time and then cancel. +%% +api_a_recvmsg_cancel_udp4(suite) -> + []; +api_a_recvmsg_cancel_udp4(doc) -> + []; +api_a_recvmsg_cancel_udp4(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(api_a_recvmsg_cancel_udp4, + fun() -> + Recv = fun(Sock) -> + case socket:recvmsg(Sock, nowait) of + {ok, _} = OK -> + OK; + {select, _} = SELECT -> + SELECT; + {error, _} = ERROR -> + ERROR + end + end, + InitState = #{domain => inet, + recv => Recv}, + ok = api_a_recv_cancel_udp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_recv_cancel_udp(InitState) -> + ServerSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{local_sa => LSA}} + end}, + #{desc => "create socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, dgram, udp) of + {ok, Sock} -> + {ok, State#{sock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind socket (to local address)", + cmd => fun(#{sock := Sock, local_sa := LSA} = State) -> + case socket:bind(Sock, LSA) of + {ok, Port} -> + {ok, State#{port => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, local_sa := LSA, port := Port}) -> + ServerSA = LSA#{port => Port}, + ?SEV_ANNOUNCE_READY(Tester, init, ServerSA), + ok + end}, + + %% The actual test + #{desc => "await continue (recv)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv) + end}, + #{desc => "try recv request (with nowait, expect select)", + cmd => fun(#{sock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, SelectInfo} -> + {ok, State#{recv_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await select message (without success)", + cmd => fun(#{sock := Sock}) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}} + after 5000 -> + ok + end + end}, + #{desc => "announce ready (no select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, no_select), + ok + end}, + #{desc => "await continue (cancel)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, cancel) + end}, + #{desc => "cancel", + cmd => fun(#{sock := Sock, recv_select_info := SelectInfo}) -> + ok = socket:cancel(Sock, SelectInfo) + end}, + #{desc => "announce ready (cancel)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, cancel), + ok + end}, + + %% Termination + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + State2 = maps:remove(tester, State), + State3 = maps:remove(recv_stag, State2), + State4 = maps:remove(recv_sref, State3), + State5 = maps:remove(req_src, State4), + {ok, State5}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close socket", + cmd => fun(#{sock := Sock} = State) -> + ok = socket:close(Sock), + {ok, maps:remove(sock, State)} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, ServerSA} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{server_sa => ServerSA}} + end}, + + %% The actual test + #{desc => "order server continue (recv)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await server ready (recv select)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, recv_select) + end}, + #{desc => "await server ready (no select)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, no_select) + end}, + #{desc => "order server continue (cancel)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, cancel), + ok + end}, + #{desc => "await server ready (cancel)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, cancel) + end}, + + %% Terminations + #{desc => "order server to terminate", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(server, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + ServerInitState = InitState, + Server = ?SEV_START("server", ServerSeq, ServerInitState), + + i("start 'tester' evaluator"), + TesterInitState = #{server => Server#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator"), + ok = ?SEV_AWAIT_FINISH([Server, Tester]). + + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make an async (Timeout = nowait) call to accept, +%% wait some time and then cancel. +%% +api_a_accept_cancel_tcp4(suite) -> + []; +api_a_accept_cancel_tcp4(doc) -> + []; +api_a_accept_cancel_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(api_a_accept_cancel_tcp4, + fun() -> + Accept = fun(Sock) -> + case socket:accept(Sock, nowait) of + {ok, _} = OK -> + OK; + {select, _} = SELECT -> + SELECT; + {error, _} = ERROR -> + ERROR + end + end, + InitState = #{domain => inet, + accept => Accept}, + ok = api_a_accept_cancel_tcp(InitState) + end). + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_accept_cancel_tcp(InitState) -> + process_flag(trap_exit, true), + ServerSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{lsa => LSA}} + end}, + #{desc => "create listen socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{lsock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{lsock := LSock, lsa := LSA} = State) -> + case socket:bind(LSock, LSA) of + {ok, Port} -> + {ok, State#{lport => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "make listen socket", + cmd => fun(#{lsock := LSock}) -> + socket:listen(LSock) + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, lport := Port}) -> + ?SEV_ANNOUNCE_READY(Tester, init, Port), + ok + end}, + + %% The actual test + #{desc => "await continue (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, accept) + end}, + #{desc => "await connection (nowait)", + cmd => fun(#{lsock := LSock, accept := Accept} = State) -> + case Accept(LSock) of + {select, {select_info, T, R} = SelectInfo} -> + ?SEV_IPRINT("accept select: " + "~n T: ~p" + "~n R: ~p", [T, R]), + {ok, State#{accept_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (accept_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, accept_select), + ok + end}, + #{desc => "await select message (without success)", + cmd => fun(#{lsock := Sock}) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}} + after 5000 -> + ok + end + end}, + #{desc => "announce ready (no select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, no_select), + ok + end}, + #{desc => "await continue (cancel)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, cancel) + end}, + #{desc => "cancel", + cmd => fun(#{lsock := Sock, accept_select_info := SelectInfo}) -> + ok = socket:cancel(Sock, SelectInfo) + end}, + #{desc => "announce ready (cancel)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, cancel), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close listen socket", + cmd => fun(#{lsock := Sock}) -> + socket:close(Sock) + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, Port} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{server_port => Port}} + end}, + + %% *** The actual test *** + #{desc => "order server to continue (with accept)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Server, accept), + ok + end}, + #{desc => "await server ready (accept select)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, accept_select) + end}, + #{desc => "await server ready (no select)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, no_select) + end}, + #{desc => "order server to continue (cancel)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, cancel), + ok + end}, + #{desc => "await server ready (cancel)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, cancel) + end}, + + %% *** Termination *** + #{desc => "order server to terminate", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Server), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Server} = State) -> + ?SEV_AWAIT_TERMINATION(Server), + State1 = maps:remove(server, State), + {ok, State1} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + Server = ?SEV_START("server", ServerSeq, InitState), + + i("start tester evaluator"), + TesterInitState = #{server => Server#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator(s)"), + ok = ?SEV_AWAIT_FINISH([Server, Tester]). + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make an async (Timeout = nowait) call to recv, +%% wait some time and then cancel. +%% +api_a_recv_cancel_tcp4(suite) -> + []; +api_a_recv_cancel_tcp4(doc) -> + []; +api_a_recv_cancel_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(api_a_recv_cancel_tcp4, + fun() -> + Recv = fun(Sock) -> + socket:recv(Sock, 0, nowait) + end, + InitState = #{domain => inet, + recv => Recv}, + ok = api_a_recv_cancel_tcp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make an async (Timeout = nowait) call to recvmsg, +%% wait some time and then cancel. +%% +api_a_recvmsg_cancel_tcp4(suite) -> + []; +api_a_recvmsg_cancel_tcp4(doc) -> + []; +api_a_recvmsg_cancel_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(api_a_recvmsg_cancel_tcp4, + fun() -> + Recv = fun(Sock) -> + socket:recvmsg(Sock, nowait) + end, + InitState = #{domain => inet, + recv => Recv}, + ok = api_a_recv_cancel_tcp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_recv_cancel_tcp(InitState) -> + process_flag(trap_exit, true), + ServerSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{lsa => LSA}} + end}, + #{desc => "create listen socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{lsock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{lsock := LSock, lsa := LSA} = State) -> + case socket:bind(LSock, LSA) of + {ok, Port} -> + {ok, State#{lport => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "make listen socket", + cmd => fun(#{lsock := LSock}) -> + socket:listen(LSock) + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, lport := Port}) -> + ?SEV_ANNOUNCE_READY(Tester, init, Port), + ok + end}, + + %% The actual test + #{desc => "await continue (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, accept) + end}, + #{desc => "await connection (nowait)", + cmd => fun(#{lsock := LSock} = State) -> + case socket:accept(LSock) of + {ok, CSock} -> + {ok, State#{csock => CSock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, accept), + ok + end}, + + #{desc => "await continue (nowait recv)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv) + end}, + #{desc => "try recv request (with nowait, expect select)", + cmd => fun(#{csock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, {select_info, T, R} = SelectInfo} -> + ?SEV_IPRINT("recv select: " + "~n Tag: ~p" + "~n Ref: ~p", [T, R]), + {ok, State#{recv_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await select message", + cmd => fun(#{csock := Sock}) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}} + after 5000 -> + ok + end + end}, + #{desc => "announce ready (no select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, no_select), + ok + end}, + #{desc => "await continue (cancel)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, cancel) + end}, + #{desc => "cancel", + cmd => fun(#{csock := Sock, recv_select_info := SelectInfo}) -> + ok = socket:cancel(Sock, SelectInfo) + end}, + #{desc => "announce ready (cancel)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, cancel), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close connection socket", + cmd => fun(#{csock := Sock}) -> + socket:close(Sock) + end}, + #{desc => "close listen socket", + cmd => fun(#{lsock := Sock}) -> + socket:close(Sock) + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + ClientSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + {Tester, Port} = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester, server_port => Port}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** The init part *** + #{desc => "which server (local) address", + cmd => fun(#{domain := Domain, server_port := Port} = State) -> + LSA = which_local_socket_addr(Domain), + SSA = LSA#{port => Port}, + {ok, State#{local_sa => LSA, server_sa => SSA}} + end}, + #{desc => "create socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{sock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{sock := Sock, local_sa := LSA} = _State) -> + case socket:bind(Sock, LSA) of + {ok, _Port} -> + ok; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + + %% *** The actual test *** + #{desc => "await continue (connect)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, connect) + end}, + #{desc => "connect to server", + cmd => fun(#{sock := Sock, server_sa := SSA}) -> + socket:connect(Sock, SSA) + end}, + #{desc => "announce ready (connect)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, connect), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close socket", + cmd => fun(#{sock := Sock}) -> + socket:close(Sock) + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor client", + cmd => fun(#{client := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, Port} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{server_port => Port}} + end}, + + %% Start the client + #{desc => "order client start", + cmd => fun(#{client := Pid, server_port := Port} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Port), + ok + end}, + #{desc => "await client ready (init)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, init) + end}, + + %% *** The actual test *** + #{desc => "order server to continue (with accept)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Server, accept), + ok + end}, + #{desc => "order client to continue (connect)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, connect), + ok + end}, + #{desc => "await client ready (connect)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, client, connect) + end}, + #{desc => "await server ready (accept)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, accept) + end}, + + #{desc => "order server to continue (recv)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await server ready (recv select)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, recv_select) + end}, + #{desc => "await server ready (no select)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, no_select) + end}, + #{desc => "order server to continue (send request)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, cancel), + ok + end}, + #{desc => "await server ready (cancel)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, server, cancel) + end}, + + %% *** Termination *** + #{desc => "order client to terminate", + cmd => fun(#{client := Client} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Client), + ok + end}, + #{desc => "await client termination", + cmd => fun(#{client := Client} = State) -> + ?SEV_AWAIT_TERMINATION(Client), + State1 = maps:remove(client, State), + {ok, State1} + end}, + #{desc => "order server to terminate", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Server), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Server} = State) -> + ?SEV_AWAIT_TERMINATION(Server), + State1 = maps:remove(server, State), + {ok, State1} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + Server = ?SEV_START("server", ServerSeq, InitState), + + i("start client evaluator"), + Client = ?SEV_START("client", ClientSeq, InitState), + + i("start tester evaluator"), + TesterInitState = #{server => Server#ev.pid, + client => Client#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator(s)"), + ok = ?SEV_AWAIT_FINISH([Server, Client, Tester]). + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make multiple async (Timeout = nowait) call(s) to recvfrom +%% (from *several* processes), wait some time and then cancel. +%% This should result in abort messages to the 'other' processes. +%% +api_a_mrecvfrom_cancel_udp4(suite) -> + []; +api_a_mrecvfrom_cancel_udp4(doc) -> + []; +api_a_mrecvfrom_cancel_udp4(_Config) when is_list(_Config) -> + ?TT(?SECS(20)), + tc_try(api_a_mrecvfrom_cancel_udp4, + fun() -> + Recv = fun(Sock) -> + case socket:recvfrom(Sock, 0, nowait) of + {ok, _} = OK -> + OK; + {select, _} = SELECT -> + SELECT; + {error, _} = ERROR -> + ERROR + end + end, + InitState = #{domain => inet, + recv => Recv}, + ok = api_a_mrecv_cancel_udp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make multiple async (Timeout = nowait) call(s) to recvmsg +%% (from *several* processes), wait some time and then cancel. +%% This should result in abort messages to the 'other' processes. +%% +api_a_mrecvmsg_cancel_udp4(suite) -> + []; +api_a_mrecvmsg_cancel_udp4(doc) -> + []; +api_a_mrecvmsg_cancel_udp4(_Config) when is_list(_Config) -> + ?TT(?SECS(20)), + tc_try(api_a_mrecvmsg_cancel_udp4, + fun() -> + Recv = fun(Sock) -> + case socket:recvmsg(Sock, nowait) of + {ok, _} = OK -> + OK; + {select, _} = SELECT -> + SELECT; + {error, _} = ERROR -> + ERROR + end + end, + InitState = #{domain => inet, + recv => Recv}, + ok = api_a_mrecv_cancel_udp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_mrecv_cancel_udp(InitState) -> + ServerSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{local_sa => LSA}} + end}, + #{desc => "create socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, dgram, udp) of + {ok, Sock} -> + {ok, State#{sock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind socket (to local address)", + cmd => fun(#{sock := Sock, local_sa := LSA} = State) -> + case socket:bind(Sock, LSA) of + {ok, Port} -> + {ok, State#{port => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, sock := Sock}) -> + ?SEV_ANNOUNCE_READY(Tester, init, Sock), + ok + end}, + + %% The actual test + #{desc => "await continue (recv)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv) + end}, + #{desc => "try recv request (with nowait, expect select)", + cmd => fun(#{sock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, SelectInfo} -> + {ok, State#{recv_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await abort message", + cmd => fun(#{sock := Sock, + recv_select_info := {select_info, _, Ref}} = State) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}}; + {'$socket', Sock, abort, {Ref, closed}} -> + {ok, maps:remove(sock, State)} + after 5000 -> + ?SEV_EPRINT("message queue: ~p", [mq()]), + {error, timeout} + end + end}, + #{desc => "announce ready (abort)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, abort), + ok + end}, + + %% Termination + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + State2 = maps:remove(tester, State), + State3 = maps:remove(recv_stag, State2), + State4 = maps:remove(recv_sref, State3), + State5 = maps:remove(req_src, State4), + {ok, State5}; + {error, _} = ERROR -> + ERROR + end + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + AltServerSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start", + cmd => fun(State) -> + {Tester, Sock} = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester, sock => Sock}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + + %% The actual test + #{desc => "await continue (recv)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv) + end}, + #{desc => "try recv request (with nowait, expect select)", + cmd => fun(#{sock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, SelectInfo} -> + {ok, State#{recv_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await abort message", + cmd => fun(#{sock := Sock, + recv_select_info := {select_info, _, Ref}} = State) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}}; + {'$socket', Sock, abort, {Ref, closed}} -> + {ok, maps:remove(sock, State)} + after 5000 -> + ?SEV_EPRINT("message queue: ~p", [mq()]), + {error, timeout} + end + end}, + #{desc => "announce ready (abort)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, abort), + ok + end}, + + %% Termination + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + ?SEV_IPRINT("terminating"), + State1 = maps:remove(recv_select_info, State), + State2 = maps:remove(tester, State1), + State3 = maps:remove(sock, State2), + {ok, State3}; + {error, _} = ERROR -> + ERROR + end + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor alt-server 1", + cmd => fun(#{alt_server1 := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor alt-server 2", + cmd => fun(#{alt_server2 := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, Sock} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{sock => Sock}} + end}, + + %% Start the alt-server 1 + #{desc => "order alt-server 1 start", + cmd => fun(#{alt_server1 := Pid, sock := Sock} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Sock), + ok + end}, + #{desc => "await alt-server 1 ready (init)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, alt_server1, init) + end}, + + %% Start the alt-server 2 + #{desc => "order alt-server 2 start", + cmd => fun(#{alt_server2 := Pid, sock := Sock} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Sock), + ok + end}, + #{desc => "await alt-server 2 ready (init)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, alt_server2, init) + end}, + + + %% The actual test + #{desc => "order server continue (recv)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await server ready (recv select)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, recv_select) + end}, + + #{desc => "order alt-server 1 continue (recv)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await alt-server 1 ready (recv select)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server1, recv_select) + end}, + + #{desc => "order alt-server 2 continue (recv)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await alt-server 2 ready (recv select)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server2, recv_select) + end}, + + ?SEV_SLEEP(?SECS(1)), + + #{desc => "close the socket", + cmd => fun(#{sock := Sock} = _State) -> + socket:close(Sock) + end}, + + #{desc => "await server ready (abort)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, abort) + end}, + #{desc => "await alt-server 1 ready (abort)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server1, abort) + end}, + #{desc => "await alt-server 2 ready (abort)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server2, abort) + end}, + + %% Terminations + #{desc => "order alt-server 2 to terminate", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await alt-server 2 termination", + cmd => fun(#{alt_server2 := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(alt_server2, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + #{desc => "order alt-server 1 to terminate", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await alt-server 1 termination", + cmd => fun(#{alt_server1 := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(alt_server1, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + #{desc => "order server to terminate", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(server, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + Server = ?SEV_START("server", ServerSeq, InitState), + + i("start alt-server 1 evaluator"), + AltServer1 = ?SEV_START("alt_server1", AltServerSeq, InitState), + + i("start alt-server 2 evaluator"), + AltServer2 = ?SEV_START("alt_server2", AltServerSeq, InitState), + + i("start 'tester' evaluator"), + TesterInitState = #{server => Server#ev.pid, + alt_server1 => AltServer1#ev.pid, + alt_server2 => AltServer2#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator(s)"), + ok = ?SEV_AWAIT_FINISH([Server, AltServer1, AltServer2, Tester]). + + + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make multiple async (Timeout = nowait) call(s) to accept +%% (from *several* processes), wait some time and then cancel, +%% This should result in abort messages to the 'other' processes. +%% +api_a_maccept_cancel_tcp4(suite) -> + []; +api_a_maccept_cancel_tcp4(doc) -> + []; +api_a_maccept_cancel_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(20)), + tc_try(api_a_maccept_cancel_tcp4, + fun() -> + Accept = fun(Sock) -> + case socket:accept(Sock, nowait) of + {ok, _} = OK -> + OK; + {select, _} = SELECT -> + SELECT; + {error, _} = ERROR -> + ERROR + end + end, + InitState = #{domain => inet, + accept => Accept}, + ok = api_a_maccept_cancel_tcp(InitState) + end). + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_maccept_cancel_tcp(InitState) -> + process_flag(trap_exit, true), + ServerSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{lsa => LSA}} + end}, + #{desc => "create listen socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{lsock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{lsock := LSock, lsa := LSA} = State) -> + case socket:bind(LSock, LSA) of + {ok, Port} -> + {ok, State#{lport => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "make listen socket", + cmd => fun(#{lsock := LSock}) -> + socket:listen(LSock) + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, lsock := Sock}) -> + ?SEV_ANNOUNCE_READY(Tester, init, Sock), + ok + end}, + + %% The actual test + #{desc => "await continue (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, accept) + end}, + #{desc => "await connection (nowait)", + cmd => fun(#{lsock := LSock, accept := Accept} = State) -> + case Accept(LSock) of + {select, {select_info, T, R} = SelectInfo} -> + ?SEV_IPRINT("accept select: " + "~n T: ~p" + "~n R: ~p", [T, R]), + {ok, State#{accept_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (accept_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, accept_select), + ok + end}, + #{desc => "await select message (without success)", + cmd => fun(#{lsock := Sock, + accept_select_info := {select_info, _, Ref}} = State) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}}; + {'$socket', Sock, abort, {Ref, closed}} -> + {ok, maps:remove(lsock, State)} + after 5000 -> + ?SEV_EPRINT("message queue: ~p", [mq()]), + {error, timeout} + end + end}, + #{desc => "announce ready (abort)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, abort), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + AltServerSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start", + cmd => fun(State) -> + {Tester, Sock} = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester, lsock => Sock}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + + %% The actual test + #{desc => "await continue (accept)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, accept) + end}, + #{desc => "try accept request (with nowait, expect select)", + cmd => fun(#{lsock := Sock, accept := Accept} = State) -> + case Accept(Sock) of + {select, SelectInfo} -> + {ok, State#{accept_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (accept_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, accept_select), + ok + end}, + #{desc => "await abort message", + cmd => fun(#{lsock := Sock, + accept_select_info := {select_info, _, Ref}} = State) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}}; + {'$socket', Sock, abort, {Ref, closed}} -> + {ok, maps:remove(sock, State)} + after 5000 -> + ?SEV_EPRINT("message queue: ~p", [mq()]), + {error, timeout} + end + end}, + #{desc => "announce ready (abort)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, abort), + ok + end}, + + %% Termination + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + ?SEV_IPRINT("terminating"), + State1 = maps:remove(tester, State), + State2 = maps:remove(accept_select_info, State1), + State3 = maps:remove(lsock, State2), + {ok, State3}; + {error, _} = ERROR -> + ERROR + end + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor alt-server 1", + cmd => fun(#{alt_server1 := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor alt-server 2", + cmd => fun(#{alt_server2 := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, Sock} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{sock => Sock}} + end}, + + %% Start the alt-server 1 + #{desc => "order alt-server 1 start", + cmd => fun(#{alt_server1 := Pid, sock := Sock} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Sock), + ok + end}, + #{desc => "await alt-server 1 ready (init)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, alt_server1, init) + end}, + + %% Start the alt-server 2 + #{desc => "order alt-server 2 start", + cmd => fun(#{alt_server2 := Pid, sock := Sock} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Sock), + ok + end}, + #{desc => "await alt-server 2 ready (init)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, alt_server2, init) + end}, + + + %% *** The actual test *** + #{desc => "order server continue (accept)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, accept), + ok + end}, + #{desc => "await server ready (accept select)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, accept_select) + end}, + + #{desc => "order alt-server 1 continue (accept)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, accept), + ok + end}, + #{desc => "await alt-server 1 ready (accept select)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server1, accept_select) + end}, + + #{desc => "order alt-server 2 continue (accept)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, accept), + ok + end}, + #{desc => "await alt-server 2 ready (accept select)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server2, accept_select) + end}, + + ?SEV_SLEEP(?SECS(1)), + + #{desc => "close the socket", + cmd => fun(#{sock := Sock} = _State) -> + socket:close(Sock) + end}, + + #{desc => "await server ready (abort)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, abort) + end}, + #{desc => "await alt-server 1 ready (abort)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server1, abort) + end}, + #{desc => "await alt-server 2 ready (abort)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server2, abort) + end}, + + + %% *** Termination *** + #{desc => "order alt-server 2 to terminate", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await alt-server 2 termination", + cmd => fun(#{alt_server2 := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(alt_server2, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + #{desc => "order alt-server 1 to terminate", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await alt-server 1 termination", + cmd => fun(#{alt_server1 := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(alt_server1, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + #{desc => "order server to terminate", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Server), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Server} = State) -> + ?SEV_AWAIT_TERMINATION(Server), + State1 = maps:remove(server, State), + {ok, State1} + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + Server = ?SEV_START("server", ServerSeq, InitState), + + i("start alt-server 1 evaluator"), + AltServer1 = ?SEV_START("alt_server1", AltServerSeq, InitState), + + i("start alt-server 2 evaluator"), + AltServer2 = ?SEV_START("alt_server2", AltServerSeq, InitState), + + i("start tester evaluator"), + TesterInitState = #{server => Server#ev.pid, + alt_server1 => AltServer1#ev.pid, + alt_server2 => AltServer2#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator(s)"), + ok = ?SEV_AWAIT_FINISH([Server, AltServer1, AltServer2, Tester]). + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make multiple async (Timeout = nowait) call(s) to recv +%% (from *several* processes), wait some time and then cancel, +%% This should result in abort messages to the 'other' processes. +%% +api_a_mrecv_cancel_tcp4(suite) -> + []; +api_a_mrecv_cancel_tcp4(doc) -> + []; +api_a_mrecv_cancel_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(20)), + tc_try(api_a_mrecv_cancel_tcp4, + fun() -> + Recv = fun(Sock) -> + socket:recv(Sock, 0, nowait) + end, + InitState = #{domain => inet, + recv => Recv}, + ok = api_a_mrecv_cancel_tcp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically we make multiple async (Timeout = nowait) call(s) to recvmsg +%% (from *several* processes), wait some time and then cancel, +%% This should result in abort messages to the 'other' processes. +%% +api_a_mrecvmsg_cancel_tcp4(suite) -> + []; +api_a_mrecvmsg_cancel_tcp4(doc) -> + []; +api_a_mrecvmsg_cancel_tcp4(_Config) when is_list(_Config) -> + ?TT(?SECS(20)), + tc_try(api_a_mrecvmsg_cancel_tcp4, + fun() -> + Recv = fun(Sock) -> + socket:recvmsg(Sock, nowait) + end, + InitState = #{domain => inet, + recv => Recv}, + ok = api_a_mrecv_cancel_tcp(InitState) + end). + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_a_mrecv_cancel_tcp(InitState) -> + process_flag(trap_exit, true), + ServerSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "which local address", + cmd => fun(#{domain := Domain} = State) -> + LSA = which_local_socket_addr(Domain), + {ok, State#{lsa => LSA}} + end}, + #{desc => "create listen socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{lsock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{lsock := LSock, lsa := LSA} = State) -> + case socket:bind(LSock, LSA) of + {ok, Port} -> + {ok, State#{lport => Port}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "make listen socket", + cmd => fun(#{lsock := LSock}) -> + socket:listen(LSock) + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, lport := Port}) -> + ?SEV_ANNOUNCE_READY(Tester, init, Port), + ok + end}, + + %% The actual test + #{desc => "await continue (accept)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, accept) + end}, + #{desc => "await connection (nowait)", + cmd => fun(#{lsock := LSock} = State) -> + case socket:accept(LSock) of + {ok, CSock} -> + {ok, State#{csock => CSock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (accept)", + cmd => fun(#{tester := Tester, csock := Sock}) -> + ?SEV_ANNOUNCE_READY(Tester, accept, Sock), + ok + end}, + + #{desc => "await continue (nowait recv)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv) + end}, + #{desc => "try recv request (with nowait, expect select)", + cmd => fun(#{csock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, {select_info, T, R} = SelectInfo} -> + ?SEV_IPRINT("recv select: " + "~n Tag: ~p" + "~n Ref: ~p", [T, R]), + {ok, State#{recv_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await select message", + cmd => fun(#{csock := Sock, + recv_select_info := {select_info, _, Ref}} = State) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}}; + {'$socket', Sock, abort, {Ref, closed}} -> + {ok, maps:remove(sock, State)} + after 5000 -> + ok + end + end}, + #{desc => "announce ready (abort)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, abort), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close listen socket", + cmd => fun(#{lsock := Sock}) -> + socket:close(Sock) + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + AltServerSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start", + cmd => fun(State) -> + {Tester, Sock} = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester, sock => Sock}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + + %% The actual test + #{desc => "await continue (recv)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, recv) + end}, + #{desc => "try recv request (with nowait, expect select)", + cmd => fun(#{sock := Sock, recv := Recv} = State) -> + case Recv(Sock) of + {select, SelectInfo} -> + {ok, State#{recv_select_info => SelectInfo}}; + {ok, X} -> + {error, {unexpected_select_info, X}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (recv_select)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, recv_select), + ok + end}, + #{desc => "await abort message", + cmd => fun(#{sock := Sock, + recv_select_info := {select_info, _, Ref}} = State) -> + receive + {'$socket', Sock, select, Ref} -> + {error, {unexpected_select, Ref}}; + {'$socket', Sock, abort, {Ref, closed}} -> + {ok, maps:remove(sock, State)} + after 5000 -> + ?SEV_EPRINT("message queue: ~p", [mq()]), + {error, timeout} + end + end}, + #{desc => "announce ready (abort)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, abort), + ok + end}, + + %% Termination + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + ?SEV_IPRINT("terminating"), + State1 = maps:remove(recv_select_info, State), + State2 = maps:remove(tester, State1), + State3 = maps:remove(sock, State2), + {ok, State3}; + {error, _} = ERROR -> + ERROR + end + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + ClientSeq = + [ + %% *** Wait for start order *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + {Tester, Port} = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester, server_port => Port}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester}) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** The init part *** + #{desc => "which server (local) address", + cmd => fun(#{domain := Domain, server_port := Port} = State) -> + LSA = which_local_socket_addr(Domain), + SSA = LSA#{port => Port}, + {ok, State#{local_sa => LSA, server_sa => SSA}} + end}, + #{desc => "create socket", + cmd => fun(#{domain := Domain} = State) -> + case socket:open(Domain, stream, tcp) of + {ok, Sock} -> + {ok, State#{sock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "bind to local address", + cmd => fun(#{sock := Sock, local_sa := LSA} = _State) -> + case socket:bind(Sock, LSA) of + {ok, _Port} -> + ok; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + + %% *** The actual test *** + #{desc => "await continue (connect)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, connect) + end}, + #{desc => "connect to server", + cmd => fun(#{sock := Sock, server_sa := SSA}) -> + socket:connect(Sock, SSA) + end}, + #{desc => "announce ready (connect)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, connect), + ok + end}, + + %% *** Termination *** + #{desc => "await terminate", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "close socket", + cmd => fun(#{sock := Sock}) -> + socket:close(Sock) + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor server", + cmd => fun(#{server := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor alt-server 1", + cmd => fun(#{alt_server1 := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor alt-server 2", + cmd => fun(#{alt_server2 := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "monitor client", + cmd => fun(#{client := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + + %% Start the server + #{desc => "order server start", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await server ready (init)", + cmd => fun(#{server := Pid} = State) -> + {ok, Port} = ?SEV_AWAIT_READY(Pid, server, init), + {ok, State#{server_port => Port}} + end}, + + %% Start the client + #{desc => "order client start", + cmd => fun(#{client := Pid, server_port := Port} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Port), + ok + end}, + #{desc => "await client ready (init)", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, init) + end}, + + #{desc => "order server to continue (with accept)", + cmd => fun(#{server := Server} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Server, accept), + ok + end}, + #{desc => "order client to continue (connect)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, connect), + ok + end}, + #{desc => "await client ready (connect)", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_AWAIT_READY(Pid, client, connect) + end}, + #{desc => "await server ready (accept)", + cmd => fun(#{server := Pid} = State) -> + {ok, Sock} = ?SEV_AWAIT_READY(Pid, server, accept), + {ok, State#{sock => Sock}} + end}, + + %% Start the alt server 1 + #{desc => "order alt-server 1 start", + cmd => fun(#{alt_server1 := Pid, sock := Sock} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Sock), + ok + end}, + #{desc => "await alt-server 1 ready (init)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server1, init) + end}, + + %% Start the alt server 2 + #{desc => "order alt-server 2 start", + cmd => fun(#{alt_server2 := Pid, sock := Sock} = _State) -> + ?SEV_ANNOUNCE_START(Pid, Sock), + ok + end}, + #{desc => "await alt-server 2 ready (init)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server2, init) + end}, + + + %% *** The actual test *** + #{desc => "order server continue (recv)", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await server ready (recv select)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, recv_select) + end}, + + #{desc => "order alt-server 1 continue (recv)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await alt-server 1 ready (recv select)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server1, recv_select) + end}, + + #{desc => "order alt-server 2 continue (recv)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, recv), + ok + end}, + #{desc => "await alt-server 2 ready (recv select)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server2, recv_select) + end}, + + ?SEV_SLEEP(?SECS(1)), + + #{desc => "close the socket", + cmd => fun(#{sock := Sock} = _State) -> + socket:close(Sock) + end}, + + #{desc => "await server ready (abort)", + cmd => fun(#{server := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, server, abort) + end}, + #{desc => "await alt-server 1 ready (abort)", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server1, abort) + end}, + #{desc => "await alt-server 2 ready (abort)", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, alt_server2, abort) + end}, + + %% Terminations + #{desc => "order client to terminate", + cmd => fun(#{client := Client} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Client), + ok + end}, + #{desc => "await client termination", + cmd => fun(#{client := Client} = State) -> + ?SEV_AWAIT_TERMINATION(Client), + State1 = maps:remove(client, State), + {ok, State1} + end}, + + #{desc => "order alt-server 2 to terminate", + cmd => fun(#{alt_server2 := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await alt-server 2 termination", + cmd => fun(#{alt_server2 := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(alt_server2, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + #{desc => "order alt-server 1 to terminate", + cmd => fun(#{alt_server1 := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await alt-server 1 termination", + cmd => fun(#{alt_server1 := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(alt_server1, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + #{desc => "order server to terminate", + cmd => fun(#{server := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await server termination", + cmd => fun(#{server := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + State1 = maps:remove(server, State), + {ok, State1}; + {error, _} = ERROR -> + ERROR + end + end}, + + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start server evaluator"), + Server = ?SEV_START("server", ServerSeq, InitState), + + i("start alt-server 1 evaluator"), + AltServer1 = ?SEV_START("alt_server1", AltServerSeq, InitState), + + i("start alt-server 2 evaluator"), + AltServer2 = ?SEV_START("alt_server2", AltServerSeq, InitState), + + i("start client evaluator"), + Client = ?SEV_START("client", ClientSeq, InitState), + + i("start tester evaluator"), + TesterInitState = #{server => Server#ev.pid, + alt_server1 => AltServer1#ev.pid, + alt_server2 => AltServer2#ev.pid, + client => Client#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator(s)"), + ok = ?SEV_AWAIT_FINISH([Server, AltServer1, AltServer2, Client, Tester]). + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% %% %% API OPTIONS %% @@ -19206,7 +23089,7 @@ which_ttest_runtime_env(false) -> %% ms: milliseconds %% s: seconds (default) %% m: minutes -which_ttest_runtime_env2([$m, $s | MS]) when (length(MS) > 0) -> +which_ttest_runtime_env2([$s, $m | MS]) when (length(MS) > 0) -> convert_time(MS, fun(X) -> X end); which_ttest_runtime_env2([$m | M]) when (length(M) > 0) -> convert_time(M, fun(X) -> ?MINS(X) end); @@ -19631,12 +23514,23 @@ ttest_tcp(InitState) -> %% Present the results #{desc => "present the results", - cmd => fun(#{result := Result} = State) -> + cmd => fun(#{result := Result, + domain := Domain, + server_mod := ServerTrans, + server_active := ServerActive, + client_mod := ClientTrans, + client_active := ClientActive, + msg_id := MsgID} = State) -> case Result of #{status := ok, runtime := RunTime, cnt := Cnt, bcnt := BCnt} -> + ttest_report(Domain, + ServerTrans, ServerActive, + ClientTrans, ClientActive, + MsgID, + RunTime, BCnt, Cnt), ?SEV_IPRINT( "TTest results: " "~n Run Time: ~s" @@ -19729,7 +23623,9 @@ ttest_tcp_server_start(Node, _Domain, gen, Active) -> socket_test_ttest_tcp_server:start_monitor(Node, Transport, Active); ttest_tcp_server_start(Node, Domain, sock, Active) -> TransportMod = socket_test_ttest_tcp_socket, - Transport = {TransportMod, #{domain => Domain, method => plain}}, + Transport = {TransportMod, #{domain => Domain, + async => true, + method => plain}}, socket_test_ttest_tcp_server:start_monitor(Node, Transport, Active). ttest_tcp_server_stop(Pid) -> @@ -19751,7 +23647,9 @@ ttest_tcp_client_start(Node, Domain, sock, ServerInfo, Active, MsgID, MaxOutstanding, RunTime) -> TransportMod = socket_test_ttest_tcp_socket, - Transport = {TransportMod, #{domain => Domain, method => plain}}, + Transport = {TransportMod, #{domain => Domain, + async => true, + method => plain}}, socket_test_ttest_tcp_client:start_monitor(Node, Notify, Transport, @@ -19762,6 +23660,183 @@ ttest_tcp_client_start(Node, %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +-define(TTEST_MANAGER, esock_ttest_manager). + +-record(ttest_report_id, + {domain :: socket:domain(), + serv_trans :: gen | sock, + serv_active :: once | boolean(), + client_trans :: gen | sock, + client_active :: once | boolean(), + msg_id :: small | medium | large}). + +-record(ttest_report, {id :: #ttest_report_id{}, + time :: non_neg_integer(), + bytes :: non_neg_integer(), + msgs :: non_neg_integer()}). + +-spec ttest_report(Domain :: socket:domain(), + ServTrans :: gen | sock, ServActive :: once | boolean(), + ClientTrans :: gen | sock, ClientActive :: once | boolean(), + MsgID :: 1 | 2 | 3, + RunTime :: non_neg_integer(), + NumBytes :: non_neg_integer(), + NumMsgs :: non_neg_integer()) -> ok. + +ttest_report(Domain, + ServTrans, ServActive, + ClientTrans, ClientActive, + MsgID, + RunTime, + NumBytes, + NumMsgs) -> + ID = #ttest_report_id{domain = Domain, + serv_trans = ServTrans, + serv_active = ServActive, + client_trans = ClientTrans, + client_active = ClientActive, + msg_id = ttest_msg_id_num_to_name(MsgID)}, + Report = #ttest_report{id = ID, + time = RunTime, + bytes = NumBytes, + msgs = NumMsgs}, + %% If we run just one test case, the group init has never been run + %% and therefor the ttest manager is not running (we also don't actually + %% care about collecting reports in that case). + (catch global:send(?TTEST_MANAGER, Report)), + ok. + +ttest_msg_id_num_to_name(1) -> + small; +ttest_msg_id_num_to_name(2) -> + medium; +ttest_msg_id_num_to_name(3) -> + large. + +ttest_manager_start() -> + Self = self(), + {Pid, MRef} = spawn_monitor(fun() -> ttest_manager_init(Self) end), + receive + {ttest_manager_started, Pid} -> + erlang:demonitor(MRef, [flush]), + ok; + {'DOWN', MRef, process, Pid, Reason} -> + exit({failed_starting, ttest_manager, Reason}) + after 5000 -> + exit(Pid, kill), + exit({failed_starting, ttest_manager, timeout}) + end. + +ttest_manager_stop() -> + case global:whereis_name(?TTEST_MANAGER) of + Pid when is_pid(Pid) -> + erlang:monitor(process, Pid), + global:send(?TTEST_MANAGER, stop), + receive + {'DOWN', _MRef, process, Pid, _} -> + ok + after 10000 -> + exit(Pid, kill), + ok + end; + _ -> + ok + end. + +ttest_manager_init(Parent) -> + yes = global:register_name(?TTEST_MANAGER, self()), + ets:new(?TTEST_MANAGER, + [{keypos, #ttest_report.id}, named_table, protected, ordered_set]), + Parent ! {ttest_manager_started, self()}, + ttest_manager_loop(). + +ttest_manager_loop() -> + receive + stop -> + ?LOGGER:format("manager stopping~n", []), + ttest_manager_done(); + + #ttest_report{id = _ID, + time = _RunTime, + bytes = _NumBytes, + msgs = _NumMsgs} = Report -> + true = ets:insert_new(?TTEST_MANAGER, Report), + ttest_manager_loop() + end. + +%% We are supposed to pretty print the result here... +ttest_manager_done() -> + format_reports(inet), + %% format_reports(inet6), + ets:delete(?TTEST_MANAGER), + exit(normal). + +format_reports(Domain) -> + ?LOGGER:format("Domain ~w reports:~n~n", [Domain]), + format_reports(Domain, small), + format_reports(Domain, medium), + format_reports(Domain, large). + +format_reports(Domain, MsgID) when is_atom(MsgID) -> + case which_ttest_reports(Domain, MsgID) of + [] -> + ?LOGGER:format(" No ~w reports~n~n", [MsgID]); + Reports -> + ?LOGGER:format(" ~w reports: ~n", [MsgID]), + lists:foreach(fun(R) -> format_report(R) end, Reports) + end. + +%% This should really be a table like this: +%% +%% client +%% server gen(false) gen(once) gen(true) sock(false) sock(once) sock(true) +%% gen(false) nnn +%% gen(once) nnn +%% gen(true) nnn +%% sock(false) nnn +%% sock(once) nnn +%% sock(true) nnn +%% +format_report(#ttest_report{id = #ttest_report_id{serv_trans = STrans, + serv_active = SActive, + client_trans = CTrans, + client_active = CActive}, + time = RunTime, + bytes = BCnt, + msgs = MCnt}) -> + ?LOGGER:format(" server ~w[~w] - client ~w[~w] => " + "~n Run Time: ~s" + "~n Bytes: ~s" + "~n Messages: ~s" + "~n", [STrans, SActive, CTrans, CActive, + ?TTEST_LIB:format_time(RunTime), + if ((BCnt =:= 0) orelse (RunTime =:= 0)) -> + ?TTEST_LIB:format("~w, ~w", + [BCnt, RunTime]); + true -> + ?TTEST_LIB:format("~p => ~p byte / ms", + [BCnt, BCnt div RunTime]) + end, + if (RunTime =:= 0) -> + "-"; + true -> + ?TTEST_LIB:format("~p => ~p iterations / ms", + [MCnt, MCnt div RunTime]) + end]), + ok. + + +which_ttest_reports(Domain, all) -> + [R || R = #ttest_report{id = #ttest_report_id{domain = D}} <- + ets:tab2list(?TTEST_MANAGER), Domain =:= D]; +which_ttest_reports(Domain, MsgID) -> + [R || R = #ttest_report{id = #ttest_report_id{domain = D, msg_id = MID}} <- + ets:tab2list(?TTEST_MANAGER), (Domain =:= D) andalso (MsgID =:= MID)]. + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + %% This mechanism has only one purpose: So that we are able to kill %% the node-starter process if it takes to long. The node-starter %% runs on the local node. @@ -19982,9 +24057,13 @@ which_addr(Domain, [_|IFL]) -> which_addr2(_Domain, []) -> {error, no_address}; -which_addr2(inet = _Domain, [{addr, Addr}|_IFO]) when (size(Addr) =:= 4) -> +which_addr2(inet = _Domain, [{addr, Addr}|_IFO]) + when (size(Addr) =:= 4) andalso (element(1, Addr) =/= 127) -> {ok, Addr}; -which_addr2(inet6 = _Domain, [{addr, Addr}|_IFO]) when (size(Addr) =:= 8) -> +which_addr2(inet6 = _Domain, [{addr, Addr}|_IFO]) + when (size(Addr) =:= 8) andalso + (element(1, Addr) =/= 0) andalso + (element(1, Addr) =/= 16#fe80) -> {ok, Addr}; which_addr2(Domain, [_|IFO]) -> which_addr2(Domain, IFO). diff --git a/erts/emulator/test/socket_test_logger.erl b/erts/emulator/test/socket_test_logger.erl index 26610e9ef3..f5d4c8c7b2 100644 --- a/erts/emulator/test/socket_test_logger.erl +++ b/erts/emulator/test/socket_test_logger.erl @@ -43,7 +43,7 @@ start(Quiet) -> ok; undefined -> Self = self(), - Pid = spawn_link(fun() -> init(Self, Quiet) end), + Pid = spawn(fun() -> init(Self, Quiet) end), yes = global:register_name(?LOGGER, Pid), ok end. diff --git a/erts/emulator/test/socket_test_ttest_tcp_client_socket.erl b/erts/emulator/test/socket_test_ttest_tcp_client_socket.erl index ccace2a560..ca7eff4437 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_client_socket.erl +++ b/erts/emulator/test/socket_test_ttest_tcp_client_socket.erl @@ -21,89 +21,95 @@ -module(socket_test_ttest_tcp_client_socket). -export([ - start/3, start/4, start/6, start/7, + start/4, start/5, start/7, start/8, stop/1 ]). -define(TRANSPORT_MOD, socket_test_ttest_tcp_socket). --define(MOD(D, M), {?TRANSPORT_MOD, #{domain => D, method => M}}). +-define(MOD(D, A, M), {?TRANSPORT_MOD, #{domain => D, + async => A, + method => M}}). -start(Method, ServerInfo, Active) +start(Method, Async, Active, ServerInfo) when is_list(ServerInfo) -> Domain = local, - socket_test_ttest_tcp_client:start_monitor(?MOD(Domain, Method), - ServerInfo, Active); -start(Method, ServerInfo = {Addr, _}, Active) + socket_test_ttest_tcp_client:start_monitor(?MOD(Domain, Async, Method), + Active, ServerInfo); +start(Method, Async, Active, ServerInfo = {Addr, _}) when is_tuple(Addr) andalso (size(Addr) =:= 4) -> Domain = inet, - socket_test_ttest_tcp_client:start_monitor(?MOD(Domain, Method), - ServerInfo, Active); -start(Method, ServerInfo = {Addr, _}, Active) + socket_test_ttest_tcp_client:start_monitor(?MOD(Domain, Async, Method), + Active, ServerInfo); +start(Method, Async, Active, ServerInfo = {Addr, _}) when is_tuple(Addr) andalso (size(Addr) =:= 8) -> Domain = inet6, - socket_test_ttest_tcp_client:start_monitor(?MOD(Domain, Method), - ServerInfo, Active). + socket_test_ttest_tcp_client:start_monitor(?MOD(Domain, Async, Method), + Active, ServerInfo). -start(Method, ServerInfo, Active, MsgID) +start(Method, Async, Active, ServerInfo, MsgID) when is_list(ServerInfo) -> %% This is just a simplification Domain = local, - socket_test_ttest_tcp_client:start(?MOD(Domain, Method), - ServerInfo, Active, MsgID); -start(Method, ServerInfo = {Addr, _}, Active, MsgID) + socket_test_ttest_tcp_client:start(?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID); +start(Method, Async, Active, ServerInfo = {Addr, _}, MsgID) when is_tuple(Addr) andalso (size(Addr) =:= 4) -> %% This is just a simplification Domain = inet, - socket_test_ttest_tcp_client:start(?MOD(Domain, Method), - ServerInfo, Active, MsgID); -start(Method, ServerInfo = {Addr, _}, Active, MsgID) + socket_test_ttest_tcp_client:start(?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID); +start(Method, Async, Active, ServerInfo = {Addr, _}, MsgID) when is_tuple(Addr) andalso (size(Addr) =:= 8) -> Domain = inet6, - socket_test_ttest_tcp_client:start(?MOD(Domain, Method), - ServerInfo, Active, MsgID). + socket_test_ttest_tcp_client:start(?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID). -start(Method, ServerInfo, Active, MsgID, MaxOutstanding, RunTime) +start(Method, Async, Active, ServerInfo, MsgID, MaxOutstanding, RunTime) when is_list(ServerInfo) -> Domain = local, socket_test_ttest_tcp_client:start(false, - ?MOD(Domain, Method), - ServerInfo, Active, + ?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID, MaxOutstanding, RunTime); -start(Method, ServerInfo = {Addr, _}, Active, MsgID, MaxOutstanding, RunTime) +start(Method, Async, Active, ServerInfo = {Addr, _}, + MsgID, MaxOutstanding, RunTime) when is_tuple(Addr) andalso (size(Addr) =:= 4) -> Domain = inet, socket_test_ttest_tcp_client:start(false, - ?MOD(Domain, Method), - ServerInfo, Active, + ?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID, MaxOutstanding, RunTime); -start(Method, ServerInfo = {Addr, _}, Active, MsgID, MaxOutstanding, RunTime) +start(Method, Async, Active, ServerInfo = {Addr, _}, + MsgID, MaxOutstanding, RunTime) when is_tuple(Addr) andalso (size(Addr) =:= 8) -> Domain = inet6, socket_test_ttest_tcp_client:start(false, - ?MOD(Domain, Method), - ServerInfo, Active, + ?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID, MaxOutstanding, RunTime). -start(Quiet, Method, ServerInfo, Active, MsgID, MaxOutstanding, RunTime) +start(Quiet, Async, Active, Method, ServerInfo, MsgID, MaxOutstanding, RunTime) when is_list(ServerInfo) -> Domain = local, socket_test_ttest_tcp_client:start(Quiet, - ?MOD(Domain, Method), - ServerInfo, Active, + ?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID, MaxOutstanding, RunTime); -start(Quiet, Method, ServerInfo = {Addr, _}, Active, MsgID, MaxOutstanding, RunTime) +start(Quiet, Async, Active, Method, ServerInfo = {Addr, _}, + MsgID, MaxOutstanding, RunTime) when is_tuple(Addr) andalso (size(Addr) =:= 4) -> Domain = inet, socket_test_ttest_tcp_client:start(Quiet, - ?MOD(Domain, Method), - ServerInfo, Active, + ?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID, MaxOutstanding, RunTime); -start(Quiet, Method, ServerInfo = {Addr, _}, Active, MsgID, MaxOutstanding, RunTime) +start(Quiet, Async, Active, Method, ServerInfo = {Addr, _}, + MsgID, MaxOutstanding, RunTime) when is_tuple(Addr) andalso (size(Addr) =:= 8) -> Domain = inet6, socket_test_ttest_tcp_client:start(Quiet, - ?MOD(Domain, Method), - ServerInfo, Active, + ?MOD(Domain, Async, Method), + Active, ServerInfo, MsgID, MaxOutstanding, RunTime). stop(Pid) -> diff --git a/erts/emulator/test/socket_test_ttest_tcp_server.erl b/erts/emulator/test/socket_test_ttest_tcp_server.erl index e916fcb93e..27b561d4b7 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_server.erl +++ b/erts/emulator/test/socket_test_ttest_tcp_server.erl @@ -90,7 +90,9 @@ start_monitor(_, Transport, Active) -> start(Transport, Active) -> do_start(self(), Transport, Active). - +%% Note that the Async option is actually only "used" for the +%% socket transport module (it details how to implement the +%% active feature). do_start(Parent, Transport, Active) when is_pid(Parent) andalso (is_atom(Transport) orelse is_tuple(Transport)) andalso @@ -179,42 +181,59 @@ process_transport({Mod, Opts}, _Active) -> server_loop(State) -> - server_loop( server_handle_message( server_accept(State) ) ). + server_loop( server_handle_message( server_accept(State, ?ACC_TIMEOUT), 0) ). -server_accept(#{mod := Mod, - active := Active, - lsock := LSock, - handlers := Handlers} = State) -> - case Mod:accept(LSock, ?ACC_TIMEOUT) of +server_accept(#{mod := Mod, lsock := LSock} = State, Timeout) -> + case Mod:accept(LSock, Timeout) of {ok, Sock} -> - ?I("accepted connection from ~s", - [case Mod:peername(Sock) of - {ok, Peer} -> - format_peername(Peer); - {error, _} -> - "-" - end]), - {Pid, _} = handler_start(), - ?I("handler ~p started -> try transfer socket control", [Pid]), - case Mod:controlling_process(Sock, Pid) of - ok -> - maybe_start_stats_timer(State, Pid), - ?I("server-accept: handler ~p started", [Pid]), - handler_continue(Pid, Mod, Sock, Active), - Handlers2 = [Pid | Handlers], - State#{handlers => Handlers2}; - {error, CPReason} -> - (catch Mod:close(Sock)), - (catch Mod:close(LSock)), - exit({controlling_process, CPReason}) - end; - {error, timeout} -> + server_handle_accepted(State, Sock); + {error, timeout} when (Timeout =/= nowait) -> State; {error, AReason} -> (catch Mod:close(LSock)), exit({accept, AReason}) end. +%% server_accept(#{mod := Mod, +%% lsock := LSock} = State) -> +%% case Mod:accept(LSock, ?ACC_TIMEOUT) of +%% {ok, Sock} -> +%% server_handle_accepted(State, Sock); +%% {error, timeout} -> +%% State; +%% {error, AReason} -> +%% (catch Mod:close(LSock)), +%% exit({accept, AReason}) +%% end. + +server_handle_accepted(#{mod := Mod, + lsock := LSock, + active := Active, + handlers := Handlers} = State, + Sock) -> + ?I("accepted connection from ~s", + [case Mod:peername(Sock) of + {ok, Peer} -> + format_peername(Peer); + {error, _} -> + "-" + end]), + {Pid, _} = handler_start(), + ?I("handler ~p started -> try transfer socket control", [Pid]), + case Mod:controlling_process(Sock, Pid) of + ok -> + maybe_start_stats_timer(State, Pid), + ?I("server-accept: handler ~p started", [Pid]), + handler_continue(Pid, Mod, Sock, Active), + Handlers2 = [Pid | Handlers], + State#{handlers => Handlers2}; + {error, CPReason} -> + (catch Mod:close(Sock)), + (catch Mod:close(LSock)), + exit({controlling_process, CPReason}) + end. + + format_peername({Addr, Port}) -> case inet:gethostbyaddr(Addr) of {ok, #hostent{h_name = N}} -> @@ -237,7 +256,7 @@ start_stats_timer(Time, ProcStr, Pid) -> server_handle_message(#{mod := Mod, lsock := LSock, parent := Parent, - handlers := H} = State) -> + handlers := H} = State, Timeout) -> receive {timeout, _TRef, {stats, Interval, ProcStr, Pid}} -> case server_handle_stats(ProcStr, Pid) of @@ -247,7 +266,7 @@ server_handle_message(#{mod := Mod, ok end, State; - + {?MODULE, Ref, Parent, stop} -> reply(Parent, Ref, ok), lists:foreach(fun(P) -> handler_stop(P) end, H), @@ -257,7 +276,7 @@ server_handle_message(#{mod := Mod, {'DOWN', _MRef, process, Pid, Reason} -> server_handle_down(Pid, Reason, State) - after 0 -> + after Timeout -> State end. @@ -342,15 +361,15 @@ handler_init(Parent) -> ?I("received continue"), reply(Parent, Ref, ok), handler_initial_activation(Mod, Sock, Active), - handler_loop(#{parent => Parent, - mod => Mod, - sock => Sock, - active => Active, - start => ?T(), - mcnt => 0, - bcnt => 0, - last_reply => none, - acc => <<>>}) + handler_loop(#{parent => Parent, + mod => Mod, + sock => Sock, + active => Active, + start => ?T(), + mcnt => 0, + bcnt => 0, + last_reply => none, + acc => <<>>}) after 5000 -> ?I("timeout when message queue: " diff --git a/erts/emulator/test/socket_test_ttest_tcp_server_socket.erl b/erts/emulator/test/socket_test_ttest_tcp_server_socket.erl index d1de230637..4045bf4e4e 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_server_socket.erl +++ b/erts/emulator/test/socket_test_ttest_tcp_server_socket.erl @@ -21,18 +21,26 @@ -module(socket_test_ttest_tcp_server_socket). -export([ - start/3, + start/4, stop/1 ]). -define(TRANSPORT_MOD, socket_test_ttest_tcp_socket). -%% -define(MOD(D, M), {?TRANSPORT_MOD, #{domain => D, +%% -define(MOD(M), {?TRANSPORT_MOD, #{async => false, %% method => M, %% stats_interval => 10000}}). --define(MOD(D, M), {?TRANSPORT_MOD, #{domain => D, method => M}}). +-define(MOD(D,M,A), {?TRANSPORT_MOD, #{domain => D, + async => A, + method => M}}). -start(Method, Domain, Active) -> - socket_test_ttest_tcp_server:start(?MOD(Domain, Method), Active). +start(Method, Domain, Async, Active) -> + socket_test_ttest_tcp_server:start(?MOD(Domain, Method, Async), Active). + %% {ok, {Pid, AddrPort}} -> + %% MRef = erlang:monitor(process, Pid), + %% {ok, {Pid, MRef, AddrPort}}; + %% {error, _} = ERROR -> + %% ERROR + %% end. stop(Pid) -> socket_test_ttest_tcp_server:stop(Pid). diff --git a/erts/emulator/test/socket_test_ttest_tcp_socket.erl b/erts/emulator/test/socket_test_ttest_tcp_socket.erl index cf68bfe591..3aa3b2c504 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_socket.erl +++ b/erts/emulator/test/socket_test_ttest_tcp_socket.erl @@ -57,7 +57,7 @@ %% ========================================================================== -%% This does not really work. Its just a placeholder for the time beeing... +%% This does not really work. Its just a placeholder for the time being... %% getopt(Sock, Opt) when is_atom(Opt) -> %% socket:getopt(Sock, socket, Opt). @@ -68,22 +68,32 @@ %% ========================================================================== -accept(#{sock := LSock, opts := #{method := Method} = Opts}) -> +%% The way we use server async its no point in doing a async accept call +%% (we do never actually run the test with more than one client). +accept(#{sock := LSock, opts := #{async := Async, + method := Method} = Opts}) -> case socket:accept(LSock) of - {ok, Sock} -> + {ok, Sock} -> Self = self(), - Reader = spawn(fun() -> reader_init(Self, Sock, false, Method) end), + Reader = spawn(fun() -> + reader_init(Self, Sock, Async, false, Method) + end), maybe_start_stats_timer(Opts, Reader), {ok, #{sock => Sock, reader => Reader, method => Method}}; {error, _} = ERROR -> ERROR end. -accept(#{sock := LSock, opts := #{method := Method} = Opts}, Timeout) -> +%% If a timeout has been explictly specified, then we do not use +%% async here. We will pass it on to the reader process. +accept(#{sock := LSock, opts := #{async := Async, + method := Method} = Opts}, Timeout) -> case socket:accept(LSock, Timeout) of {ok, Sock} -> Self = self(), - Reader = spawn(fun() -> reader_init(Self, Sock, false, Method) end), + Reader = spawn(fun() -> + reader_init(Self, Sock, Async, false, Method) + end), maybe_start_stats_timer(Opts, Reader), {ok, #{sock => Sock, reader => Reader, method => Method}}; {error, _} = ERROR -> @@ -153,7 +163,8 @@ connect(Addr, Port, #{domain := Domain} = Opts) -> do_connect(LocalSA, ServerSA, Cleanup, Opts#{proto => tcp}). do_connect(LocalSA, ServerSA, Cleanup, #{domain := Domain, - proto := Proto, + proto := Proto, + async := Async, method := Method} = Opts) -> try begin @@ -181,7 +192,9 @@ do_connect(LocalSA, ServerSA, Cleanup, #{domain := Domain, throw({error, {connect, CReason}}) end, Self = self(), - Reader = spawn(fun() -> reader_init(Self, Sock, false, Method) end), + Reader = spawn(fun() -> + reader_init(Self, Sock, Async, false, Method) + end), maybe_start_stats_timer(Opts, Reader), {ok, #{sock => Sock, reader => Reader, method => Method}} end @@ -219,9 +232,9 @@ listen() -> listen(0). listen(Port) when is_integer(Port) -> - listen(Port, #{domain => inet, method => plain}); + listen(Port, #{domain => inet, async => false, method => plain}); listen(Path) when is_list(Path) -> - listen(Path, #{domain => local, method => plain}). + listen(Path, #{domain => local, async => false, method => plain}). listen(0, #{domain := local} = Opts) -> listen(mk_unique_path(), Opts); @@ -241,8 +254,10 @@ listen(Port, #{domain := Domain} = Opts) do_listen(SA, Cleanup, - #{domain := Domain, proto := Proto, method := Method} = Opts) - when (Method =:= plain) orelse (Method =:= msg) -> + #{domain := Domain, proto := Proto, + async := Async, method := Method} = Opts) + when (Method =:= plain) orelse (Method =:= msg) andalso + is_boolean(Async) -> try begin Sock = case socket:open(Domain, stream, Proto) of @@ -339,13 +354,18 @@ sockname(#{sock := Sock}) -> %% ========================================================================== -reader_init(ControllingProcess, Sock, Active, Method) +reader_init(ControllingProcess, Sock, Async, Active, Method) when is_pid(ControllingProcess) andalso + is_boolean(Async) andalso (is_boolean(Active) orelse (Active =:= once)) andalso ((Method =:= plain) orelse (Method =:= msg)) -> + put(verbose, false), MRef = erlang:monitor(process, ControllingProcess), reader_loop(#{ctrl_proc => ControllingProcess, ctrl_proc_mref => MRef, + async => Async, + select_info => undefined, + select_num => 0, % Count the number of select messages active => Active, sock => Sock, method => Method}). @@ -356,11 +376,11 @@ reader_loop(#{active := false, ctrl_proc := Pid} = State) -> receive {?MODULE, stop} -> - exit(normal); + reader_exit(State, stop); {?MODULE, Pid, controlling_process, NewPid} -> - MRef = maps:get(ctrl_proc_mref, State), - erlang:demonitor(MRef, [flush]), + OldMRef = maps:get(ctrl_proc_mref, State), + erlang:demonitor(OldMRef, [flush]), NewMRef = erlang:monitor(process, NewPid), Pid ! {?MODULE, self(), controlling_process}, reader_loop(State#{ctrl_proc => NewPid, @@ -371,10 +391,8 @@ reader_loop(#{active := false, {'DOWN', MRef, process, Pid, Reason} -> case maps:get(ctrl_proc_mref, State) of - MRef when (Reason =:= normal) -> - exit(normal); MRef -> - exit({controlling_process, Reason}); + reader_exit(State, {ctrl_exit, Reason}); _ -> reader_loop(State) end @@ -382,7 +400,8 @@ reader_loop(#{active := false, %% Read *once* and then change to false reader_loop(#{active := once, - sock := Sock, + async := false, + sock := Sock, method := Method, ctrl_proc := Pid} = State) -> case do_recv(Method, Sock) of @@ -392,25 +411,23 @@ reader_loop(#{active := once, {error, timeout} -> receive {?MODULE, stop} -> - exit(normal); + reader_exit(State, stop); {?MODULE, Pid, controlling_process, NewPid} -> - MRef = maps:get(ctrl_proc_mref, State), - erlang:demonitor(MRef, [flush]), - MRef = erlang:monitor(process, NewPid), + OldMRef = maps:get(ctrl_proc_mref, State), + erlang:demonitor(OldMRef, [flush]), + NewMRef = erlang:monitor(process, NewPid), Pid ! {?MODULE, self(), controlling_process}, reader_loop(State#{ctrl_proc => NewPid, - ctrl_proc_mref => MRef}); + ctrl_proc_mref => NewMRef}); {?MODULE, active, NewActive} -> reader_loop(State#{active => NewActive}); {'DOWN', MRef, process, Pid, Reason} -> case maps:get(ctrl_proc_mref, State) of - MRef when (Reason =:= normal) -> - exit(normal); - MRef -> - exit({controlling_process, Reason}); + MRef -> + reader_exit(State, {ctrl_exit, Reason}); _ -> reader_loop(State) end @@ -418,17 +435,86 @@ reader_loop(#{active := once, reader_loop(State) end; - {error, closed} -> + {error, closed} = E1 -> + Pid ! ?CLOSED_MSG(Sock, Method), + reader_exit(State, E1); + + {error, Reason} = E2 -> + Pid ! ?ERROR_MSG(Sock, Method, Reason), + reader_exit(State, E2) + end; +reader_loop(#{active := once, + async := true, + select_info := undefined, + sock := Sock, + method := Method, + ctrl_proc := Pid} = State) -> + case do_recv(Method, Sock, nowait) of + {select, SelectInfo} -> + reader_loop(State#{select_info => SelectInfo}); + {ok, Data} -> + Pid ! ?DATA_MSG(Sock, Method, Data), + reader_loop(State#{active => false}); + + {error, closed} = E1 -> Pid ! ?CLOSED_MSG(Sock, Method), - exit(normal); + reader_exit(State, E1); - {error, Reason} -> + {error, Reason} = E2 -> Pid ! ?ERROR_MSG(Sock, Method, Reason), - exit(Reason) + reader_exit(State, E2) + end; +reader_loop(#{active := once, + async := true, + select_info := {select_info, _, Ref}, + select_num := N, + sock := Sock, + method := Method, + ctrl_proc := Pid} = State) -> + receive + {?MODULE, stop} -> + reader_exit(State, stop); + + {?MODULE, Pid, controlling_process, NewPid} -> + OldMRef = maps:get(ctrl_proc_mref, State), + erlang:demonitor(OldMRef, [flush]), + NewMRef = erlang:monitor(process, NewPid), + Pid ! {?MODULE, self(), controlling_process}, + reader_loop(State#{ctrl_proc => NewPid, + ctrl_proc_mref => NewMRef}); + + {?MODULE, active, NewActive} -> + reader_loop(State#{active => NewActive}); + + {'DOWN', MRef, process, Pid, Reason} -> + case maps:get(ctrl_proc_mref, State) of + MRef -> + reader_exit(State, {ctrl_exit, Reason}); + _ -> + reader_loop(State) + end; + + {'$socket', Sock, select, Ref} -> + case do_recv(Method, Sock, nowait) of + {ok, Data} when is_binary(Data) -> + Pid ! ?DATA_MSG(Sock, Method, Data), + reader_loop(State#{active => false, + select_info => undefined, + select_num => N+1}); + + {error, closed} = E1 -> + Pid ! ?CLOSED_MSG(Sock, Method), + reader_exit(State, E1); + + {error, Reason} = E2 -> + Pid ! ?ERROR_MSG(Sock, Method, Reason), + reader_exit(State, E2) + end end; %% Read and forward data continuously reader_loop(#{active := true, + async := false, sock := Sock, method := Method, ctrl_proc := Pid} = State) -> @@ -439,25 +525,23 @@ reader_loop(#{active := true, {error, timeout} -> receive {?MODULE, stop} -> - exit(normal); + reader_exit(State, stop); {?MODULE, Pid, controlling_process, NewPid} -> - MRef = maps:get(ctrl_proc_mref, State), - erlang:demonitor(MRef, [flush]), - MRef = erlang:monitor(process, NewPid), + OldMRef = maps:get(ctrl_proc_mref, State), + erlang:demonitor(OldMRef, [flush]), + NewMRef = erlang:monitor(process, NewPid), Pid ! {?MODULE, self(), controlling_process}, reader_loop(State#{ctrl_proc => NewPid, - ctrl_proc_mref => MRef}); + ctrl_proc_mref => NewMRef}); {?MODULE, active, NewActive} -> reader_loop(State#{active => NewActive}); {'DOWN', MRef, process, Pid, Reason} -> case maps:get(ctrl_proc_mref, State) of - MRef when (Reason =:= normal) -> - exit(normal); MRef -> - exit({controlling_process, Reason}); + reader_exit(State, {ctrl_exit, Reason}); _ -> reader_loop(State) end @@ -465,27 +549,170 @@ reader_loop(#{active := true, reader_loop(State) end; - {error, closed} -> + {error, closed} = E1 -> Pid ! ?CLOSED_MSG(Sock, Method), - exit(normal); + reader_exit(State, E1); - {error, Reason} -> + {error, Reason} = E2 -> Pid ! ?ERROR_MSG(Sock, Method, Reason), - exit(Reason) + reader_exit(State, E2) + end; +reader_loop(#{active := true, + async := true, + select_info := undefined, + sock := Sock, + method := Method, + ctrl_proc := Pid} = State) -> + case do_recv(Method, Sock) of + {select, SelectInfo} -> + reader_loop(State#{select_info => SelectInfo}); + {ok, Data} -> + Pid ! ?DATA_MSG(Sock, Method, Data), + reader_loop(State); + + {error, closed} = E1 -> + Pid ! ?CLOSED_MSG(Sock, Method), + reader_exit(State, E1); + + {error, Reason} = E2 -> + Pid ! ?ERROR_MSG(Sock, Method, Reason), + reader_exit(State, E2) + end; +reader_loop(#{active := true, + async := true, + select_info := {select_info, _, Ref}, + select_num := N, + sock := Sock, + method := Method, + ctrl_proc := Pid} = State) -> + receive + {?MODULE, stop} -> + reader_exit(State, stop); + + {?MODULE, Pid, controlling_process, NewPid} -> + OldMRef = maps:get(ctrl_proc_mref, State), + erlang:demonitor(OldMRef, [flush]), + NewMRef = erlang:monitor(process, NewPid), + Pid ! {?MODULE, self(), controlling_process}, + reader_loop(State#{ctrl_proc => NewPid, + ctrl_proc_mref => NewMRef}); + + {?MODULE, active, NewActive} -> + reader_loop(State#{active => NewActive}); + + {'DOWN', MRef, process, Pid, Reason} -> + case maps:get(ctrl_proc_mref, State) of + MRef -> + reader_exit(State, {ctrl_exit, Reason}); + _ -> + reader_loop(State) + end; + + {'$socket', Sock, select, Ref} -> + case do_recv(Method, Sock, nowait) of + {ok, Data} when is_binary(Data) -> + Pid ! ?DATA_MSG(Sock, Method, Data), + reader_loop(State#{select_info => undefined, + select_num => N+1}); + + {error, closed} = E1 -> + Pid ! ?CLOSED_MSG(Sock, Method), + reader_exit(State, E1); + + {error, Reason} = E2 -> + Pid ! ?ERROR_MSG(Sock, Method, Reason), + reader_exit(State, E2) + end end. -do_recv(plain, Sock) -> - socket:recv(Sock, 0, ?READER_RECV_TIMEOUT); -do_recv(msg, Sock) -> - case socket:recvmsg(Sock, 0, 0, [], ?READER_RECV_TIMEOUT) of +do_recv(Method, Sock) -> + do_recv(Method, Sock, ?READER_RECV_TIMEOUT). + +do_recv(plain, Sock, Timeout) -> + socket:recv(Sock, 0, Timeout); +do_recv(msg, Sock, Timeout) -> + case socket:recvmsg(Sock, 0, 0, [], Timeout) of {ok, #{iov := [Bin]}} -> {ok, Bin}; + {select, _} = SELECT -> + SELECT; {error, _} = ERROR -> ERROR end. - - + + +reader_exit(#{async := false, active := Active}, stop) -> + vp("reader stopped when active: ~w", [Active]), + exit(normal); +reader_exit(#{async := true, + active := Active, + select_info := SelectInfo, + select_num := N}, stop) -> + vp("reader stopped when active: ~w" + "~n Current select info: ~p" + "~n Number of select messages: ~p", [Active, SelectInfo, N]), + exit(normal); +reader_exit(#{async := false, active := Active}, {ctrl_exit, normal}) -> + vp("reader ctrl exit when active: ~w", [Active]), + exit(normal); +reader_exit(#{async := true, + active := Active, + select_info := SelectInfo, + select_num := N}, {ctrl_exit, normal}) -> + vp("reader ctrl exit when active: ~w" + "~n Current select info: ~p" + "~n Number of select messages: ~p", [Active, SelectInfo, N]), + exit(normal); +reader_exit(#{async := false, active := Active}, {ctrl_exit, Reason}) -> + vp("reader exit when ctrl crash when active: ~w", [Active]), + exit({controlling_process, Reason}); +reader_exit(#{async := true, + active := Active, + select_info := SelectInfo, + select_num := N}, {ctrl_exit, Reason}) -> + vp("reader exit when ctrl crash when active: ~w" + "~n Current select info: ~p" + "~n Number of select messages: ~p", [Active, SelectInfo, N]), + exit({controlling_process, Reason}); +reader_exit(#{async := false, active := Active}, {error, closed}) -> + vp("reader exit when socket closed when active: ~w", [Active]), + exit(normal); +reader_exit(#{async := true, + active := Active, + select_info := SelectInfo, + select_num := N}, {error, closed}) -> + vp("reader exit when socket closed when active: ~w " + "~n Current select info: ~p" + "~n Number of select messages: ~p", [Active, SelectInfo, N]), + exit(normal); +reader_exit(#{async := false, active := Active}, {error, Reason}) -> + vp("reader exit when socket error when active: ~w", [Active]), + exit(Reason); +reader_exit(#{async := true, + active := Active, + select_info := SelectInfo, + select_num := N}, {error, Reason}) -> + vp("reader exit when socket error when active: ~w: " + "~n Current select info: ~p" + "~n Number of select messages: ~p", [Active, SelectInfo, N]), + exit(Reason). + + + + + %% ========================================================================== +vp(F, A) -> + vp(get(verbose), F, A). + +vp(true, F, A) -> + p(F, A); +vp(_, _, _) -> + ok. + +p(F, A) -> + io:format(F ++ "~n", A). + diff --git a/erts/etc/unix/gcov-gen-html b/erts/etc/unix/gcov-gen-html new file mode 100755 index 0000000000..3fd9f1ca49 --- /dev/null +++ b/erts/etc/unix/gcov-gen-html @@ -0,0 +1,62 @@ +#!/usr/bin/perl + +use strict; +use warnings; + +@ARGV == 2 or die "Usage: gcov-gen-html \$ERL_TOP <output directory>\n"; + +my $srcdir = shift @ARGV; +my $outdir = shift @ARGV; + +my $verbose = 1; +my $flavor = "smp"; + +# setup filenames and paths, observe geninfos --base-directory +# it needs a correct path just after the $geninfo + +my $lcov_path = ""; #/usr/local/bin/"; + +my $geninfo = $lcov_path . "geninfo --no-checksum --base-directory"; +my $genhtml = $lcov_path . "genhtml"; + +# src paths + +my $emu_src_path = "$srcdir/erts/emulator"; +my $elib_src_path = "$srcdir/erts/lib_src"; +my $pcre_src_path = "$emu_src_path"; +my $zlib_src_path = "$emu_src_path"; + +# obj paths + +my $emu_obj_path = <$emu_src_path/obj/*-linux-*/gcov/$flavor>; +my $elib_obj_path = <$elib_src_path/obj/*-linux-*/gcov>; +my $pcre_obj_path = <$emu_src_path/pcre/obj/*-linux-*/gcov>; +my $zlib_obj_path = <$emu_src_path/zlib/obj/*-linux-*/gcov>; + +# info files + +my $emu_info = "$srcdir/emulator-cover.info"; +my $elib_info = "$srcdir/elib-cover.info"; +my $pcre_info = "$srcdir/pcre-cover.info"; + +my $infofiles = "$emu_info $elib_info $pcre_info"; + +run("$geninfo $emu_src_path -o $emu_info $emu_obj_path"); +run("$geninfo $elib_src_path -o $elib_info $elib_obj_path"); +run("$geninfo $pcre_src_path -o $pcre_info $pcre_obj_path"); + +if (<$zlib_obj_path/*.o>) { + my $zlib_info = "$srcdir/zlib-cover.info"; + $infofiles .= " $zlib_info"; + run("$geninfo $zlib_src_path -o $zlib_info $zlib_obj_path"); +} + +run("$genhtml -o $outdir $infofiles"); + + + +sub run { + my $cmd = shift; + print STDERR "\nrun($cmd)\n" if $verbose > 0; + system("$cmd 2>&1") == 0 or die "\nCan't run \"$cmd\": $?"; +} diff --git a/erts/etc/win32/msys_tools/vc/cc.sh b/erts/etc/win32/msys_tools/vc/cc.sh index 2b0482e876..71af14b9b2 100644 --- a/erts/etc/win32/msys_tools/vc/cc.sh +++ b/erts/etc/win32/msys_tools/vc/cc.sh @@ -195,7 +195,6 @@ mkdir $TMPOBJDIR # Compile for x in $SOURCES; do - start_time=`date '+%s'` # Compile each source if [ $LINKING = false ]; then # We should have an output defined, which is a directory @@ -260,16 +259,28 @@ for x in $SOURCES; do else tail -n +2 $ERR_FILE >&2 if test $DEPENDENCIES = true; then - if test `grep -v $x $MSG_FILE | grep -c '#line'` != "0"; then - o=`echo $x | sed 's,.*/,,' | sed 's,\.cp*$,.o,'` - echo -n $o':' -# cat $MSG_FILE | grep '#line' | grep -v $x | awk -F\" '{printf("%s\n",$2)}' | sort -u | grep -v " " | xargs -n 1 win2msys_path.sh | awk '{printf("\\\n %s ",$0)}' - cat $MSG_FILE | grep '#line' | grep -v $x | awk -F\" '{printf("%s\n",$2)}' | sort -u | grep -v " " | sed 's,^\([A-Za-z]\):[\\/]*,/\1/,;s,\\\\*,/,g'| awk '{printf("\\\n %s ",$0)}' - echo - echo - after_sed=`date '+%s'` - echo Made dependencies for $x':' `expr $after_sed '-' $start_time` 's' >&2 - fi + perl -e ' +my $file = "'$x'"; +while (<>) { + next unless /^#line/; + next if /$file/o; + (undef,$_) = split(/\"/); + next if / /; + $all{$_} = 1; +} +foreach (sort keys %all) { + s@^([A-Za-z]):@/$1@; + s@\\\\@/@g; + push @f, "\\\n $_ "; +} +if (@f) { + my $oname = $file; + $oname =~ s@.*/@@; + $oname =~ s@[.]cp*@.o@; + print $oname, ":", @f; + print "\n\n"; + print STDERR "Made dependencies for $file\n"; +}' $MSG_FILE else cat $MSG_FILE fi diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex b7c4cbe0a8..f67b660a08 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/socket.beam b/erts/preloaded/ebin/socket.beam Binary files differindex e37aa81b7c..134b4eac13 100644 --- a/erts/preloaded/ebin/socket.beam +++ b/erts/preloaded/ebin/socket.beam diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index ac73946dc0..06f0ee1dc6 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -3361,7 +3361,7 @@ dist_ctrl_get_opt(_DHandle, _Opt) -> DHandle :: dist_handle(), InputPackets :: non_neg_integer(), OutputPackets :: non_neg_integer(), - PendingOutputPackets :: boolean(), + PendingOutputPackets :: non_neg_integer(), Res :: {'ok', InputPackets, OutputPackets, PendingOutputPackets}. dist_get_stat(_DHandle) -> diff --git a/erts/preloaded/src/prim_inet.erl b/erts/preloaded/src/prim_inet.erl index d5abdd2483..374facb2a3 100644 --- a/erts/preloaded/src/prim_inet.erl +++ b/erts/preloaded/src/prim_inet.erl @@ -553,34 +553,49 @@ send(S, Data) -> %% "sendto" is for UDP. IP and Port are set by the caller to 0 if the socket %% is known to be connected. -sendto(S, Addr, _, Data) when is_port(S), tuple_size(Addr) =:= 2 -> - case type_value(set, addr, Addr) of - true -> - ?DBG_FORMAT("prim_inet:sendto(~p, ~p, ~p)~n", [S,Addr,Data]), - try - erlang:port_command(S, [enc_value(set, addr, Addr),Data]) - of - true -> - receive - {inet_reply,S,Reply} -> - ?DBG_FORMAT( - "prim_inet:sendto() -> ~p~n", [Reply]), - Reply - end - catch - error:_ -> - ?DBG_FORMAT( - "prim_inet:sendto() -> {error,einval}~n", []), - {error,einval} - end; - false -> - ?DBG_FORMAT( - "prim_inet:sendto() -> {error,einval}~n", []), - {error,einval} - end; -sendto(S, IP, Port, Data) -> - sendto(S, {IP, Port}, 0, Data). - +sendto(S, {_, _} = Address, AncOpts, Data) + when is_port(S), is_list(AncOpts) -> + case encode_opt_val(AncOpts) of + {ok, AncData} -> + AncDataLen = iolist_size(AncData), + case + type_value(set, addr, Address) andalso + type_value(set, uint32, AncDataLen) + of + true -> + ?DBG_FORMAT("prim_inet:sendto(~p, ~p, ~p, ~p)~n", + [S,Address,AncOpts,Data]), + PortCommandData = + [enc_value(set, addr, Address), + enc_value(set, uint32, AncDataLen), AncData, + Data], + try erlang:port_command(S, PortCommandData) of + true -> + receive + {inet_reply,S,Reply} -> + ?DBG_FORMAT( + "prim_inet:sendto() -> ~p~n", [Reply]), + Reply + end + catch + _:_ -> + ?DBG_FORMAT( + "prim_inet:sendto() -> {error,einval}~n", []), + {error,einval} + end; + false -> + ?DBG_FORMAT( + "prim_inet:sendto() -> {error,einval}~n", []), + {error,einval} + end; + {error,_} -> + ?DBG_FORMAT( + "prim_inet:sendto() -> {error,einval}~n", []), + {error,einval} + end; +sendto(S, IP, Port, Data) + when is_port(S), is_integer(Port) -> + sendto(S, {IP, Port}, [], Data). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% @@ -1993,15 +2008,15 @@ enc_value_2(addr, {File,_}) when is_list(File); is_binary(File) -> [?INET_AF_LOCAL,iolist_size(File)|File]; %% enc_value_2(addr, {inet,{any,Port}}) -> - [?INET_AF_INET,?int16(Port),0,0,0,0]; + [?INET_AF_INET,?int16(Port)|ip4_to_bytes({0,0,0,0})]; enc_value_2(addr, {inet,{loopback,Port}}) -> - [?INET_AF_INET,?int16(Port),127,0,0,1]; + [?INET_AF_INET,?int16(Port)|ip4_to_bytes({127,0,0,1})]; enc_value_2(addr, {inet,{IP,Port}}) -> [?INET_AF_INET,?int16(Port)|ip4_to_bytes(IP)]; enc_value_2(addr, {inet6,{any,Port}}) -> - [?INET_AF_INET6,?int16(Port),0,0,0,0,0,0,0,0]; + [?INET_AF_INET6,?int16(Port)|ip6_to_bytes({0,0,0,0,0,0,0,0})]; enc_value_2(addr, {inet6,{loopback,Port}}) -> - [?INET_AF_INET6,?int16(Port),0,0,0,0,0,0,0,1]; + [?INET_AF_INET6,?int16(Port)|ip6_to_bytes({0,0,0,0,0,0,0,1})]; enc_value_2(addr, {inet6,{IP,Port}}) -> [?INET_AF_INET6,?int16(Port)|ip6_to_bytes(IP)]; enc_value_2(addr, {local,Addr}) -> @@ -2149,10 +2164,10 @@ enum_name(_, []) -> false. %% encode opt/val REVERSED since options are stored in reverse order %% i.e. the recent options first (we must process old -> new) encode_opt_val(Opts) -> - try - enc_opt_val(Opts, []) + try + {ok, enc_opt_val(Opts, [])} catch - Reason -> {error,Reason} + throw:Reason -> {error,Reason} end. %% {active, once} and {active, N} are specially optimized because they will @@ -2171,17 +2186,21 @@ enc_opt_val([binary|Opts], Acc) -> enc_opt_val(Opts, Acc, mode, binary); enc_opt_val([list|Opts], Acc) -> enc_opt_val(Opts, Acc, mode, list); -enc_opt_val([_|_], _) -> {error,einval}; -enc_opt_val([], Acc) -> {ok,Acc}. +enc_opt_val([_|_], _) -> + throw(einval); +enc_opt_val([], Acc) -> + Acc. enc_opt_val(Opts, Acc, Opt, Val) when is_atom(Opt) -> Type = type_opt(set, Opt), case type_value(set, Type, Val) of true -> enc_opt_val(Opts, [enc_opt(Opt),enc_value(set, Type, Val)|Acc]); - false -> {error,einval} + false -> + throw(einval) end; -enc_opt_val(_, _, _, _) -> {error,einval}. +enc_opt_val(_, _, _, _) -> + throw(einval). diff --git a/erts/preloaded/src/socket.erl b/erts/preloaded/src/socket.erl index 0f0d8f7a02..ae1ffdb4ac 100644 --- a/erts/preloaded/src/socket.erl +++ b/erts/preloaded/src/socket.erl @@ -53,10 +53,16 @@ getopt/3, sockname/1, - peername/1 + peername/1, + + cancel/2 ]). -export_type([ + select_tag/0, + select_ref/0, + select_info/0, + domain/0, type/0, protocol/0, @@ -585,6 +591,17 @@ #{level := integer(), type := integer(), data := binary()}. +-opaque select_tag() :: atom(). +-opaque select_ref() :: reference(). + +-record(select_info, {tag :: select_tag(), ref :: select_ref()}). + +-type select_info() :: #select_info{}. + +-define(SELECT_INFO(T, R), #select_info{tag = T, ref = R}). +-define(SELECT(T, R), {select, ?SELECT_INFO(T, R)}). + + %% This is used in messages sent from the nif-code to erlang processes: %% %% {?SOCKET_TAG, Socket :: socket(), Tag :: atom(), Info :: term()} @@ -1189,18 +1206,24 @@ validate_inet6_addrs(Addrs) -> %% -spec connect(Socket, SockAddr) -> ok | {error, Reason} when - Socket :: socket(), - SockAddr :: sockaddr(), - Reason :: term(). + Socket :: socket(), + SockAddr :: sockaddr(), + Reason :: term(). connect(Socket, SockAddr) -> connect(Socket, SockAddr, infinity). --spec connect(Socket, SockAddr, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - SockAddr :: sockaddr(), - Timeout :: timeout(), - Reason :: term(). +-spec connect(Socket, SockAddr, nowait) -> + ok | {select, SelectInfo} | {error, Reason} when + Socket :: socket(), + SockAddr :: sockaddr(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, SockAddr, Timeout) -> ok | {error, Reason} when + Socket :: socket(), + SockAddr :: sockaddr(), + Timeout :: timeout(), + Reason :: term(). %% <KOLLA> %% Is it possible to connect with family = local for the (dest) sockaddr? @@ -1210,12 +1233,18 @@ connect(_Socket, _SockAddr, Timeout) {error, timeout}; connect(#socket{ref = SockRef}, #{family := Fam} = SockAddr, Timeout) when ((Fam =:= inet) orelse (Fam =:= inet6) orelse (Fam =:= local)) andalso - ((Timeout =:= infinity) orelse is_integer(Timeout)) -> + ((Timeout =:= nowait) orelse + (Timeout =:= infinity) orelse is_integer(Timeout)) -> TS = timestamp(Timeout), case nif_connect(SockRef, SockAddr) of ok -> %% Connected! ok; + + {ok, Ref} when (Timeout =:= nowait) -> + %% Connecting, but the caller does not want to wait... + ?SELECT(connect, Ref); + {ok, Ref} -> %% Connecting... NewTimeout = next_timeout(TS, Timeout), @@ -1268,17 +1297,27 @@ listen(#socket{ref = SockRef}, Backlog) accept(Socket) -> accept(Socket, ?SOCKET_ACCEPT_TIMEOUT_DEFAULT). --spec accept(LSocket, Timeout) -> {ok, Socket} | {error, Reason} when - LSocket :: socket(), - Timeout :: timeout(), - Socket :: socket(), - Reason :: term(). +-spec accept(LSocket, nowait) -> + {ok, Socket} | + {select, SelectInfo} | + {error, Reason} when + LSocket :: socket(), + Socket :: socket(), + SelectInfo :: select_info(), + Reason :: term() + ; (LSocket, Timeout) -> {ok, Socket} | {error, Reason} when + LSocket :: socket(), + Timeout :: timeout(), + Socket :: socket(), + Reason :: term(). %% Do we really need this optimization? accept(_, Timeout) when is_integer(Timeout) andalso (Timeout =< 0) -> {error, timeout}; accept(#socket{ref = LSockRef}, Timeout) - when is_integer(Timeout) orelse (Timeout =:= infinity) -> + when is_integer(Timeout) orelse + (Timeout =:= infinity) orelse + (Timeout =:= nowait) -> do_accept(LSockRef, Timeout). do_accept(LSockRef, Timeout) -> @@ -1289,6 +1328,11 @@ do_accept(LSockRef, Timeout) -> Socket = #socket{ref = SockRef}, {ok, Socket}; + + {error, eagain} when (Timeout =:= nowait) -> + ?SELECT(accept, AccRef); + + {error, eagain} -> %% Each call is non-blocking, but even then it takes %% *some* time, so just to be sure, recalculate before @@ -1327,33 +1371,56 @@ send(Socket, Data) -> send(Socket, Data, ?SOCKET_SEND_FLAGS_DEFAULT, ?SOCKET_SEND_TIMEOUT_DEFAULT). -spec send(Socket, Data, Flags) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Flags :: send_flags(), - Reason :: term() + Socket :: socket(), + Data :: iodata(), + Flags :: send_flags(), + Reason :: term() + ; (Socket, Data, Timeout :: nowait) -> ok | + {select, SelectInfo} | + {ok, {RestData, SelectInfo}} | + {error, Reason} when + Socket :: socket(), + Data :: iodata(), + RestData :: binary(), + SelectInfo :: select_info(), + Reason :: term() ; (Socket, Data, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Timeout :: timeout(), - Reason :: term(). + Socket :: socket(), + Data :: iodata(), + Timeout :: timeout(), + Reason :: term(). send(Socket, Data, Flags) when is_list(Flags) -> send(Socket, Data, Flags, ?SOCKET_SEND_TIMEOUT_DEFAULT); send(Socket, Data, Timeout) -> send(Socket, Data, ?SOCKET_SEND_FLAGS_DEFAULT, Timeout). --spec send(Socket, Data, Flags, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Flags :: send_flags(), - Timeout :: timeout(), - Reason :: term(). +-spec send(Socket, Data, Flags, nowait) -> ok | + {select, SelectInfo} | + {ok, {RestData, SelectInfo}} | + {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Flags :: send_flags(), + RestData :: binary(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, Data, Flags, Timeout) -> ok | {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Flags :: send_flags(), + Timeout :: timeout(), + Reason :: term(). send(Socket, Data, Flags, Timeout) when is_list(Data) -> Bin = erlang:list_to_binary(Data), send(Socket, Bin, Flags, Timeout); send(#socket{ref = SockRef}, Data, Flags, Timeout) - when is_binary(Data) andalso is_list(Flags) -> + when is_binary(Data) andalso + is_list(Flags) andalso + ((Timeout =:= nowait) orelse + (Timeout =:= infinity) orelse + (is_integer(Timeout) andalso (Timeout > 0))) -> EFlags = enc_send_flags(Flags), do_send(SockRef, Data, EFlags, Timeout). @@ -1363,6 +1430,15 @@ do_send(SockRef, Data, EFlags, Timeout) -> case nif_send(SockRef, SendRef, Data, EFlags) of ok -> ok; + + + {ok, Written} when (Timeout =:= nowait) -> + <<_:Written/binary, Rest/binary>> = Data, + %% We are partially done, but the user don't want to wait (here) + %% for completion + {ok, {Rest, ?SELECT_INFO(send, SendRef)}}; + + {ok, Written} -> NewTimeout = next_timeout(TS, Timeout), %% We are partially done, wait for continuation @@ -1384,6 +1460,12 @@ do_send(SockRef, Data, EFlags, Timeout) -> cancel(SockRef, send, SendRef), {error, {timeout, size(Data)}} end; + + + {error, eagain} when (Timeout =:= nowait) -> + ?SELECT(send, SendRef); + + {error, eagain} -> receive {?SOCKET_TAG, #socket{ref = SockRef}, select, SendRef} -> @@ -1419,17 +1501,25 @@ sendto(Socket, Data, Dest) -> sendto(Socket, Data, Dest, ?SOCKET_SENDTO_FLAGS_DEFAULT). -spec sendto(Socket, Data, Dest, Flags) -> ok | {error, Reason} when - Socket :: socket(), - Data :: binary(), - Dest :: null | sockaddr(), - Flags :: send_flags(), - Reason :: term() + Socket :: socket(), + Data :: binary(), + Dest :: null | sockaddr(), + Flags :: send_flags(), + Reason :: term() + ; (Socket, Data, Dest, Timeout :: nowait) -> ok | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Dest :: null | sockaddr(), + SelectInfo :: select_info(), + Reason :: term() ; (Socket, Data, Dest, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Dest :: null | sockaddr(), - Timeout :: timeout(), - Reason :: term(). + Socket :: socket(), + Data :: iodata(), + Dest :: null | sockaddr(), + Timeout :: timeout(), + Reason :: term(). sendto(Socket, Data, Dest, Flags) when is_list(Flags) -> sendto(Socket, Data, Dest, Flags, ?SOCKET_SENDTO_TIMEOUT_DEFAULT); @@ -1437,13 +1527,22 @@ sendto(Socket, Data, Dest, Timeout) -> sendto(Socket, Data, Dest, ?SOCKET_SENDTO_FLAGS_DEFAULT, Timeout). --spec sendto(Socket, Data, Dest, Flags, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - Data :: binary(), - Dest :: null | sockaddr(), - Flags :: send_flags(), - Timeout :: timeout(), - Reason :: term(). +-spec sendto(Socket, Data, Dest, Flags, nowait) -> ok | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Data :: binary(), + Dest :: null | sockaddr(), + Flags :: send_flags(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, Data, Dest, Flags, Timeout) -> ok | {error, Reason} when + Socket :: socket(), + Data :: binary(), + Dest :: null | sockaddr(), + Flags :: send_flags(), + Timeout :: timeout(), + Reason :: term(). sendto(Socket, Data, Dest, Flags, Timeout) when is_list(Data) -> Bin = erlang:list_to_binary(Data), @@ -1452,14 +1551,18 @@ sendto(#socket{ref = SockRef}, Data, Dest, Flags, Timeout) when is_binary(Data) andalso (Dest =:= null) andalso is_list(Flags) andalso - (is_integer(Timeout) orelse (Timeout =:= infinity)) -> + ((Timeout =:= nowait) orelse + (Timeout =:= infinity) orelse + (is_integer(Timeout) andalso (Timeout > 0))) -> EFlags = enc_send_flags(Flags), do_sendto(SockRef, Data, Dest, EFlags, Timeout); sendto(#socket{ref = SockRef}, Data, #{family := Fam} = Dest, Flags, Timeout) when is_binary(Data) andalso ((Fam =:= inet) orelse (Fam =:= inet6) orelse (Fam =:= local)) andalso is_list(Flags) andalso - (is_integer(Timeout) orelse (Timeout =:= infinity)) -> + ((Timeout =:= nowait) orelse + (Timeout =:= infinity) orelse + (is_integer(Timeout) andalso (Timeout > 0))) -> EFlags = enc_send_flags(Flags), do_sendto(SockRef, Data, Dest, EFlags, Timeout). @@ -1471,6 +1574,11 @@ do_sendto(SockRef, Data, Dest, EFlags, Timeout) -> %% We are done ok; + {ok, Written} when (Timeout =:= nowait) -> + <<_:Written/binary, Rest/binary>> = Data, + {ok, {Rest, ?SELECT_INFO(sendto, SendRef)}}; + + {ok, Written} -> %% We are partially done, wait for continuation receive @@ -1492,6 +1600,11 @@ do_sendto(SockRef, Data, Dest, EFlags, Timeout) -> {error, timeout} end; + + {error, eagain} when (Timeout =:= nowait) -> + ?SELECT(sendto, SendRef); + + {error, eagain} -> receive {?SOCKET_TAG, #socket{ref = SockRef}, select, SendRef} -> @@ -1535,11 +1648,18 @@ sendmsg(Socket, MsgHdr) -> MsgHdr :: msghdr(), Flags :: send_flags(), Reason :: term() + ; (Socket, MsgHdr, Timeout :: nowait) -> ok | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + SelectInfo :: select_info(), + Reason :: term() ; (Socket, MsgHdr, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Timeout :: timeout(), - Reason :: term(). + Socket :: socket(), + MsgHdr :: msghdr(), + Timeout :: timeout(), + Reason :: term(). sendmsg(Socket, MsgHdr, Flags) when is_list(Flags) -> sendmsg(Socket, MsgHdr, Flags, ?SOCKET_SENDMSG_TIMEOUT_DEFAULT); @@ -1548,19 +1668,34 @@ sendmsg(Socket, MsgHdr, Timeout) sendmsg(Socket, MsgHdr, ?SOCKET_SENDMSG_FLAGS_DEFAULT, Timeout). --spec sendmsg(Socket, MsgHdr, Flags, Timeout) -> - ok | {ok, Remaining} | {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Flags :: send_flags(), - Timeout :: timeout(), - Remaining :: erlang:iovec(), - Reason :: term(). +-spec sendmsg(Socket, MsgHdr, Flags, nowait) -> + ok | + {ok, Remaining} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Flags :: send_flags(), + Remaining :: erlang:iovec(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, MsgHdr, Flags, Timeout) -> + ok | + {ok, Remaining} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Flags :: send_flags(), + Timeout :: timeout(), + Remaining :: erlang:iovec(), + Reason :: term(). sendmsg(#socket{ref = SockRef}, #{iov := IOV} = MsgHdr, Flags, Timeout) when is_list(IOV) andalso is_list(Flags) andalso - (is_integer(Timeout) orelse (Timeout =:= infinity)) -> + ((Timeout =:= nowait) orelse + (Timeout =:= infinity) orelse + (is_integer(Timeout) andalso (Timeout > 0))) -> try ensure_msghdr(MsgHdr) of M -> EFlags = enc_send_flags(Flags), @@ -1580,6 +1715,7 @@ do_sendmsg(SockRef, MsgHdr, EFlags, Timeout) -> %% We are done ok; + {ok, Written} when is_integer(Written) andalso (Written > 0) -> %% We should not retry here since the protocol may not %% be able to handle a message being split. Leave it to @@ -1591,6 +1727,11 @@ do_sendmsg(SockRef, MsgHdr, EFlags, Timeout) -> cancel(SockRef, sendmsg, SendRef), {ok, do_sendmsg_rest(maps:get(iov, MsgHdr), Written)}; + + {error, eagain} when (Timeout =:= nowait) -> + ?SELECT(sendmsg, SendRef); + + {error, eagain} -> receive {?SOCKET_TAG, #socket{ref = SockRef}, select, SendRef} -> @@ -1664,13 +1805,24 @@ recv(Socket, Length) -> ?SOCKET_RECV_FLAGS_DEFAULT, ?SOCKET_RECV_TIMEOUT_DEFAULT). --spec recv(Socket, Length, Flags) -> {ok, Data} | {error, Reason} when +-spec recv(Socket, Length, Flags) -> {ok, Data} | + {error, Reason} when Socket :: socket(), Length :: non_neg_integer(), Flags :: recv_flags(), Data :: binary(), Reason :: term() - ; (Socket, Length, Timeout) -> {ok, Data} | {error, Reason} when + ; (Socket, Length, Timeout :: nowait) -> {ok, Data} | + {select, SelectInfo} | + {ok, {Data, SelectInfo}} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Data :: binary(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, Length, Timeout) -> {ok, Data} | + {error, Reason} when Socket :: socket(), Length :: non_neg_integer(), Timeout :: timeout(), @@ -1682,18 +1834,31 @@ recv(Socket, Length, Flags) when is_list(Flags) -> recv(Socket, Length, Timeout) -> recv(Socket, Length, ?SOCKET_RECV_FLAGS_DEFAULT, Timeout). --spec recv(Socket, Length, Flags, Timeout) -> {ok, Data} | {error, Reason} when - Socket :: socket(), - Length :: non_neg_integer(), - Flags :: recv_flags(), - Timeout :: timeout(), - Data :: binary(), - Reason :: term(). +-spec recv(Socket, Length, Flags, nowait) -> {ok, Data} | + {select, SelectInfo} | + {ok, {Data, SelectInfo}} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Flags :: recv_flags(), + Data :: binary(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, Length, Flags, Timeout) -> {ok, Data} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Flags :: recv_flags(), + Timeout :: timeout(), + Data :: binary(), + Reason :: term(). recv(#socket{ref = SockRef}, Length, Flags, Timeout) when (is_integer(Length) andalso (Length >= 0)) andalso is_list(Flags) andalso - (is_integer(Timeout) orelse (Timeout =:= infinity)) -> + (is_integer(Timeout) orelse + (Timeout =:= infinity) orelse + (Timeout =:= nowait)) -> EFlags = enc_recv_flags(Flags), do_recv(SockRef, undefined, Length, EFlags, <<>>, Timeout). @@ -1701,8 +1866,12 @@ recv(#socket{ref = SockRef}, Length, Flags, Timeout) %% with Length = 0. This case makes it neccessary to have a timeout function %% clause since we may never wait for anything (no receive select), and so the %% the only timeout check will be the function clause. +%% Note that the Timeout value of 'nowait' has a special meaning. It means +%% that we will either return with data or with the with {error, NNNN}. In +%% wich case the caller will receive a select message at some later time. do_recv(SockRef, _OldRef, Length, EFlags, Acc, Timeout) - when (Timeout =:= infinity) orelse + when (Timeout =:= nowait) orelse + (Timeout =:= infinity) orelse (is_integer(Timeout) andalso (Timeout > 0)) -> TS = timestamp(Timeout), RecvRef = make_ref(), @@ -1723,6 +1892,15 @@ do_recv(SockRef, _OldRef, Length, EFlags, Acc, Timeout) <<Acc/binary, Bin/binary>>, next_timeout(TS, Timeout)); + + %% Did not get all the user asked for, but the user also + %% specified 'nowait', so deliver what we got and the + %% select info. + {ok, false = _Completed, Bin} when (Timeout =:= nowait) andalso + (size(Acc) =:= 0) -> + {ok, {Bin, ?SELECT_INFO(recv, RecvRef)}}; + + {ok, false = _Completed, Bin} when (size(Acc) =:= 0) -> %% We got the first chunk of it. %% We will be notified (select message) when there @@ -1761,6 +1939,17 @@ do_recv(SockRef, _OldRef, Length, EFlags, Acc, Timeout) {error, {timeout, Acc}} end; + + %% The user does not want to wait! + %% The user will be informed that there is something to read + %% via the select socket message (see below). + + {error, eagain} when (Timeout =:= nowait) andalso (size(Acc) =:= 0) -> + ?SELECT(recv, RecvRef); + {error, eagain} when (Timeout =:= nowait) -> + {ok, {Acc, ?SELECT_INFO(recv, RecvRef)}}; + + %% We return with the accumulated binary (if its non-empty) {error, eagain} when (Length =:= 0) andalso (size(Acc) > 0) -> %% CAN WE REALLY DO THIS? THE NIF HAS SELECTED!! OR? @@ -1851,30 +2040,51 @@ recvfrom(Socket, BufSz) -> ?SOCKET_RECV_FLAGS_DEFAULT, ?SOCKET_RECV_TIMEOUT_DEFAULT). --spec recvfrom(Socket, Flags, Timeout) -> - {ok, {Source, Data}} | {error, Reason} when - Socket :: socket(), - Flags :: recv_flags(), - Timeout :: timeout(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term() +-spec recvfrom(Socket, Flags, nowait) -> + {ok, {Source, Data}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + Source :: sockaddr() | undefined, + Data :: binary(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, Flags, Timeout) -> + {ok, {Source, Data}} | + {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + Timeout :: timeout(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: term() ; (Socket, BufSz, Flags) -> {ok, {Source, Data}} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Flags :: recv_flags(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term() + Socket :: socket(), + BufSz :: non_neg_integer(), + Flags :: recv_flags(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: term() + ; (Socket, BufSz, nowait) -> + {ok, {Source, Data}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Source :: sockaddr() | undefined, + Data :: binary(), + SelectInfo :: select_info(), + Reason :: term() ; (Socket, BufSz, Timeout) -> {ok, {Source, Data}} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Timeout :: timeout(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term(). + Socket :: socket(), + BufSz :: non_neg_integer(), + Timeout :: timeout(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: term(). recvfrom(Socket, Flags, Timeout) when is_list(Flags) -> recvfrom(Socket, 0, Flags, Timeout); @@ -1883,20 +2093,34 @@ recvfrom(Socket, BufSz, Flags) when is_list(Flags) -> recvfrom(Socket, BufSz, Timeout) -> recvfrom(Socket, BufSz, ?SOCKET_RECV_FLAGS_DEFAULT, Timeout). --spec recvfrom(Socket, BufSz, Flags, Timeout) -> - {ok, {Source, Data}} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Flags :: recv_flags(), - Timeout :: timeout(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term(). +-spec recvfrom(Socket, BufSz, Flags, nowait) -> + {ok, {Source, Data}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Flags :: recv_flags(), + Source :: sockaddr() | undefined, + Data :: binary(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, BufSz, Flags, Timeout) -> + {ok, {Source, Data}} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Flags :: recv_flags(), + Timeout :: timeout(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: term(). recvfrom(#socket{ref = SockRef}, BufSz, Flags, Timeout) when (is_integer(BufSz) andalso (BufSz >= 0)) andalso is_list(Flags) andalso - (is_integer(Timeout) orelse (Timeout =:= infinity)) -> + (is_integer(Timeout) orelse + (Timeout =:= infinity) orelse + (Timeout =:= nowait)) -> EFlags = enc_recv_flags(Flags), do_recvfrom(SockRef, BufSz, EFlags, Timeout). @@ -1907,6 +2131,11 @@ do_recvfrom(SockRef, BufSz, EFlags, Timeout) -> {ok, {_Source, _NewData}} = OK -> OK; + + {error, eagain} when (Timeout =:= nowait) -> + ?SELECT(recvfrom, RecvRef); + + {error, eagain} -> %% There is nothing just now, but we will be notified when there %% is something to read (a select message). @@ -1947,29 +2176,44 @@ recvmsg(Socket) -> Flags :: recv_flags(), MsgHdr :: msghdr(), Reason :: term() + ; (Socket, Timeout :: nowait) -> {ok, MsgHdr} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + SelectInfo :: select_info(), + Reason :: term() ; (Socket, Timeout) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - Timeout :: timeout(), - MsgHdr :: msghdr(), - Reason :: term(). + Socket :: socket(), + Timeout :: timeout(), + MsgHdr :: msghdr(), + Reason :: term(). recvmsg(Socket, Flags) when is_list(Flags) -> recvmsg(Socket, 0, 0, Flags, ?SOCKET_RECV_TIMEOUT_DEFAULT); recvmsg(Socket, Timeout) -> recvmsg(Socket, 0, 0, ?SOCKET_RECV_FLAGS_DEFAULT, Timeout). --spec recvmsg(Socket, Flags, Timeout) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - Flags :: recv_flags(), - Timeout :: timeout(), - MsgHdr :: msghdr(), - Reason :: term() +-spec recvmsg(Socket, Flags, nowait) -> {ok, MsgHdr} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + MsgHdr :: msghdr(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, Flags, Timeout) -> {ok, MsgHdr} | {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + Timeout :: timeout(), + MsgHdr :: msghdr(), + Reason :: term() ; (Socket, BufSz, CtrlSz) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - CtrlSz :: non_neg_integer(), - MsgHdr :: msghdr(), - Reason :: term(). + Socket :: socket(), + BufSz :: non_neg_integer(), + CtrlSz :: non_neg_integer(), + MsgHdr :: msghdr(), + Reason :: term(). recvmsg(Socket, Flags, Timeout) when is_list(Flags) -> recvmsg(Socket, 0, 0, Flags, Timeout); @@ -1980,20 +2224,34 @@ recvmsg(Socket, BufSz, CtrlSz) when is_integer(BufSz) andalso is_integer(CtrlSz) -spec recvmsg(Socket, BufSz, CtrlSz, - Flags, Timeout) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - CtrlSz :: non_neg_integer(), - Flags :: recv_flags(), - Timeout :: timeout(), - MsgHdr :: msghdr(), - Reason :: term(). + Flags, nowait) -> {ok, MsgHdr} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + CtrlSz :: non_neg_integer(), + Flags :: recv_flags(), + MsgHdr :: msghdr(), + SelectInfo :: select_info(), + Reason :: term() + ; (Socket, + BufSz, CtrlSz, + Flags, Timeout) -> {ok, MsgHdr} | {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + CtrlSz :: non_neg_integer(), + Flags :: recv_flags(), + Timeout :: timeout(), + MsgHdr :: msghdr(), + Reason :: term(). recvmsg(#socket{ref = SockRef}, BufSz, CtrlSz, Flags, Timeout) when (is_integer(BufSz) andalso (BufSz >= 0)) andalso (is_integer(CtrlSz) andalso (CtrlSz >= 0)) andalso is_list(Flags) andalso - (is_integer(Timeout) orelse (Timeout =:= infinity)) -> + (is_integer(Timeout) orelse + (Timeout =:= infinity) orelse + (Timeout =:= nowait)) -> EFlags = enc_recv_flags(Flags), do_recvmsg(SockRef, BufSz, CtrlSz, EFlags, Timeout). @@ -2004,6 +2262,11 @@ do_recvmsg(SockRef, BufSz, CtrlSz, EFlags, Timeout) -> {ok, _MsgHdr} = OK -> OK; + + {error, eagain} when (Timeout =:= nowait) -> + ?SELECT(recvmsg, RecvRef); + + {error, eagain} -> %% There is nothing just now, but we will be notified when there %% is something to read (a select message). @@ -2339,6 +2602,25 @@ peername(#socket{ref = SockRef}) -> nif_peername(SockRef). +%% =========================================================================== +%% +%% cancel - cancel an operation resulting in a select +%% +%% A call to accept, recv/recvfrom/recvmsg and send/sendto/sendmsg +%% can result in a select if they are called with the Timeout argument +%% set to nowait. This is indicated by the return of the select-info. +%% Such a operation can be cancelled by calling this function. +%% + +-spec cancel(Socket, SelectInfo) -> ok | {error, Reason} when + Socket :: socket(), + SelectInfo :: select_info(), + Reason :: term(). + +cancel(#socket{ref = SockRef}, #select_info{tag = Tag, ref = Ref}) -> + cancel(SockRef, Tag, Ref). + + %% =========================================================================== %% @@ -3462,15 +3744,18 @@ flush_select_msgs(SockRef, Ref) -> %% A timestamp in ms +timestamp(nowait = T) -> + T; timestamp(infinity) -> undefined; timestamp(_) -> timestamp(). timestamp() -> - {A,B,C} = os:timestamp(), - A*1000000000+B*1000+(C div 1000). + erlang:monotonic_time(milli_seconds). +next_timeout(_, nowait = Timeout) -> + Timeout; next_timeout(_, infinity = Timeout) -> Timeout; next_timeout(TS, Timeout) -> diff --git a/erts/vsn.mk b/erts/vsn.mk index 4d8282bf15..40a9685e9d 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 10.4.1 +VSN = 10.4.3 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index a64818da7b..c454608bbe 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -62,6 +62,37 @@ </section> +<section><title>Common_Test 1.17.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If a ct hook is installed in the <c>suite/0</c> function + in a test suite, then the hook's <c>terminate/1</c> + function would be called several times without it's + <c>init/2</c> function being called first. This is now + corrected.</p> + <p> + Own Id: OTP-15863 Aux Id: ERIERL-370 </p> + </item> + <item> + <p> + If <c>init_per_testcase</c> fails, the test itself is + skipped. According to the documentation, it should be + possible to change the result to failed in a hook + function. The only available hook function in this case + is <c>post_init_per_testcase</c>, but changing the return + value there did not affect the test case result. This is + now corrected.</p> + <p> + Own Id: OTP-15869 Aux Id: ERIERL-350 </p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.17.2</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -239,6 +270,37 @@ </section> +<section><title>Common_Test 1.15.4.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If a ct hook is installed in the <c>suite/0</c> function + in a test suite, then the hook's <c>terminate/1</c> + function would be called several times without it's + <c>init/2</c> function being called first. This is now + corrected.</p> + <p> + Own Id: OTP-15863 Aux Id: ERIERL-370 </p> + </item> + <item> + <p> + If <c>init_per_testcase</c> fails, the test itself is + skipped. According to the documentation, it should be + possible to change the result to failed in a hook + function. The only available hook function in this case + is <c>post_init_per_testcase</c>, but changing the return + value there did not affect the test case result. This is + now corrected.</p> + <p> + Own Id: OTP-15869 Aux Id: ERIERL-350 </p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.15.4.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/src/ct_hooks.erl b/lib/common_test/src/ct_hooks.erl index 97c349578f..94551d6815 100644 --- a/lib/common_test/src/ct_hooks.erl +++ b/lib/common_test/src/ct_hooks.erl @@ -363,7 +363,16 @@ terminate_if_scope_ends(HookId, Function0, Hooks) -> Function = strip_config(Function0), case lists:keyfind(HookId, #ct_hook_config.id, Hooks) of #ct_hook_config{ id = HookId, scope = Function} = Hook -> - terminate([Hook]), + case Function of + [AllOrGroup,_] when AllOrGroup=:=post_all; + AllOrGroup=:=post_groups -> + %% The scope only contains one function (post_all + %% or post_groups), and init has not been called, + %% so skip terminate as well. + ok; + _ -> + terminate([Hook]) + end, lists:keydelete(HookId, #ct_hook_config.id, Hooks); _ -> Hooks diff --git a/lib/common_test/src/test_server.erl b/lib/common_test/src/test_server.erl index 756cd4d692..588396f101 100644 --- a/lib/common_test/src/test_server.erl +++ b/lib/common_test/src/test_server.erl @@ -1364,23 +1364,29 @@ do_end_tc_call(Mod, IPTC={init_per_testcase,Func}, Res, Return) -> {NOk,_} when NOk == auto_skip; NOk == fail; NOk == skip ; NOk == skipped -> {_,Args} = Res, - IPTCEndRes = + {NewConfig,IPTCEndRes} = case do_end_tc_call1(Mod, IPTC, Res, Return) of IPTCEndConfig when is_list(IPTCEndConfig) -> - IPTCEndConfig; + {IPTCEndConfig,IPTCEndConfig}; + {failed,RetReason} when Return=:={fail,RetReason} -> + %% Fail reason not changed by framework or hook + {Args,Return}; + {SF,_} = IPTCEndResult when SF=:=skip; SF=:=skipped; + SF=:=fail; SF=:=failed -> + {Args,IPTCEndResult}; _ -> - Args + {Args,Return} end, EPTCInitRes = case do_init_tc_call(Mod,{end_per_testcase_not_run,Func}, - IPTCEndRes,Return) of + NewConfig,IPTCEndRes) of {ok,EPTCInitConfig} when is_list(EPTCInitConfig) -> - {Return,EPTCInitConfig}; + {IPTCEndRes,EPTCInitConfig}; _ -> - {Return,IPTCEndRes} + {IPTCEndRes,NewConfig} end, do_end_tc_call1(Mod, {end_per_testcase_not_run,Func}, - EPTCInitRes, Return); + EPTCInitRes, IPTCEndRes); _Ok -> do_end_tc_call1(Mod, IPTC, Res, Return) end; diff --git a/lib/common_test/test/ct_hooks_SUITE.erl b/lib/common_test/test/ct_hooks_SUITE.erl index 340b8f3d52..b87464f5e4 100644 --- a/lib/common_test/test/ct_hooks_SUITE.erl +++ b/lib/common_test/test/ct_hooks_SUITE.erl @@ -86,7 +86,7 @@ all(suite) -> scope_suite_state_cth, fail_pre_suite_cth, double_fail_pre_suite_cth, fail_post_suite_cth, skip_pre_suite_cth, skip_pre_end_cth, - skip_pre_init_tc_cth, + skip_pre_init_tc_cth, fail_post_init_tc_cth, skip_post_suite_cth, recover_post_suite_cth, update_config_cth, state_update_cth, update_result_cth, options_cth, same_id_cth, fail_n_skip_with_minimal_cth, prio_cth, no_config, @@ -206,6 +206,10 @@ skip_pre_init_tc_cth(Config) -> do_test(skip_pre_init_tc_cth, "ct_cth_empty_SUITE.erl", [skip_pre_init_tc_cth],Config). +fail_post_init_tc_cth(Config) -> + do_test(fail_post_init_tc_cth, "ct_fail_init_tc_SUITE.erl", + [fail_post_init_tc_cth],Config). + recover_post_suite_cth(Config) when is_list(Config) -> do_test(recover_post_suite_cth, "ct_cth_fail_per_suite_SUITE.erl", [recover_post_suite_cth],Config). @@ -671,9 +675,15 @@ test_events(scope_suite_cth) -> {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, %% check that post_groups and post_all comes before init when hook %% is installed in suite/0 + %% And there should be no terminate after these, since init is + %% not yet called. {?eh,cth,{'_',post_groups,['_',[]]}}, - {?eh,cth,{'_',post_all,['_','_',[]]}}, - {?eh,tc_start,{ct_scope_suite_cth_SUITE,init_per_suite}}, + {negative, + {?eh,cth,{'_',terminate,['_']}}, + {?eh,cth,{'_',post_all,['_','_',[]]}}}, + {negative, + {?eh,cth,{'_',terminate,['_']}}, + {?eh,tc_start,{ct_scope_suite_cth_SUITE,init_per_suite}}}, {?eh,cth,{'_',id,[[]]}}, {?eh,cth,{'_',init,['_',[]]}}, {?eh,cth,{'_',pre_init_per_suite,[ct_scope_suite_cth_SUITE,'$proplist',[]]}}, @@ -1036,6 +1046,44 @@ test_events(skip_pre_init_tc_cth) -> {?eh,stop_logging,[]} ]; +test_events(fail_post_init_tc_cth) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,cth,{empty_cth,init,['_',[]]}}, + {?eh,start_info,{1,1,1}}, + {?eh,tc_start,{ct_fail_init_tc_SUITE,init_per_suite}}, + {?eh,cth,{empty_cth,pre_init_per_suite,[ct_fail_init_tc_SUITE,'$proplist',[]]}}, + {?eh,cth,{empty_cth,post_init_per_suite, + [ct_fail_init_tc_SUITE,'$proplist','$proplist',[]]}}, + {?eh,tc_done,{ct_fail_init_tc_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{ct_fail_init_tc_SUITE,test_case}}, + {?eh,cth,{empty_cth,pre_init_per_testcase, + [ct_fail_init_tc_SUITE,test_case,'$proplist',[]]}}, + {?eh,cth,{empty_cth,post_init_per_testcase, + [ct_fail_init_tc_SUITE,test_case,'$proplist', + {skip, + {failed, + {ct_fail_init_tc_SUITE,init_per_testcase, + {{test_case_failed,"Failed in init_per_testcase"},'_'}}}}, + []]}}, + {?eh,tc_done,{ct_fail_init_tc_SUITE,test_case, + {failed,"Changed skip to fail in post_init_per_testcase"}}}, + {?eh,cth,{empty_cth,on_tc_fail, + [ct_fail_init_tc_SUITE,test_case, + "Changed skip to fail in post_init_per_testcase", + []]}}, + {?eh,test_stats,{0,1,{0,0}}}, + {?eh,tc_start,{ct_fail_init_tc_SUITE,end_per_suite}}, + {?eh,cth,{empty_cth,pre_end_per_suite,[ct_fail_init_tc_SUITE,'$proplist',[]]}}, + {?eh,cth,{empty_cth,post_end_per_suite, + [ct_fail_init_tc_SUITE,'$proplist',ok,[]]}}, + {?eh,tc_done,{ct_fail_init_tc_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,cth,{empty_cth,terminate,[[]]}}, + {?eh,stop_logging,[]} + ]; + test_events(recover_post_suite_cth) -> Suite = ct_cth_fail_per_suite_SUITE, [ diff --git a/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/ct_fail_init_tc_SUITE.erl b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/ct_fail_init_tc_SUITE.erl new file mode 100644 index 0000000000..96ddfc5782 --- /dev/null +++ b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/ct_fail_init_tc_SUITE.erl @@ -0,0 +1,49 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-module(ct_fail_init_tc_SUITE). + +-suite_defaults([{timetrap, {minutes, 10}}]). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include("ct.hrl"). + +%% Test server callback functions +init_per_suite(Config) -> + Config. + +end_per_suite(_Config) -> + ok. + +init_per_testcase(TestCase, _Config) -> + ct:fail("Failed in init_per_testcase"). + +end_per_testcase(_TestCase, _Config) -> + ok. + +all() -> + [test_case]. + +%% Test cases starts here. +test_case(Config) when is_list(Config) -> + ok. + diff --git a/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/fail_post_init_tc_cth.erl b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/fail_post_init_tc_cth.erl new file mode 100644 index 0000000000..ca9f05c40f --- /dev/null +++ b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/fail_post_init_tc_cth.erl @@ -0,0 +1,81 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + + +-module(fail_post_init_tc_cth). + + +-include_lib("common_test/src/ct_util.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + + +%% CT Hooks +-compile(export_all). + +init(Id, Opts) -> + empty_cth:init(Id, Opts). + +pre_init_per_suite(Suite, Config, State) -> + empty_cth:pre_init_per_suite(Suite,Config,State). + +post_init_per_suite(Suite,Config,Return,State) -> + empty_cth:post_init_per_suite(Suite,Config,Return,State). + +pre_end_per_suite(Suite,Config,State) -> + empty_cth:pre_end_per_suite(Suite,Config,State). + +post_end_per_suite(Suite,Config,Return,State) -> + empty_cth:post_end_per_suite(Suite,Config,Return,State). + +pre_init_per_group(Suite,Group,Config,State) -> + empty_cth:pre_init_per_group(Suite,Group,Config,State). + +post_init_per_group(Suite,Group,Config,Return,State) -> + empty_cth:post_init_per_group(Suite,Group,Config,Return,State). + +pre_end_per_group(Suite,Group,Config,State) -> + empty_cth:pre_end_per_group(Suite,Group,Config,State). + +post_end_per_group(Suite,Group,Config,Return,State) -> + empty_cth:post_end_per_group(Suite,Group,Config,Return,State). + +pre_init_per_testcase(Suite,TC,Config,State) -> + empty_cth:pre_init_per_testcase(Suite,TC,Config,State). + +post_init_per_testcase(Suite,TC,Config,{skip,_}=Return,State) -> + empty_cth:post_init_per_testcase(Suite,TC,Config,Return,State), + {{fail,"Changed skip to fail in post_init_per_testcase"},State}; +post_init_per_testcase(Suite,TC,Config,Return,State) -> + empty_cth:post_init_per_testcase(Suite,TC,Config,Return,State). + +pre_end_per_testcase(Suite,TC,Config,State) -> + empty_cth:pre_end_per_testcase(Suite,TC,Config,State). + +post_end_per_testcase(Suite,TC,Config,Return,State) -> + empty_cth:post_end_per_testcase(Suite,TC,Config,Return,State). + +on_tc_fail(Suite,TC, Reason, State) -> + empty_cth:on_tc_fail(Suite,TC,Reason,State). + +on_tc_skip(Suite,TC, Reason, State) -> + empty_cth:on_tc_skip(Suite,TC,Reason,State). + +terminate(State) -> + empty_cth:terminate(State). diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index 3090a99c35..f0d869381b 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -32,6 +32,22 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 7.4.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed an incorrect type determination for constructed + binaries, which could cause <c>is_binary</c> checks to + succeed when they shouldn't have.</p> + <p> + Own Id: OTP-15872</p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 7.4.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/src/beam_ssa_type.erl b/lib/compiler/src/beam_ssa_type.erl index 57fd7fec60..68920e7dd3 100644 --- a/lib/compiler/src/beam_ssa_type.erl +++ b/lib/compiler/src/beam_ssa_type.erl @@ -840,15 +840,8 @@ type({bif,Bif}, Args, Ts, _Ds) -> Type -> Type end; -type(bs_init, [#b_literal{val=Type}|Args], _Ts, _Ds) -> - case {Type,Args} of - {new,[_,#b_literal{val=Unit}]} -> - {binary,Unit}; - {append,[_,_,#b_literal{val=Unit}]} -> - {binary,Unit}; - {private_append,[_,_,#b_literal{val=Unit}]} -> - {binary,Unit} - end; +type(bs_init, _Args, _Ts, _Ds) -> + {binary, 1}; type(bs_extract, [Ctx], Ts, _Ds) -> #t_bs_match{type=Type} = get_type(Ctx, Ts), Type; diff --git a/lib/compiler/test/beam_type_SUITE.erl b/lib/compiler/test/beam_type_SUITE.erl index 2297c2e0f5..076a604aa4 100644 --- a/lib/compiler/test/beam_type_SUITE.erl +++ b/lib/compiler/test/beam_type_SUITE.erl @@ -24,7 +24,7 @@ integers/1,numbers/1,coverage/1,booleans/1,setelement/1, cons/1,tuple/1,record_float/1,binary_float/1,float_compare/1, arity_checks/1,elixir_binaries/1,find_best/1, - test_size/1,cover_lists_functions/1,list_append/1]). + test_size/1,cover_lists_functions/1,list_append/1,bad_binary_unit/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. @@ -48,7 +48,8 @@ groups() -> find_best, test_size, cover_lists_functions, - list_append + list_append, + bad_binary_unit ]}]. init_per_suite(Config) -> @@ -508,5 +509,14 @@ list_append(_Config) -> hello = id([]) ++ id(hello), ok. +%% OTP-15872: The compiler would treat the "Unit" of bs_init instructions as +%% the unit of the result instead of the required unit of the input, causing +%% is_binary checks to be wrongly optimized away. +bad_binary_unit(_Config) -> + Bin = id(<<1,2,3>>), + Bitstring = <<Bin/binary,1:1>>, + false = is_binary(Bitstring), + ok. + id(I) -> I. diff --git a/lib/compiler/test/match_SUITE.erl b/lib/compiler/test/match_SUITE.erl index 94bfbb0efe..aac9de278d 100644 --- a/lib/compiler/test/match_SUITE.erl +++ b/lib/compiler/test/match_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2018. All Rights Reserved. +%% Copyright Ericsson AB 2004-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -25,7 +25,7 @@ match_in_call/1,untuplify/1,shortcut_boolean/1,letify_guard/1, selectify/1,deselectify/1,underscore/1,match_map/1,map_vars_used/1, coverage/1,grab_bag/1,literal_binary/1, - unary_op/1,eq_types/1,match_after_return/1]). + unary_op/1,eq_types/1,match_after_return/1,match_right_tuple/1]). -include_lib("common_test/include/ct.hrl"). @@ -41,7 +41,7 @@ groups() -> shortcut_boolean,letify_guard,selectify,deselectify, underscore,match_map,map_vars_used,coverage, grab_bag,literal_binary,unary_op,eq_types, - match_after_return]}]. + match_after_return,match_right_tuple]}]. init_per_suite(Config) -> @@ -902,4 +902,24 @@ match_after_return(Config) when is_list(Config) -> mar_test_tuple(I) -> {gurka, I}. +match_right_tuple(Config) when is_list(Config) -> + %% The loader wrongly coalesced certain get_tuple_element sequences, fusing + %% the code below into a single i_get_tuple_element2 operating on {x,0} + %% even though the first one overwrites it. + %% + %% {get_tuple_element,{x,0},0,{x,0}}. + %% {get_tuple_element,{x,0},1,{x,1}}. + + Inner = {id(wrong_element), id(ok)}, + Outer = {Inner, id(wrong_tuple)}, + ok = match_right_tuple_1(Outer). + +match_right_tuple_1(T) -> + {A, _} = T, + {_, B} = A, + %% The call ensures that A is in {x,0} and B is in {x,1} + id(force_succ_regs(A, B)). + +force_succ_regs(_A, B) -> B. + id(I) -> I. diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index 090579e7ee..508bbc902c 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 7.4.1 +COMPILER_VSN = 7.4.2 diff --git a/lib/crypto/c_src/Makefile.in b/lib/crypto/c_src/Makefile.in index b6a65d7488..2512013ed6 100644 --- a/lib/crypto/c_src/Makefile.in +++ b/lib/crypto/c_src/Makefile.in @@ -92,9 +92,9 @@ CRYPTO_OBJS = $(OBJDIR)/crypto$(TYPEMARKER).o \ $(OBJDIR)/hash$(TYPEMARKER).o \ $(OBJDIR)/hmac$(TYPEMARKER).o \ $(OBJDIR)/info$(TYPEMARKER).o \ + $(OBJDIR)/mac$(TYPEMARKER).o \ $(OBJDIR)/math$(TYPEMARKER).o \ $(OBJDIR)/pkey$(TYPEMARKER).o \ - $(OBJDIR)/poly1305$(TYPEMARKER).o \ $(OBJDIR)/rand$(TYPEMARKER).o \ $(OBJDIR)/rsa$(TYPEMARKER).o \ $(OBJDIR)/srp$(TYPEMARKER).o diff --git a/lib/crypto/c_src/algorithms.c b/lib/crypto/c_src/algorithms.c index 75cddeb1e9..53b8b7eaa9 100644 --- a/lib/crypto/c_src/algorithms.c +++ b/lib/crypto/c_src/algorithms.c @@ -20,13 +20,12 @@ #include "algorithms.h" #include "cipher.h" +#include "mac.h" static unsigned int algo_hash_cnt, algo_hash_fips_cnt; static ERL_NIF_TERM algo_hash[14]; /* increase when extending the list */ static unsigned int algo_pubkey_cnt, algo_pubkey_fips_cnt; static ERL_NIF_TERM algo_pubkey[12]; /* increase when extending the list */ -static unsigned int algo_mac_cnt, algo_mac_fips_cnt; -static ERL_NIF_TERM algo_mac[3]; /* increase when extending the list */ static unsigned int algo_curve_cnt, algo_curve_fips_cnt; static ERL_NIF_TERM algo_curve[89]; /* increase when extending the list */ static unsigned int algo_rsa_opts_cnt, algo_rsa_opts_fips_cnt; @@ -101,19 +100,6 @@ void init_algorithms_types(ErlNifEnv* env) #endif algo_pubkey[algo_pubkey_cnt++] = enif_make_atom(env, "srp"); - - // Validated algorithms first - algo_mac_cnt = 0; - algo_mac[algo_mac_cnt++] = enif_make_atom(env,"hmac"); -#ifdef HAVE_CMAC - algo_mac[algo_mac_cnt++] = enif_make_atom(env,"cmac"); -#endif -#ifdef HAVE_POLY1305 - algo_mac[algo_mac_cnt++] = enif_make_atom(env,"poly1305"); -#endif - // Non-validated algorithms follow - algo_mac_fips_cnt = algo_mac_cnt; - // Validated algorithms first algo_curve_cnt = 0; #if defined(HAVE_EC) @@ -250,7 +236,6 @@ void init_algorithms_types(ErlNifEnv* env) // Check that the max number of algos is updated ASSERT(algo_hash_cnt <= sizeof(algo_hash)/sizeof(ERL_NIF_TERM)); ASSERT(algo_pubkey_cnt <= sizeof(algo_pubkey)/sizeof(ERL_NIF_TERM)); - ASSERT(algo_mac_cnt <= sizeof(algo_mac)/sizeof(ERL_NIF_TERM)); ASSERT(algo_curve_cnt <= sizeof(algo_curve)/sizeof(ERL_NIF_TERM)); ASSERT(algo_rsa_opts_cnt <= sizeof(algo_rsa_opts)/sizeof(ERL_NIF_TERM)); } @@ -284,18 +269,12 @@ ERL_NIF_TERM cipher_algorithms(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv return cipher_types_as_list(env); /* Exclude old api ciphers */ } + ERL_NIF_TERM mac_algorithms(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { - unsigned int cnt = -#ifdef FIPS_SUPPORT - FIPS_mode() ? algo_mac_fips_cnt : -#endif - algo_mac_cnt; - - return enif_make_list_from_array(env, algo_mac, cnt); + return mac_types_as_list(env); } - ERL_NIF_TERM curve_algorithms(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { unsigned int cnt = diff --git a/lib/crypto/c_src/api_ng.c b/lib/crypto/c_src/api_ng.c index 3408ba1b88..a109f444cf 100644 --- a/lib/crypto/c_src/api_ng.c +++ b/lib/crypto/c_src/api_ng.c @@ -100,7 +100,7 @@ static int get_init_args(ErlNifEnv* env, } - if (FORBIDDEN_IN_FIPS(*cipherp)) + if (CIPHER_FORBIDDEN_IN_FIPS(*cipherp)) { *return_term = EXCP_NOTSUP(env, "Forbidden in FIPS"); goto err; diff --git a/lib/crypto/c_src/atoms.c b/lib/crypto/c_src/atoms.c index 059c14690f..bbeb329fa2 100644 --- a/lib/crypto/c_src/atoms.c +++ b/lib/crypto/c_src/atoms.c @@ -30,6 +30,10 @@ ERL_NIF_TERM atom_rsa_no_padding; ERL_NIF_TERM atom_signature_md; ERL_NIF_TERM atom_undefined; +ERL_NIF_TERM atom_hmac; +ERL_NIF_TERM atom_cmac; +ERL_NIF_TERM atom_poly1305; + ERL_NIF_TERM atom_ok; ERL_NIF_TERM atom_none; ERL_NIF_TERM atom_notsup; @@ -155,6 +159,11 @@ int init_atoms(ErlNifEnv *env, const ERL_NIF_TERM fips_mode, const ERL_NIF_TERM atom_rsa_no_padding = enif_make_atom(env,"rsa_no_padding"); atom_signature_md = enif_make_atom(env,"signature_md"); atom_undefined = enif_make_atom(env,"undefined"); + + atom_hmac = enif_make_atom(env,"hmac"); + atom_cmac = enif_make_atom(env,"cmac"); + atom_poly1305 = enif_make_atom(env,"poly1305"); + atom_ok = enif_make_atom(env,"ok"); atom_none = enif_make_atom(env,"none"); atom_notsup = enif_make_atom(env,"notsup"); diff --git a/lib/crypto/c_src/atoms.h b/lib/crypto/c_src/atoms.h index f5913de96f..0e2f1a0022 100644 --- a/lib/crypto/c_src/atoms.h +++ b/lib/crypto/c_src/atoms.h @@ -34,6 +34,10 @@ extern ERL_NIF_TERM atom_rsa_no_padding; extern ERL_NIF_TERM atom_signature_md; extern ERL_NIF_TERM atom_undefined; +extern ERL_NIF_TERM atom_hmac; +extern ERL_NIF_TERM atom_cmac; +extern ERL_NIF_TERM atom_poly1305; + extern ERL_NIF_TERM atom_ok; extern ERL_NIF_TERM atom_none; extern ERL_NIF_TERM atom_notsup; diff --git a/lib/crypto/c_src/cipher.c b/lib/crypto/c_src/cipher.c index 0532fb7566..e144a891a6 100644 --- a/lib/crypto/c_src/cipher.c +++ b/lib/crypto/c_src/cipher.c @@ -214,7 +214,7 @@ ERL_NIF_TERM cipher_info_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[] if ((cipherp = get_cipher_type_no_key(argv[0])) == NULL) return enif_make_badarg(env); - if (FORBIDDEN_IN_FIPS(cipherp)) + if (CIPHER_FORBIDDEN_IN_FIPS(cipherp)) return enif_raise_exception(env, atom_notsup); if ((cipher = cipherp->cipher.p) == NULL) return enif_raise_exception(env, atom_notsup); @@ -330,7 +330,7 @@ ERL_NIF_TERM cipher_types_as_list(ErlNifEnv* env) for (p = cipher_types; (p->type.atom & (p->type.atom != atom_false)); p++) { if ((prev == p->type.atom) || - FORBIDDEN_IN_FIPS(p) ) + CIPHER_FORBIDDEN_IN_FIPS(p) ) continue; if ((p->cipher.p != NULL) || diff --git a/lib/crypto/c_src/cipher.h b/lib/crypto/c_src/cipher.h index 0e51c410eb..c23e128824 100644 --- a/lib/crypto/c_src/cipher.h +++ b/lib/crypto/c_src/cipher.h @@ -52,10 +52,10 @@ struct cipher_type_t { #ifdef FIPS_SUPPORT /* May have FIPS support, must check dynamically if it is enabled */ -# define FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_CIPHER) && FIPS_mode()) +# define CIPHER_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_CIPHER) && FIPS_mode()) #else /* No FIPS support since the symbol FIPS_SUPPORT is undefined */ -# define FORBIDDEN_IN_FIPS(P) 0 +# define CIPHER_FORBIDDEN_IN_FIPS(P) 0 #endif extern ErlNifResourceType* evp_cipher_ctx_rtype; diff --git a/lib/crypto/c_src/cmac.c b/lib/crypto/c_src/cmac.c index 49e67ccf29..a1564f6661 100644 --- a/lib/crypto/c_src/cmac.c +++ b/lib/crypto/c_src/cmac.c @@ -18,71 +18,56 @@ * %CopyrightEnd% */ -#include "cmac.h" -#include "cipher.h" +#include "common.h" -ERL_NIF_TERM cmac_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{/* (Type, Key, Data) */ -#if defined(HAVE_CMAC) - const struct cipher_type_t *cipherp; - const EVP_CIPHER *cipher; - CMAC_CTX *ctx = NULL; - ErlNifBinary key; - ErlNifBinary data; - ERL_NIF_TERM ret; - size_t ret_size; - unsigned char *outp; - int cipher_len; +/***************************************************************** + * + * This file has functions for compatibility with cryptolibs + * lacking the EVP_Digest API. + * + * See mac.c for the implementation using the EVP interface. + * + ****************************************************************/ - ASSERT(argc == 3); +#if defined(HAVE_CMAC) && !defined(HAVE_EVP_PKEY_new_CMAC_key) - if (!enif_inspect_iolist_as_binary(env, argv[1], &key)) - goto bad_arg; - if ((cipherp = get_cipher_type(argv[0], key.size)) == NULL) - goto bad_arg; - if (cipherp->flags & (NON_EVP_CIPHER | AEAD_CIPHER)) - goto bad_arg; - if (!enif_inspect_iolist_as_binary(env, argv[2], &data)) - goto bad_arg; +#include "cmac.h" - if (FORBIDDEN_IN_FIPS(cipherp)) - return enif_raise_exception(env, atom_notsup); - if ((cipher = cipherp->cipher.p) == NULL) - return enif_raise_exception(env, atom_notsup); +int cmac_low_level(ErlNifEnv* env, + ErlNifBinary key_bin, const EVP_CIPHER* cipher, ErlNifBinary text, + ErlNifBinary *ret_bin, int *ret_bin_alloc, ERL_NIF_TERM *return_term) +{ + CMAC_CTX *ctx = NULL; + size_t size; if ((ctx = CMAC_CTX_new()) == NULL) - goto err; - if (!CMAC_Init(ctx, key.data, key.size, cipher, NULL)) - goto err; - if (!CMAC_Update(ctx, data.data, data.size)) - goto err; - if ((cipher_len = EVP_CIPHER_block_size(cipher)) < 0) - goto err; - if ((outp = enif_make_new_binary(env, (size_t)cipher_len, &ret)) == NULL) - goto err; - if (!CMAC_Final(ctx, outp, &ret_size)) - goto err; + goto local_err; - ASSERT(ret_size == (unsigned)EVP_CIPHER_block_size(cipher)); - CONSUME_REDS(env, data); - goto done; + if (!CMAC_Init(ctx, key_bin.data, key_bin.size, cipher, NULL)) + goto local_err; - bad_arg: - return enif_make_badarg(env); + if (!CMAC_Update(ctx, text.data, text.size)) + goto local_err; - err: - ret = atom_notsup; + if ((size = (size_t)EVP_CIPHER_block_size(cipher)) < 0) + goto local_err; - done: + if (!enif_alloc_binary(size, ret_bin)) + goto local_err; + *ret_bin_alloc = 1; + + if (!CMAC_Final(ctx, ret_bin->data, &ret_bin->size)) + goto local_err; + + CMAC_CTX_free(ctx); + return 1; + + local_err: if (ctx) CMAC_CTX_free(ctx); - return ret; -#else - /* The CMAC functionality was introduced in OpenSSL 1.0.1 - * Although OTP requires at least version 0.9.8, the versions 0.9.8 and 1.0.0 are - * no longer maintained. */ - return atom_notsup; -#endif + *return_term = EXCP_ERROR(env,"Compat cmac"); + return 0; } +#endif diff --git a/lib/crypto/c_src/cmac.h b/lib/crypto/c_src/cmac.h index 14488def58..04c742b2dc 100644 --- a/lib/crypto/c_src/cmac.h +++ b/lib/crypto/c_src/cmac.h @@ -23,6 +23,12 @@ #include "common.h" -ERL_NIF_TERM cmac_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +#if defined(HAVE_CMAC) && !defined(HAVE_EVP_PKEY_new_CMAC_key) + +int cmac_low_level(ErlNifEnv* env, + ErlNifBinary key_bin, const EVP_CIPHER* cipher, ErlNifBinary text, + ErlNifBinary *ret_bin, int *ret_bin_alloc, ERL_NIF_TERM *return_term); + +#endif #endif /* E_CMAC_H__ */ diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index d533cba140..802818541b 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -31,7 +31,7 @@ #include "api_ng.h" #include "bn.h" #include "cipher.h" -#include "cmac.h" +#include "mac.h" #include "dh.h" #include "digest.h" #include "dss.h" @@ -46,7 +46,6 @@ #include "info.h" #include "math.h" #include "pkey.h" -#include "poly1305.h" #include "rand.h" #include "rsa.h" #include "srp.h" @@ -74,13 +73,10 @@ static ErlNifFunc nif_funcs[] = { {"hash_init_nif", 1, hash_init_nif, 0}, {"hash_update_nif", 2, hash_update_nif, 0}, {"hash_final_nif", 1, hash_final_nif, 0}, - {"hmac_nif", 3, hmac_nif, 0}, - {"hmac_nif", 4, hmac_nif, 0}, - {"hmac_init_nif", 2, hmac_init_nif, 0}, - {"hmac_update_nif", 2, hmac_update_nif, 0}, - {"hmac_final_nif", 1, hmac_final_nif, 0}, - {"hmac_final_nif", 2, hmac_final_nif, 0}, - {"cmac_nif", 3, cmac_nif, 0}, + {"mac_nif", 4, mac_nif, 0}, + {"mac_init_nif", 3, mac_init_nif, 0}, + {"mac_update_nif", 2, mac_update_nif, 0}, + {"mac_final_nif", 1, mac_final_nif, 0}, {"cipher_info_nif", 1, cipher_info_nif, 0}, {"aes_ige_crypt_nif", 4, aes_ige_crypt_nif, 0}, {"ng_crypto_init_nif", 4, ng_crypto_init_nif, 0}, @@ -112,8 +108,6 @@ static ErlNifFunc nif_funcs[] = { {"aead_cipher", 7, aead_cipher, 0}, - {"poly1305_nif", 2, poly1305_nif, 0}, - {"engine_by_id_nif", 1, engine_by_id_nif, 0}, {"engine_init_nif", 1, engine_init_nif, 0}, {"engine_finish_nif", 1, engine_finish_nif, 0}, @@ -181,9 +175,15 @@ static int initialize(ErlNifEnv* env, ERL_NIF_TERM load_info) if (!enif_inspect_binary(env, tpl_array[1], &lib_bin)) return __LINE__; +#ifdef HAS_EVP_PKEY_CTX + if (!init_mac_ctx(env)) { + return __LINE__; + } +#else if (!init_hmac_ctx(env)) { return __LINE__; } +#endif if (!init_hash_ctx(env)) { return __LINE__; } @@ -248,6 +248,7 @@ static int initialize(ErlNifEnv* env, ERL_NIF_TERM load_info) #endif /* OPENSSL_THREADS */ init_digest_types(env); + init_mac_types(env); init_cipher_types(env); init_algorithms_types(env); diff --git a/lib/crypto/c_src/digest.c b/lib/crypto/c_src/digest.c index c987a664d5..0f887ab765 100644 --- a/lib/crypto/c_src/digest.c +++ b/lib/crypto/c_src/digest.c @@ -22,7 +22,7 @@ static struct digest_type_t digest_types[] = { - {{"md4"}, + {{"md4"}, NO_FIPS_DIGEST, #ifdef HAVE_MD4 {&EVP_md4} #else @@ -30,7 +30,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"md5"}, + {{"md5"}, NO_FIPS_DIGEST, #ifdef HAVE_MD5 {&EVP_md5} #else @@ -38,7 +38,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"ripemd160"}, + {{"ripemd160"}, NO_FIPS_DIGEST, #ifdef HAVE_RIPEMD160 {&EVP_ripemd160} #else @@ -46,9 +46,9 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha"}, {&EVP_sha1}}, + {{"sha"}, 0, {&EVP_sha1}}, - {{"sha224"}, + {{"sha224"}, 0, #ifdef HAVE_SHA224 {&EVP_sha224} #else @@ -56,7 +56,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha256"}, + {{"sha256"}, 0, #ifdef HAVE_SHA256 {&EVP_sha256} #else @@ -64,7 +64,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha384"}, + {{"sha384"}, 0, #ifdef HAVE_SHA384 {&EVP_sha384} #else @@ -72,7 +72,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha512"}, + {{"sha512"}, 0, #ifdef HAVE_SHA512 {&EVP_sha512} #else @@ -80,7 +80,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha3_224"}, + {{"sha3_224"}, 0, #ifdef HAVE_SHA3_224 {&EVP_sha3_224} #else @@ -88,7 +88,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha3_256"}, + {{"sha3_256"}, 0, #ifdef HAVE_SHA3_256 {&EVP_sha3_256} #else @@ -96,7 +96,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha3_384"}, + {{"sha3_384"}, 0, #ifdef HAVE_SHA3_384 {&EVP_sha3_384} #else @@ -104,7 +104,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha3_512"}, + {{"sha3_512"}, 0, #ifdef HAVE_SHA3_512 {&EVP_sha3_512} #else @@ -112,7 +112,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"blake2b"}, + {{"blake2b"}, 0, #ifdef HAVE_BLAKE2 {&EVP_blake2b512} #else @@ -120,7 +120,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"blake2s"}, + {{"blake2s"}, 0, #ifdef HAVE_BLAKE2 {&EVP_blake2s256} #else @@ -128,7 +128,8 @@ static struct digest_type_t digest_types[] = #endif }, - {{NULL}, {NULL}} + /*==== End of list ==== */ + {{NULL}, 0, {NULL}} }; void init_digest_types(ErlNifEnv* env) diff --git a/lib/crypto/c_src/digest.h b/lib/crypto/c_src/digest.h index 06852416cf..b1f8128a1f 100644 --- a/lib/crypto/c_src/digest.h +++ b/lib/crypto/c_src/digest.h @@ -28,12 +28,25 @@ struct digest_type_t { const char* str; /* before init, NULL for end-of-table */ ERL_NIF_TERM atom; /* after init, 'false' for end-of-table */ }type; + unsigned flags; union { const EVP_MD* (*funcp)(void); /* before init, NULL if notsup */ const EVP_MD* p; /* after init, NULL if notsup */ }md; }; +/* masks in the flags field if digest_type_t */ +#define NO_FIPS_DIGEST 1 + +#ifdef FIPS_SUPPORT +/* May have FIPS support, must check dynamically if it is enabled */ +# define DIGEST_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_DIGEST) && FIPS_mode()) +#else +/* No FIPS support since the symbol FIPS_SUPPORT is undefined */ +# define DIGEST_FORBIDDEN_IN_FIPS(P) 0 +#endif + + void init_digest_types(ErlNifEnv* env); struct digest_type_t* get_digest_type(ERL_NIF_TERM type); diff --git a/lib/crypto/c_src/hmac.c b/lib/crypto/c_src/hmac.c index ff7005d75e..5e2c68bfee 100644 --- a/lib/crypto/c_src/hmac.c +++ b/lib/crypto/c_src/hmac.c @@ -18,6 +18,18 @@ * %CopyrightEnd% */ + +/***************************************************************** + * + * This file has functions for compatibility with cryptolibs + * lacking the EVP_Digest API. + * + * See mac.c for the implementation using the EVP interface. + * + ****************************************************************/ + +#ifndef HAS_EVP_PKEY_CTX + #include "hmac.h" #include "digest.h" @@ -47,61 +59,6 @@ int init_hmac_ctx(ErlNifEnv *env) { return 0; } -ERL_NIF_TERM hmac_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{/* (Type, Key, Data) or (Type, Key, Data, MacSize) */ - struct digest_type_t *digp = NULL; - ErlNifBinary key, data; - unsigned char buff[EVP_MAX_MD_SIZE]; - unsigned size = 0, req_size = 0; - ERL_NIF_TERM ret; - unsigned char *outp; - - ASSERT(argc == 3 || argc == 4); - - if ((digp = get_digest_type(argv[0])) == NULL) - goto bad_arg; - if (!enif_inspect_iolist_as_binary(env, argv[1], &key)) - goto bad_arg; - if (key.size > INT_MAX) - goto bad_arg; - if (!enif_inspect_iolist_as_binary(env, argv[2], &data)) - goto bad_arg; - if (argc == 4) { - if (!enif_get_uint(env, argv[3], &req_size)) - goto bad_arg; - } - - if (digp->md.p == NULL) - goto err; - if (HMAC(digp->md.p, - key.data, (int)key.size, - data.data, data.size, - buff, &size) == NULL) - goto err; - - ASSERT(0 < size && size <= EVP_MAX_MD_SIZE); - CONSUME_REDS(env, data); - - if (argc == 4) { - if (req_size > size) - goto bad_arg; - - size = req_size; - } - - if ((outp = enif_make_new_binary(env, size, &ret)) == NULL) - goto err; - - memcpy(outp, buff, size); - return ret; - - bad_arg: - return enif_make_badarg(env); - - err: - return atom_notsup; -} - static void hmac_context_dtor(ErlNifEnv* env, struct hmac_context *obj) { if (obj == NULL) @@ -118,17 +75,17 @@ static void hmac_context_dtor(ErlNifEnv* env, struct hmac_context *obj) } ERL_NIF_TERM hmac_init_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{/* (Type, Key) */ +{/* (hmac, Type, Key) */ struct digest_type_t *digp = NULL; ErlNifBinary key; ERL_NIF_TERM ret; struct hmac_context *obj = NULL; - ASSERT(argc == 2); + ASSERT(argc == 3); - if ((digp = get_digest_type(argv[0])) == NULL) + if ((digp = get_digest_type(argv[1])) == NULL) goto bad_arg; - if (!enif_inspect_iolist_as_binary(env, argv[1], &key)) + if (!enif_inspect_iolist_as_binary(env, argv[2], &key)) goto bad_arg; if (key.size > INT_MAX) goto bad_arg; @@ -268,3 +225,44 @@ ERL_NIF_TERM hmac_final_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) return ret; } + + +int hmac_low_level(ErlNifEnv* env, const EVP_MD *md, + ErlNifBinary key_bin, ErlNifBinary text, + ErlNifBinary *ret_bin, int *ret_bin_alloc, ERL_NIF_TERM *return_term) +{ + unsigned int size_int; + size_t size; + + /* Find the needed space */ + if (HMAC(md, + key_bin.data, (int)key_bin.size, + text.data, text.size, + NULL, &size_int) == NULL) + { + *return_term = EXCP_ERROR(env, "Get HMAC size failed"); + return 0; + } + + size = (size_t)size_int; /* Otherwise "size" is unused in 0.9.8.... */ + if (!enif_alloc_binary(size, ret_bin)) + { + *return_term = EXCP_ERROR(env, "Alloc binary"); + return 0; + } + *ret_bin_alloc = 1; + + /* And do the real HMAC calc */ + if (HMAC(md, + key_bin.data, (int)key_bin.size, + text.data, text.size, + ret_bin->data, &size_int) == NULL) + { + *return_term = EXCP_ERROR(env, "HMAC sign failed"); + return 0; + } + + return 1; +} + +#endif diff --git a/lib/crypto/c_src/hmac.h b/lib/crypto/c_src/hmac.h index 1f0e0ca632..f5805e13e5 100644 --- a/lib/crypto/c_src/hmac.h +++ b/lib/crypto/c_src/hmac.h @@ -21,13 +21,19 @@ #ifndef E_HMAC_H__ #define E_HMAC_H__ 1 +#ifndef HAS_EVP_PKEY_CTX + #include "common.h" int init_hmac_ctx(ErlNifEnv *env); -ERL_NIF_TERM hmac_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); ERL_NIF_TERM hmac_init_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); ERL_NIF_TERM hmac_update_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); ERL_NIF_TERM hmac_final_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +int hmac_low_level(ErlNifEnv* env, const EVP_MD *md, + ErlNifBinary key_bin, ErlNifBinary text, + ErlNifBinary *ret_bin, int *ret_bin_alloc, ERL_NIF_TERM *return_term); +#endif + #endif /* E_HMAC_H__ */ diff --git a/lib/crypto/c_src/mac.c b/lib/crypto/c_src/mac.c new file mode 100644 index 0000000000..149975ba9d --- /dev/null +++ b/lib/crypto/c_src/mac.c @@ -0,0 +1,751 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010-2019. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + */ + +#include "common.h" +#include "cipher.h" +#include "digest.h" +#include "cmac.h" +#include "hmac.h" +#include "mac.h" + +/*************************** + MAC type declaration +***************************/ + +struct mac_type_t { + union { + const char* str; /* before init, NULL for end-of-table */ + ERL_NIF_TERM atom; /* after init, 'false' for end-of-table */ + }name; + unsigned flags; + union { + const int pkey_type; + }alg; + int type; + size_t key_len; /* != 0 to also match on key_len */ +}; + +/* masks in the flags field if mac_type_t */ +#define NO_FIPS_MAC 1 + +#define NO_mac 0 +#define HMAC_mac 1 +#define CMAC_mac 2 +#define POLY1305_mac 3 + +static struct mac_type_t mac_types[] = +{ + {{"poly1305"}, NO_FIPS_MAC, +#ifdef HAVE_POLY1305 + /* If we have POLY then we have EVP_PKEY */ + {EVP_PKEY_POLY1305}, POLY1305_mac, 32 +#else + {EVP_PKEY_NONE}, NO_mac, 0 +#endif + }, + + {{"hmac"}, 0, +#ifdef HAS_EVP_PKEY_CTX + {EVP_PKEY_HMAC}, HMAC_mac, 0 +#else + /* HMAC is always supported, but possibly with low-level routines */ + {EVP_PKEY_NONE}, HMAC_mac, 0 +#endif + }, + + {{"cmac"}, 0, +#ifdef HAVE_CMAC + /* If we have CMAC then we have EVP_PKEY */ + {EVP_PKEY_CMAC}, CMAC_mac, 0 +#else + {EVP_PKEY_NONE}, NO_mac, 0 +#endif + }, + + /*==== End of list ==== */ + {{NULL}, 0, + {0}, NO_mac, 0 + } +}; + + +#ifdef FIPS_SUPPORT +/* May have FIPS support, must check dynamically if it is enabled */ +# define MAC_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_MAC) && FIPS_mode()) +#else +/* No FIPS support since the symbol FIPS_SUPPORT is undefined */ +# define MAC_FORBIDDEN_IN_FIPS(P) 0 +#endif + + +/*************************** + Mandatory prototypes +***************************/ + +struct mac_type_t* get_mac_type(ERL_NIF_TERM type, size_t key_len); +struct mac_type_t* get_mac_type_no_key(ERL_NIF_TERM type); + +ERL_NIF_TERM mac_one_time(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); + +ERL_NIF_TERM mac_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); + + +/******************************** + Support functions for type array +*********************************/ + +void init_mac_types(ErlNifEnv* env) +{ + struct mac_type_t* p = mac_types; + + for (p = mac_types; p->name.str; p++) { + p->name.atom = enif_make_atom(env, p->name.str); + } + p->name.atom = atom_false; /* end marker */ +} + + +ERL_NIF_TERM mac_types_as_list(ErlNifEnv* env) +{ + struct mac_type_t* p; + ERL_NIF_TERM prev, hd; + + hd = enif_make_list(env, 0); + prev = atom_undefined; + + for (p = mac_types; (p->name.atom & (p->name.atom != atom_false)); p++) { + if (prev == p->name.atom) + continue; + + if (p->type != NO_mac) + { + hd = enif_make_list_cell(env, p->name.atom, hd); + } + } + + return hd; +} + +struct mac_type_t* get_mac_type(ERL_NIF_TERM type, size_t key_len) +{ + struct mac_type_t* p = NULL; + for (p = mac_types; p->name.atom != atom_false; p++) { + if (type == p->name.atom) { + if ((p->key_len == 0) || (p->key_len == key_len)) + return p; + } + } + return NULL; +} + +struct mac_type_t* get_mac_type_no_key(ERL_NIF_TERM type) +{ + struct mac_type_t* p = NULL; + for (p = mac_types; p->name.atom != atom_false; p++) { + if (type == p->name.atom) { + return p; + } + } + return NULL; +} + +/******************************************************************* + * + * Mac nif + * + ******************************************************************/ +ERL_NIF_TERM mac_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (MacType, SubType, Key, Text) */ + ErlNifBinary text; + + if (!enif_inspect_iolist_as_binary(env, argv[3], &text)) + return EXCP_BADARG(env, "Bad text"); + + if (text.size > INT_MAX) + return EXCP_BADARG(env, "Too long text"); + + /* Run long jobs on a dirty scheduler to not block the current emulator thread */ + if (text.size > MAX_BYTES_TO_NIF) { + return enif_schedule_nif(env, "mac_one_time", + ERL_NIF_DIRTY_JOB_CPU_BOUND, + mac_one_time, argc, argv); + } + + return mac_one_time(env, argc, argv); +} + + + +ERL_NIF_TERM mac_one_time(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (MacType, SubType, Key, Text) */ + + struct mac_type_t *macp; + ErlNifBinary key_bin, text; + int ret_bin_alloc = 0; + ERL_NIF_TERM return_term; + const EVP_MD *md = NULL; + ErlNifBinary ret_bin; +#ifdef HAS_EVP_PKEY_CTX + size_t size; + EVP_PKEY *pkey = NULL; + EVP_MD_CTX *mctx = NULL; +#endif + + /*--------------------------------- + Get common indata and validate it + */ + if (!enif_inspect_iolist_as_binary(env, argv[2], &key_bin)) + { + return_term = EXCP_BADARG(env, "Bad key"); + goto err; + } + + if (!enif_inspect_iolist_as_binary(env, argv[3], &text)) + { + return_term = EXCP_BADARG(env, "Bad text"); + goto err; + } + + if (!(macp = get_mac_type(argv[0], key_bin.size))) + { + if (!get_mac_type_no_key(argv[0])) + return_term = EXCP_BADARG(env, "Unknown mac algorithm"); + else + return_term = EXCP_BADARG(env, "Bad key length"); + goto err; + } + + if (MAC_FORBIDDEN_IN_FIPS(macp)) + { + return_term = EXCP_NOTSUP(env, "MAC algorithm forbidden in FIPS"); + goto err; + } + + /*-------------------------------------------------- + Algorithm dependent indata checking and computation. + If EVP_PKEY is available, only set the pkey variable + and do the computation after the switch statement. + If not available, do the low-level calls in the + corresponding case part + */ + switch (macp->type) { + + /******** + * HMAC * + ********/ + case HMAC_mac: + { + struct digest_type_t *digp; + + if ((digp = get_digest_type(argv[1])) == NULL) + { + return_term = EXCP_BADARG(env, "Bad digest algorithm for HMAC"); + goto err; + } + if (digp->md.p == NULL) + { + return_term = EXCP_NOTSUP(env, "Unsupported digest algorithm"); + goto err; + } + if (DIGEST_FORBIDDEN_IN_FIPS(digp)) + { + return_term = EXCP_NOTSUP(env, "Digest algorithm for HMAC forbidden in FIPS"); + goto err; + } + md = digp->md.p; + +#ifdef HAS_EVP_PKEY_CTX +# ifdef HAVE_PKEY_new_raw_private_key + /* Prefered for new applications according to EVP_PKEY_new_mac_key(3) */ + pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_HMAC, /*engine*/ NULL, key_bin.data, key_bin.size); +# else + /* Available in older versions */ + pkey = EVP_PKEY_new_mac_key(EVP_PKEY_HMAC, /*engine*/ NULL, key_bin.data, key_bin.size); +# endif + +#else + if (!hmac_low_level(env, md, key_bin, text, &ret_bin, &ret_bin_alloc, &return_term)) + goto err; + else + goto success; +#endif + } + break; + + + /******** + * CMAC * + ********/ +#ifdef HAVE_CMAC + case CMAC_mac: + { + const struct cipher_type_t *cipherp; + if (!(cipherp = get_cipher_type(argv[1], key_bin.size))) + { /* Something went wrong. Find out what by retrying in another way. */ + if (!get_cipher_type_no_key(argv[1])) + return_term = EXCP_BADARG(env, "Unknown cipher"); + else + /* Cipher exists, so it must be the key size that is wrong */ + return_term = EXCP_BADARG(env, "Bad key size"); + goto err; + } + + if (CIPHER_FORBIDDEN_IN_FIPS(cipherp)) + { + return_term = EXCP_NOTSUP(env, "Cipher algorithm not supported in FIPS"); + goto err; + } + + if (cipherp->cipher.p == NULL) + { + return_term = EXCP_NOTSUP(env, "Unsupported cipher algorithm"); + goto err; + } + +# ifdef HAVE_EVP_PKEY_new_CMAC_key + pkey = EVP_PKEY_new_CMAC_key(/*engine*/ NULL, key_bin.data, key_bin.size, cipherp->cipher.p); +# else + if (!cmac_low_level(env, key_bin, cipherp->cipher.p, text, &ret_bin, &ret_bin_alloc, &return_term)) + goto err; + else + goto success; +# endif + } + break; +#endif /* HAVE_CMAC */ + + + /************ + * POLY1305 * + ************/ +#ifdef HAVE_POLY1305 + case POLY1305_mac: + /* poly1305 implies that EVP_PKEY_new_raw_private_key exists */ + pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_POLY1305, /*engine*/ NULL, key_bin.data, key_bin.size); + break; +#endif + + + /*************** + * Unknown MAC * + ***************/ + case NO_mac: + default: + /* We know that this mac is supported with some version(s) of cryptolib */ + return_term = EXCP_NOTSUP(env, "Unsupported mac algorithm"); + goto err; + } + + /*----------------------------------------- + Common computations when we have EVP_PKEY + */ +#ifdef HAS_EVP_PKEY_CTX + if (!pkey) + { + return_term = EXCP_ERROR(env, "EVP_PKEY_key creation"); + goto err; + } + + if ((mctx = EVP_MD_CTX_new()) == NULL) + { + return_term = EXCP_ERROR(env, "EVP_MD_CTX_new"); + goto err; + } + + if (EVP_DigestSignInit(mctx, /*&pctx*/ NULL, md, /*engine*/ NULL, pkey) != 1) + { + return_term = EXCP_ERROR(env, "EVP_DigestSign"); + goto err; + } + +# ifdef HAVE_DigestSign_as_single_op + if (EVP_DigestSign(mctx, NULL, &size, text.data, text.size) != 1) + { + return_term = EXCP_ERROR(env, "Can't get sign size"); + goto err; + } +# else + if (EVP_DigestSignUpdate(mctx, text.data, text.size) != 1) + { + return_term = EXCP_ERROR(env, "EVP_DigestSignUpdate"); + goto err; + } + + if (EVP_DigestSignFinal(mctx, NULL, &size) != 1) + { + return_term = EXCP_ERROR(env, "Can't get sign size"); + goto err; + } +# endif + + if (!enif_alloc_binary(size, &ret_bin)) + { + return_term = EXCP_ERROR(env, "Alloc binary"); + goto err; + } + ret_bin_alloc = 1; + +# ifdef HAVE_DigestSign_as_single_op + if (EVP_DigestSign(mctx, ret_bin.data, &size, text.data, text.size) != 1) +# else + if (EVP_DigestSignFinal(mctx, ret_bin.data, &size) != 1) +# endif + { + return_term = EXCP_ERROR(env, "Signing"); + goto err; + } + + goto success; /* The label "success:" could be left without any "goto success" + in some combination of flags. This prevents a compiler warning + */ +#endif /* ifdef HAS_EVP_PKEY_CTX */ + + + /**************************** + Exit when we got a signature + *****************************/ + success: + CONSUME_REDS(env, text); + + return_term = enif_make_binary(env, &ret_bin); + ret_bin_alloc = 0; + + err: + +#ifdef HAS_EVP_PKEY_CTX + if (pkey) + EVP_PKEY_free(pkey); + if (mctx) + EVP_MD_CTX_free(mctx); +#endif + + if (ret_bin_alloc) + enif_release_binary(&ret_bin); + + return return_term; +} + + +/******************************************************************* + * + * Mac ctx + * + ******************************************************************/ + +int init_mac_ctx(ErlNifEnv *env); + +struct mac_context +{ + EVP_MD_CTX *ctx; +}; + +static ErlNifResourceType* mac_context_rtype; + +static void mac_context_dtor(ErlNifEnv* env, struct mac_context*); + +int init_mac_ctx(ErlNifEnv *env) { + mac_context_rtype = enif_open_resource_type(env, NULL, "mac_context", + (ErlNifResourceDtor*) mac_context_dtor, + ERL_NIF_RT_CREATE|ERL_NIF_RT_TAKEOVER, + NULL); + if (mac_context_rtype == NULL) + goto err; + + return 1; + + err: + PRINTF_ERR0("CRYPTO: Could not open resource type 'mac_context'"); + return 0; +} + + +static void mac_context_dtor(ErlNifEnv* env, struct mac_context *obj) +{ + if (obj == NULL) + return; + + if (obj->ctx) + EVP_MD_CTX_free(obj->ctx); +} + +/******************************************************************* + * + * mac_init, mac_update, mac_final nifs + * + ******************************************************************/ + +ERL_NIF_TERM mac_init_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (MacType, SubType, Key) */ +#ifdef HAS_EVP_PKEY_CTX + struct mac_context *obj = NULL; + struct mac_type_t *macp; + ErlNifBinary key_bin; + ERL_NIF_TERM return_term; + const EVP_MD *md = NULL; + EVP_PKEY *pkey = NULL; + + /*--------------------------------- + Get common indata and validate it + */ + if (!enif_inspect_iolist_as_binary(env, argv[2], &key_bin)) + { + return_term = EXCP_BADARG(env, "Bad key"); + goto err; + } + + if (!(macp = get_mac_type(argv[0], key_bin.size))) + { + if (!get_mac_type_no_key(argv[0])) + return_term = EXCP_BADARG(env, "Unknown mac algorithm"); + else + return_term = EXCP_BADARG(env, "Bad key length"); + goto err; + } + + if (MAC_FORBIDDEN_IN_FIPS(macp)) + { + return_term = EXCP_NOTSUP(env, "MAC algorithm forbidden in FIPS"); + goto err; + } + + /*-------------------------------------------------- + Algorithm dependent indata checking and computation. + If EVP_PKEY is available, only set the pkey variable + and do the computation after the switch statement. + If not available, do the low-level calls in the + corresponding case part + */ + switch (macp->type) { + + /******** + * HMAC * + ********/ + case HMAC_mac: + { + struct digest_type_t *digp; + + if ((digp = get_digest_type(argv[1])) == NULL) + { + return_term = EXCP_BADARG(env, "Bad digest algorithm for HMAC"); + goto err; + } + if (digp->md.p == NULL) + { + return_term = EXCP_NOTSUP(env, "Unsupported digest algorithm"); + goto err; + } + if (DIGEST_FORBIDDEN_IN_FIPS(digp)) + { + return_term = EXCP_NOTSUP(env, "Digest algorithm for HMAC forbidden in FIPS"); + goto err; + } + md = digp->md.p; + +# ifdef HAVE_PKEY_new_raw_private_key + /* Prefered for new applications according to EVP_PKEY_new_mac_key(3) */ + pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_HMAC, /*engine*/ NULL, key_bin.data, key_bin.size); +# else + /* Available in older versions */ + pkey = EVP_PKEY_new_mac_key(EVP_PKEY_HMAC, /*engine*/ NULL, key_bin.data, key_bin.size); +# endif + } + break; + + + /******** + * CMAC * + ********/ +#if defined(HAVE_CMAC) && defined(HAVE_EVP_PKEY_new_CMAC_key) + case CMAC_mac: + { + const struct cipher_type_t *cipherp; + if (!(cipherp = get_cipher_type(argv[1], key_bin.size))) + { /* Something went wrong. Find out what by retrying in another way. */ + if (!get_cipher_type_no_key(argv[1])) + return_term = EXCP_BADARG(env, "Unknown cipher"); + else + /* Cipher exists, so it must be the key size that is wrong */ + return_term = EXCP_BADARG(env, "Bad key size"); + goto err; + } + + if (CIPHER_FORBIDDEN_IN_FIPS(cipherp)) + { + return_term = EXCP_NOTSUP(env, "Cipher algorithm not supported in FIPS"); + goto err; + } + + if (cipherp->cipher.p == NULL) + { + return_term = EXCP_NOTSUP(env, "Unsupported cipher algorithm"); + goto err; + } + + pkey = EVP_PKEY_new_CMAC_key(/*engine*/ NULL, key_bin.data, key_bin.size, cipherp->cipher.p); + } + break; +#endif /* HAVE_CMAC && HAVE_EVP_PKEY_new_CMAC_key */ + + + /************ + * POLY1305 * + ************/ +#ifdef HAVE_POLY1305 + case POLY1305_mac: + /* poly1305 implies that EVP_PKEY_new_raw_private_key exists */ + pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_POLY1305, /*engine*/ NULL, key_bin.data, key_bin.size); + break; +#endif + + + /*************** + * Unknown MAC * + ***************/ + case NO_mac: + default: + /* We know that this mac is supported with some version(s) of cryptolib */ + return_term = EXCP_NOTSUP(env, "Unsupported mac algorithm"); + goto err; + } + + /*----------------------------------------- + Common computations + */ + if (!pkey) + { + return_term = EXCP_ERROR(env, "EVP_PKEY_key creation"); + goto err; + } + + if ((obj = enif_alloc_resource(mac_context_rtype, sizeof(struct mac_context))) == NULL) + { + return_term = EXCP_ERROR(env, "Can't allocate mac_context_rtype"); + goto err; + } + + if ((obj->ctx = EVP_MD_CTX_new()) == NULL) + { + return_term = EXCP_ERROR(env, "EVP_MD_CTX_new"); + goto err; + } + + if (EVP_DigestSignInit(obj->ctx, /*&pctx*/ NULL, md, /*engine*/ NULL, pkey) != 1) + { + return_term = EXCP_ERROR(env, "EVP_DigestSign"); + goto err; + } + + return_term = enif_make_resource(env, obj); + + err: + + if (obj) + enif_release_resource(obj); + + if (pkey) + EVP_PKEY_free(pkey); + + return return_term; + +#else + if (argv[0] != atom_hmac) + return EXCP_NOTSUP(env, "Unsupported mac algorithm"); + + return hmac_init_nif(env, argc, argv); +#endif +} + + + +ERL_NIF_TERM mac_update_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Ref, Text) */ + ErlNifBinary text; + + if (!enif_inspect_iolist_as_binary(env, argv[1], &text)) + return EXCP_BADARG(env, "Bad text"); + + if (text.size > INT_MAX) + return EXCP_BADARG(env, "Too long text"); + + /* Run long jobs on a dirty scheduler to not block the current emulator thread */ + if (text.size > MAX_BYTES_TO_NIF) { + return enif_schedule_nif(env, "mac_update", + ERL_NIF_DIRTY_JOB_CPU_BOUND, + mac_update, argc, argv); + } + + return mac_update(env, argc, argv); +} + + +ERL_NIF_TERM mac_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Ref, Text) */ +#ifdef HAS_EVP_PKEY_CTX + struct mac_context *obj = NULL; + ErlNifBinary text; + + if (!enif_get_resource(env, argv[0], (ErlNifResourceType*)mac_context_rtype, (void**)&obj)) + return EXCP_BADARG(env, "Bad ref"); + + if (!enif_inspect_iolist_as_binary(env, argv[1], &text)) + return EXCP_BADARG(env, "Bad text"); + + if (EVP_DigestSignUpdate(obj->ctx, text.data, text.size) != 1) + return EXCP_ERROR(env, "EVP_DigestSignUpdate"); + + CONSUME_REDS(env, text); + return argv[0]; + +#else + return hmac_update_nif(env, argc, argv); +#endif +} + + + +ERL_NIF_TERM mac_final_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Ref) */ +#ifdef HAS_EVP_PKEY_CTX + struct mac_context *obj; + size_t size; + ErlNifBinary ret_bin; + + if (!enif_get_resource(env, argv[0], (ErlNifResourceType*)mac_context_rtype, (void**)&obj)) + return EXCP_BADARG(env, "Bad ref"); + + if (EVP_DigestSignFinal(obj->ctx, NULL, &size) != 1) + return EXCP_ERROR(env, "Can't get sign size"); + + if (!enif_alloc_binary(size, &ret_bin)) + return EXCP_ERROR(env, "Alloc binary"); + + if (EVP_DigestSignFinal(obj->ctx, ret_bin.data, &size) != 1) + { + enif_release_binary(&ret_bin); + return EXCP_ERROR(env, "Signing"); + } + + return enif_make_binary(env, &ret_bin); + +#else + return hmac_final_nif(env, argc, argv); +#endif +} + diff --git a/lib/crypto/c_src/poly1305.h b/lib/crypto/c_src/mac.h index 4bf45e6218..053a331324 100644 --- a/lib/crypto/c_src/poly1305.h +++ b/lib/crypto/c_src/mac.h @@ -18,11 +18,21 @@ * %CopyrightEnd% */ -#ifndef E_POLY1305_H__ -#define E_POLY1305_H__ 1 +#ifndef E_MAC_H__ +#define E_MAC_H__ 1 #include "common.h" -ERL_NIF_TERM poly1305_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +int init_mac_ctx(ErlNifEnv *env); -#endif /* E_POLY1305_H__ */ +void init_mac_types(ErlNifEnv* env); + +ERL_NIF_TERM mac_types_as_list(ErlNifEnv* env); + +ERL_NIF_TERM mac_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); + +ERL_NIF_TERM mac_init_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +ERL_NIF_TERM mac_update_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +ERL_NIF_TERM mac_final_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); + +#endif /* E_MAC_H__ */ diff --git a/lib/crypto/c_src/openssl_config.h b/lib/crypto/c_src/openssl_config.h index 339eb5b8f4..32a0830717 100644 --- a/lib/crypto/c_src/openssl_config.h +++ b/lib/crypto/c_src/openssl_config.h @@ -110,6 +110,12 @@ # define HAS_EVP_PKEY_CTX # define HAVE_EVP_CIPHER_CTX_COPY # endif + +# if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION_PLAIN(1,1,1) +# define HAVE_PKEY_new_raw_private_key +# define HAVE_EVP_PKEY_new_CMAC_key +# define HAVE_DigestSign_as_single_op +# endif #endif diff --git a/lib/crypto/c_src/pkey.c b/lib/crypto/c_src/pkey.c index a1e2677b34..d53d91c25b 100644 --- a/lib/crypto/c_src/pkey.c +++ b/lib/crypto/c_src/pkey.c @@ -59,8 +59,9 @@ static int get_pkey_public_key(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_T EVP_PKEY **pkey); static int get_pkey_crypt_options(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_TERM options, PKeyCryptOptions *opt); +#ifdef HAVE_RSA_SSLV23_PADDING static size_t size_of_RSA(EVP_PKEY *pkey); - +#endif static int get_pkey_digest_type(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_TERM type, const EVP_MD **md) @@ -1031,6 +1032,7 @@ static int get_pkey_crypt_options(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NI return PKEY_BADARG; } +#ifdef HAVE_RSA_SSLV23_PADDING static size_t size_of_RSA(EVP_PKEY *pkey) { int ret = 0; RSA *rsa = NULL; @@ -1045,6 +1047,7 @@ static size_t size_of_RSA(EVP_PKEY *pkey) { return (ret < 0) ? 0 : (size_t)ret; } +#endif ERL_NIF_TERM pkey_crypt_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) {/* (Algorithm, Data, PublKey=[E,N]|[E,N,D]|[E,N,D,P1,P2,E1,E2,C], Options, IsPrivate, IsEncrypt) */ diff --git a/lib/crypto/c_src/poly1305.c b/lib/crypto/c_src/poly1305.c deleted file mode 100644 index 76579c0a29..0000000000 --- a/lib/crypto/c_src/poly1305.c +++ /dev/null @@ -1,90 +0,0 @@ -/* - * %CopyrightBegin% - * - * Copyright Ericsson AB 2010-2018. All Rights Reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - * %CopyrightEnd% - */ - -#include "poly1305.h" - -/* For OpenSSL >= 1.1.1 the hmac_nif and cmac_nif could be integrated into poly1305 (with 'type' as parameter) */ -ERL_NIF_TERM poly1305_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{/* (Key, Text) */ -#ifdef HAVE_POLY1305 - ErlNifBinary key_bin, text, ret_bin; - ERL_NIF_TERM ret; - EVP_PKEY *key = NULL; - EVP_MD_CTX *mctx = NULL; - EVP_PKEY_CTX *pctx = NULL; - const EVP_MD *md = NULL; - size_t size; - int ret_bin_alloc = 0; - - ASSERT(argc == 2); - - if (!enif_inspect_binary(env, argv[0], &key_bin)) - goto bad_arg; - if (key_bin.size != 32) - goto bad_arg; - if (!enif_inspect_binary(env, argv[1], &text)) - goto bad_arg; - - if ((key = EVP_PKEY_new_raw_private_key(EVP_PKEY_POLY1305, /*engine*/ NULL, key_bin.data, key_bin.size)) == NULL) - goto err; - - if ((mctx = EVP_MD_CTX_new()) == NULL) - goto err; - if (EVP_DigestSignInit(mctx, &pctx, md, /*engine*/ NULL, key) != 1) - goto err; - if (EVP_DigestSignUpdate(mctx, text.data, text.size) != 1) - goto err; - - if (EVP_DigestSignFinal(mctx, NULL, &size) != 1) - goto err; - if (!enif_alloc_binary(size, &ret_bin)) - goto err; - ret_bin_alloc = 1; - if (EVP_DigestSignFinal(mctx, ret_bin.data, &size) != 1) - goto err; - - if (size != ret_bin.size) { - if (!enif_realloc_binary(&ret_bin, size)) - goto err; - } - - ret = enif_make_binary(env, &ret_bin); - ret_bin_alloc = 0; - goto done; - - bad_arg: - return enif_make_badarg(env); - - err: - if (ret_bin_alloc) - enif_release_binary(&ret_bin); - ret = atom_error; - - done: - if (mctx) - EVP_MD_CTX_free(mctx); - if (key) - EVP_PKEY_free(key); - return ret; - -#else - return enif_raise_exception(env, atom_notsup); -#endif -} diff --git a/lib/crypto/doc/src/algorithm_details.xml b/lib/crypto/doc/src/algorithm_details.xml index 854bfbb4b1..71014764c8 100644 --- a/lib/crypto/doc/src/algorithm_details.xml +++ b/lib/crypto/doc/src/algorithm_details.xml @@ -37,122 +37,163 @@ <section> <title>Ciphers</title> + <p>A <seealso marker="crypto#type-cipher">cipher</seealso> in the + <seealso marker="crypto:new_api#the-new-api">new api</seealso> + is categorized as either + <seealso marker="crypto#type-cipher_no_iv">cipher_no_iv()</seealso>, + <seealso marker="crypto#type-cipher_iv">cipher_iv()</seealso> or + <seealso marker="crypto#type-cipher_aead">cipher_aead()</seealso>. + The letters IV are short for <i>Initialization Vector</i> and + AEAD is an abreviation of <i>Authenticated Encryption with Associated Data</i>. + </p> + <p>Due to irregular naming conventions, some cipher names in the old api are + substitued by new names in the new api. For a list of retired names, see + <seealso marker="crypto:new_api#retired-cipher-names">Retired cipher names</seealso>. + </p> + <p>To dynamically check availability, check that the name in the <i>Cipher and Mode</i> column is present in the + list returned by <seealso marker="crypto#supports-1">crypto:supports(ciphers)</seealso>. + </p> + <section> - <title>Block Ciphers</title> - <p>To be used in - <seealso marker="crypto#block_encrypt-3">block_encrypt/3</seealso>, - <seealso marker="crypto#block_encrypt-4">block_encrypt/4</seealso>, - <seealso marker="crypto#block_decrypt-3">block_decrypt/3</seealso> and - <seealso marker="crypto#block_decrypt-4">block_decrypt/4</seealso>. - </p> - <p>Available in all OpenSSL compatible with Erlang CRYPTO if not disabled by configuration. + <title>Ciphers without an IV - cipher_no_iv()</title> + <p>To be used with: </p> - <p>To dynamically check availability, check that the name in the <i>Cipher and Mode</i> column is present in the - list with the <c>cipher</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + <list> + <item><seealso marker="crypto#crypto_one_time/4">crypto_one_time/4</seealso></item> + <item><seealso marker="crypto#crypto_init/3">crypto_init/3</seealso></item> + </list> + <p>The ciphers are: </p> <table> - <row><cell><strong>Cipher and Mode</strong></cell><cell><strong>Key length</strong><br/><strong>[bytes]</strong></cell><cell><strong>IV length</strong><br/><strong>[bytes]</strong></cell><cell><strong>Block size</strong><br/><strong>[bytes]</strong></cell></row> - <row><cell><c>aes_cbc</c></cell> <cell>16, 24, 32</cell><cell>16</cell><cell>16</cell></row> - <row><cell><c>aes_cbc128</c></cell><cell>16</cell><cell>16</cell><cell>16</cell></row> - <row><cell><c>aes_cbc256</c></cell><cell>32</cell><cell>16</cell><cell>16</cell></row> - - <row><cell><c>aes_cfb8</c></cell> <cell>16, 24, 32</cell><cell>16</cell><cell>any</cell></row> - - <row><cell><c>aes_ecb</c></cell><cell>16, 24, 32</cell><cell> </cell><cell>16</cell></row> - - <row><cell><c>aes_ige256</c></cell><cell>16</cell><cell>32</cell><cell>16</cell></row> - <row><cell><c>blowfish_cbc</c></cell> <cell>4-56</cell> <cell>8</cell> <cell>8</cell></row> - <row><cell><c>blowfish_cfb64</c></cell> <cell>≥1</cell> <cell>8</cell> <cell>any</cell></row> - <row><cell><c>blowfish_ecb</c></cell><cell>≥1</cell><cell> </cell><cell>8</cell></row> - <row><cell><c>blowfish_ofb64</c></cell><cell>≥1</cell><cell>8</cell><cell>any</cell></row> - - <row><cell><c>des3_cbc</c><br/><i>(=DES EDE3 CBC)</i></cell><cell>[8,8,8]</cell><cell>8</cell><cell>8</cell></row> - <row><cell><c>des3_cfb</c><br/><i>(=DES EDE3 CFB)</i></cell><cell>[8,8,8]</cell><cell>8</cell><cell>any</cell></row> - - <row><cell><c>des_cbc</c></cell><cell>8</cell><cell>8</cell> <cell>8</cell></row> - <row><cell><c>des_cfb</c></cell><cell>8</cell><cell>8</cell><cell>any</cell></row> - <row><cell><c>des_ecb</c></cell><cell>8</cell><cell> </cell><cell>8</cell></row> - <row><cell><c>des_ede3</c><br/><i>(=DES EDE3 CBC)</i></cell><cell>[8,8,8]</cell><cell>8</cell><cell>8</cell></row> - <row><cell><c>rc2_cbc</c></cell><cell>≥1</cell><cell>8</cell><cell>8</cell></row> - <tcaption>Block cipher key lengths</tcaption> + <row> + <cell><strong>Cipher and Mode</strong></cell> + <cell><strong>Key length</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>Block size</strong><br/><strong>[bytes]</strong></cell> + </row> + <row><cell><c>aes_128_ecb</c></cell> <cell>16</cell> <cell>16</cell></row> + <row><cell><c>aes_192_ecb</c></cell> <cell>24</cell> <cell>16</cell></row> + <row><cell><c>aes_256_ecb</c></cell> <cell>32</cell> <cell>16</cell></row> + <row><cell><c>blowfish_ecb</c></cell> <cell>16</cell> <cell> 8</cell></row> + <row><cell><c>des_ecb</c></cell> <cell> 8</cell> <cell> 8</cell></row> + <row><cell><c>rc4</c></cell> <cell>16</cell> <cell> 1</cell></row> + <tcaption>Ciphers without IV</tcaption> </table> </section> <section> - <title>AEAD Ciphers</title> - <p>To be used in <seealso marker="crypto#block_encrypt-4">block_encrypt/4</seealso> and - <seealso marker="crypto#block_decrypt-4">block_decrypt/4</seealso>. + <title>Ciphers with an IV - cipher_iv()</title> + <p>To be used with: </p> - <p>To dynamically check availability, check that the name in the <i>Cipher and Mode</i> column is present in the - list with the <c>cipher</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + <list> + <item><seealso marker="crypto#crypto_one_time/5">crypto_one_time/5</seealso></item> + <item><seealso marker="crypto#crypto_init/4">crypto_init/4</seealso></item> + <item><seealso marker="crypto#crypto_dyn_iv_init/3">crypto_dyn_iv_init/3</seealso></item> + </list> + <p>The ciphers are: </p> <table> - <row><cell><strong>Cipher and Mode</strong></cell><cell><strong>Key length</strong><br/><strong>[bytes]</strong></cell><cell><strong>IV length</strong><br/><strong>[bytes]</strong></cell><cell><strong>AAD length</strong><br/><strong>[bytes]</strong></cell><cell><strong>Tag length</strong><br/><strong>[bytes]</strong></cell><cell><strong>Block size</strong><br/><strong>[bytes]</strong></cell><cell><strong>Supported with</strong><br/><strong>OpenSSL versions</strong></cell></row> - <row><cell><c>aes_ccm</c></cell> <cell>16,24,32</cell> <cell>7-13</cell> <cell>any</cell> <cell>even 4-16<br/>default: 12</cell> <cell>any</cell><cell>≥1.1.0</cell></row> - <row><cell><c>aes_gcm</c></cell> <cell>16,24,32</cell> <cell>≥1</cell> <cell>any</cell> <cell>1-16<br/>default: 16</cell> <cell>any</cell><cell>≥1.1.0</cell></row> - <row><cell><c>chacha20_poly1305</c></cell><cell>32</cell> <cell>1-16</cell> <cell>any</cell> <cell>16</cell> <cell>any</cell><cell>≥1.1.0</cell></row> - <tcaption>AEAD cipher key lengths</tcaption> + <row> + <cell><strong>Cipher and Mode</strong></cell> + <cell><strong>Key length</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>IV length</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>Block size</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>Limited to</strong><br/><strong>OpenSSL versions</strong></cell> + </row> + <row><cell><c>aes_128_cbc</c></cell> <cell>16</cell> <cell>16</cell> <cell>16</cell> <cell></cell></row> + <row><cell><c>aes_192_cbc</c></cell> <cell>24</cell> <cell>16</cell> <cell>16</cell> <cell></cell></row> + <row><cell><c>aes_256_cbc</c></cell> <cell>32</cell> <cell>16</cell> <cell>16</cell> <cell></cell></row> + <row><cell><c>aes_128_cfb8</c></cell> <cell>16</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_192_cfb8</c></cell> <cell>24</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_256_cfb8</c></cell> <cell>32</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_128_cfb128</c></cell><cell>16</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_192_cfb128</c></cell><cell>24</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_256_cfb128</c></cell><cell>32</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_128_ctr</c></cell> <cell>16</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_192_ctr</c></cell> <cell>24</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_256_ctr</c></cell> <cell>32</cell> <cell>16</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>aes_ige256</c></cell> <cell>16</cell> <cell>32</cell> <cell>16</cell> <cell></cell></row> + <row><cell><c>blowfish_cbc</c></cell> <cell>16</cell> <cell> 8</cell> <cell> 8</cell> <cell></cell></row> + <row><cell><c>blowfish_cfb64</c></cell><cell>16</cell> <cell> 8</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>blowfish_ofb64</c></cell><cell>16</cell> <cell> 8</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>chacha20</c></cell> <cell>32</cell> <cell>16</cell> <cell> 1</cell> <cell>≥1.1.0d</cell></row> + <row><cell><c>des_cbc</c></cell> <cell> 8</cell> <cell> 8</cell> <cell> 8</cell> <cell></cell></row> + <row><cell><c>des_ede3_cbc</c></cell> <cell>24</cell> <cell> 8</cell> <cell> 8</cell> <cell></cell></row> + <row><cell><c>des_cfb</c></cell> <cell> 8</cell> <cell> 8</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>des_ede3_cfb</c></cell> <cell>24</cell> <cell> 8</cell> <cell> 1</cell> <cell></cell></row> + <row><cell><c>rc2_cbc</c></cell> <cell>16</cell> <cell> 8</cell> <cell> 8</cell> <cell></cell></row> + <tcaption>Ciphers with IV</tcaption> </table> </section> <section> - <title>Stream Ciphers</title> - <p>To be used in <seealso marker="crypto#stream_init-2">stream_init/2</seealso> and - <seealso marker="crypto#stream_init/3">stream_init/3</seealso>. + <title>Ciphers with AEAD - cipher_aead()</title> + <p>To be used with: </p> - <p>To dynamically check availability, check that the name in the <i>Cipher and Mode</i> column is present in the - list with the <c>cipher</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + <list> + <item><seealso marker="crypto#crypto_one_time_aead/6">crypto_one_time_aead/6</seealso></item> + <item><seealso marker="crypto#crypto_one_time_aead/7">crypto_one_time_aead/7</seealso></item> + </list> + <p>The ciphers are: </p> <table> - <row><cell><strong>Cipher and Mode</strong></cell><cell><strong>Key length</strong><br/><strong>[bytes]</strong></cell><cell><strong>IV length</strong><br/><strong>[bytes]</strong></cell><cell><strong>Supported with</strong><br/><strong>OpenSSL versions</strong></cell></row> - <row><cell><c>aes_ctr</c></cell><cell>16, 24, 32</cell><cell>16</cell><cell>≥1.0.1</cell></row> - <row><cell><c>rc4</c></cell><cell>≥1</cell><cell> </cell> <cell>all</cell></row> - <tcaption>Stream cipher key lengths</tcaption> + <row> + <cell><strong>Cipher and Mode</strong></cell> + <cell><strong>Key length</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>IV length</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>AAD length</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>Tag length</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>Block size</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>Limited to</strong><br/><strong>OpenSSL versions</strong></cell> + </row> + <row><cell><c>aes_128_ccm</c></cell> <cell>16</cell> <cell>7-13</cell> <cell>any</cell> <cell>even 4-16<br/>default: 12</cell> <cell>any</cell><cell>≥1.0.1</cell></row> + <row><cell><c>aes_192_ccm</c></cell> <cell>24</cell> <cell>7-13</cell> <cell>any</cell> <cell>even 4-16<br/>default: 12</cell> <cell>any</cell><cell>≥1.0.1</cell></row> + <row><cell><c>aes_256_ccm</c></cell> <cell>32</cell> <cell>7-13</cell> <cell>any</cell> <cell>even 4-16<br/>default: 12</cell> <cell>any</cell><cell>≥1.0.1</cell></row> + + <row><cell><c>aes_128_gcm</c></cell> <cell>16</cell> <cell>≥1</cell> <cell>any</cell> <cell>1-16<br/>default: 16</cell> <cell>any</cell><cell>≥1.0.1</cell></row> + <row><cell><c>aes_192_gcm</c></cell> <cell>24</cell> <cell>≥1</cell> <cell>any</cell> <cell>1-16<br/>default: 16</cell> <cell>any</cell><cell>≥1.0.1</cell></row> + <row><cell><c>aes_256_gcm</c></cell> <cell>32</cell> <cell>≥1</cell> <cell>any</cell> <cell>1-16<br/>default: 16</cell> <cell>any</cell><cell>≥1.0.1</cell></row> + + <row><cell><c>chacha20_poly1305</c></cell><cell>32</cell> <cell>1-16</cell> <cell>any</cell> <cell>16</cell> <cell>any</cell><cell>≥1.1.0</cell></row> + <tcaption>AEAD ciphers</tcaption> </table> </section> </section> + <section> <title>Message Authentication Codes (MACs)</title> + <p>To be used in <seealso marker="crypto#mac-4">mac/4</seealso> and + <seealso marker="crypto:new_api#macs--message-authentication-codes-">related functions</seealso>. + </p> <section> <title>CMAC</title> - <p>To be used in <seealso marker="crypto#cmac-3">cmac/3</seealso> and - <seealso marker="crypto#cmac-3">cmac/4</seealso>. - </p> <p>CMAC with the following ciphers are available with OpenSSL 1.0.1 or later if not disabled by configuration. </p> <p>To dynamically check availability, check that the name <c>cmac</c> is present in the - list with the <c>macs</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(macs)</seealso>. Also check that the name in the <i>Cipher and Mode</i> column is present in the - list with the <c>cipher</c> tag in the return value. + list returned by <seealso marker="crypto#supports-1">crypto:supports(ciphers)</seealso>. </p> <table> - <row><cell><strong>Cipher and Mode</strong></cell><cell><strong>Key length</strong><br/><strong>[bytes]</strong></cell><cell><strong>Max Mac Length</strong><br/><strong>[bytes]</strong></cell></row> - <row><cell><c>aes_cbc</c></cell> <cell>16, 24, 32</cell><cell>16</cell></row> - <row><cell><c>aes_cbc128</c></cell><cell>16</cell><cell>16</cell></row> - <row><cell><c>aes_cbc256</c></cell><cell>32</cell><cell>16</cell></row> - - <row><cell><c>aes_cfb8</c></cell> <cell>16</cell><cell>1</cell></row> - - <row><cell><c>blowfish_cbc</c></cell> <cell>4-56</cell> <cell>8</cell></row> - <row><cell><c>blowfish_cfb64</c></cell> <cell>≥1</cell> <cell>1</cell></row> - <row><cell><c>blowfish_ecb</c></cell><cell>≥1</cell> <cell>8</cell></row> - <row><cell><c>blowfish_ofb64</c></cell><cell>≥1</cell> <cell>1</cell></row> - - <row><cell><c>des3_cbc</c><br/><i>(=DES EDE3 CBC)</i></cell><cell>[8,8,8]</cell><cell>8</cell></row> - <row><cell><c>des3_cfb</c><br/><i>(=DES EDE3 CFB)</i></cell><cell>[8,8,8]</cell><cell>1</cell></row> - - <row><cell><c>des_cbc</c></cell><cell>8</cell><cell>8</cell></row> - - <row><cell><c>des_cfb</c></cell><cell>8</cell><cell>1</cell></row> - <row><cell><c>des_ecb</c></cell><cell>8</cell><cell>1</cell></row> - <row><cell><c>rc2_cbc</c></cell><cell>≥1</cell><cell>8</cell></row> + <row> + <cell><strong>Cipher and Mode</strong></cell> + <cell><strong>Key length</strong><br/><strong>[bytes]</strong></cell> + <cell><strong>Max Mac Length</strong><br/><strong>(= default length)</strong><br/><strong>[bytes]</strong></cell> + </row> + <row><cell><c>aes_128_cbc</c></cell> <cell>16</cell> <cell>16</cell></row> + <row><cell><c>aes_192_cbc</c></cell> <cell>24</cell> <cell>16</cell></row> + <row><cell><c>aes_256_cbc</c></cell> <cell>32</cell> <cell>16</cell></row> + <row><cell><c>aes_128_ecb</c></cell> <cell>16</cell> <cell>16</cell></row> + <row><cell><c>aes_192_ecb</c></cell> <cell>24</cell> <cell>16</cell></row> + <row><cell><c>aes_256_ecb</c></cell> <cell>32</cell> <cell>16</cell></row> + <row><cell><c>blowfish_cbc</c></cell> <cell>16</cell> <cell> 8</cell></row> + <row><cell><c>blowfish_ecb</c></cell> <cell>16</cell> <cell> 8</cell></row> + <row><cell><c>des_cbc</c></cell> <cell> 8</cell> <cell> 8</cell></row> + <row><cell><c>des_ecb</c></cell> <cell> 8</cell> <cell> 8</cell></row> + <row><cell><c>des_ede3_cbc</c></cell> <cell>24</cell> <cell> 8</cell></row> + <row><cell><c>rc2_cbc</c></cell> <cell>16</cell> <cell> 8</cell></row> <tcaption>CMAC cipher key lengths</tcaption> </table> </section> @@ -162,9 +203,34 @@ <p>Available in all OpenSSL compatible with Erlang CRYPTO if not disabled by configuration. </p> <p>To dynamically check availability, check that the name <c>hmac</c> is present in the - list with the <c>macs</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(macs)</seealso> and + that the hash name is present in the + list returned by <seealso marker="crypto#supports-1">crypto:supports(hashs)</seealso>. </p> + + <table> + <row> + <cell><strong>Hash</strong></cell> + <cell><strong>Max Mac Length</strong><br/><strong>(= default length)</strong><br/><strong>[bytes]</strong></cell> + </row> + <row><cell><c>sha</c></cell> <cell>20</cell></row> + <row><cell><c>sha224</c></cell> <cell>28</cell></row> + <row><cell><c>sha256</c></cell> <cell>32</cell></row> + <row><cell><c>sha384</c></cell> <cell>48</cell></row> + <row><cell><c>sha512</c></cell> <cell>64</cell></row> + <row><cell><c>sha3_224</c></cell> <cell>28</cell></row> + <row><cell><c>sha3_256</c></cell> <cell>32</cell></row> + <row><cell><c>sha3_384</c></cell> <cell>48</cell></row> + <row><cell><c>sha3_512</c></cell> <cell>64</cell></row> + <row><cell><c>blake2b</c></cell> <cell>64</cell></row> + <row><cell><c>blake2s</c></cell> <cell>32</cell></row> + <row><cell><c>md4</c></cell> <cell>16</cell></row> + <row><cell><c>md5</c></cell> <cell>16</cell></row> + <row><cell><c>ripemd160</c></cell> <cell>20</cell></row> + <tcaption>HMAC output sizes</tcaption> + </table> + + </section> <section> @@ -172,8 +238,9 @@ <p>POLY1305 is available with OpenSSL 1.1.1 or later if not disabled by configuration. </p> <p>To dynamically check availability, check that the name <c>poly1305</c> is present in the - list with the <c>macs</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(macs)</seealso>. + </p> + <p>The poly1305 mac wants an 32 bytes key and produces a 16 byte MAC by default. </p> </section> @@ -183,22 +250,20 @@ <title>Hash</title> <p>To dynamically check availability, check that the wanted name in the <i>Names</i> column is present in the - list with the <c>hashs</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(hashs)</seealso>. </p> - <table> <row><cell><strong>Type</strong></cell> <cell><strong>Names</strong></cell> - <cell><strong>Supported with</strong><br/><strong>OpenSSL versions</strong></cell> + <cell><strong>Limitated to</strong><br/><strong>OpenSSL versions</strong></cell> </row> - <row><cell>SHA1</cell><cell>sha</cell><cell>all</cell></row> - <row><cell>SHA2</cell><cell>sha224, sha256, sha384, sha512</cell><cell>all</cell></row> + <row><cell>SHA1</cell><cell>sha</cell><cell></cell></row> + <row><cell>SHA2</cell><cell>sha224, sha256, sha384, sha512</cell><cell></cell></row> <row><cell>SHA3</cell><cell>sha3_224, sha3_256, sha3_384, sha3_512</cell><cell>≥1.1.1</cell></row> - <row><cell>MD4</cell><cell>md4</cell><cell>all</cell></row> - <row><cell>MD5</cell><cell>md5</cell><cell>all</cell></row> - <row><cell>RIPEMD</cell><cell>ripemd160</cell><cell>all</cell></row> + <row><cell>MD4</cell><cell>md4</cell><cell></cell></row> + <row><cell>MD5</cell><cell>md5</cell><cell></cell></row> + <row><cell>RIPEMD</cell><cell>ripemd160</cell><cell></cell></row> <tcaption></tcaption> </table> </section> @@ -210,8 +275,7 @@ <title>RSA</title> <p>RSA is available with all OpenSSL versions compatible with Erlang CRYPTO if not disabled by configuration. To dynamically check availability, check that the atom <c>rsa</c> is present in the - list with the <c>public_keys</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(public_keys)</seealso>. </p> <warning> <!-- In RefMan rsa_opt(), rsa_sign_verify_opt() and User's man RSA --> @@ -283,8 +347,7 @@ <title>DSS</title> <p>DSS is available with OpenSSL versions compatible with Erlang CRYPTO if not disabled by configuration. To dynamically check availability, check that the atom <c>dss</c> is present in the - list with the <c>public_keys</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(public_keys)</seealso>. </p> </section> @@ -292,13 +355,11 @@ <title>ECDSA</title> <p>ECDSA is available with OpenSSL 0.9.8o or later if not disabled by configuration. To dynamically check availability, check that the atom <c>ecdsa</c> is present in the - list with the <c>public_keys</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. - If the atom <c>ec_gf2m</c> characteristic two field curves are available. + list returned by <seealso marker="crypto#supports-1">crypto:supports(public_keys)</seealso>. + If the atom <c>ec_gf2m</c> also is present, the characteristic two field curves are available. </p> - <p>The actual supported named curves could be checked by examining the list with the - <c>curves</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + <p>The actual supported named curves could be checked by examining the + list returned by <seealso marker="crypto#supports-1">crypto:supports(curves)</seealso>. </p> </section> @@ -306,13 +367,11 @@ <title>EdDSA</title> <p>EdDSA is available with OpenSSL 1.1.1 or later if not disabled by configuration. To dynamically check availability, check that the atom <c>eddsa</c> is present in the - list with the <c>public_keys</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(public_keys)</seealso>. </p> <p>Support for the curves ed25519 and ed448 is implemented. The actual supported named curves could be checked by examining the list with the - <c>curves</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(curves)</seealso>. </p> </section> @@ -321,8 +380,7 @@ <p>Diffie-Hellman computations are available with OpenSSL versions compatible with Erlang CRYPTO if not disabled by configuration. To dynamically check availability, check that the atom <c>dh</c> is present in the - list with the <c>public_keys</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(public_keys)</seealso>. </p> </section> @@ -330,17 +388,15 @@ <title>Elliptic Curve Diffie-Hellman</title> <p>Elliptic Curve Diffie-Hellman is available with OpenSSL 0.9.8o or later if not disabled by configuration. To dynamically check availability, check that the atom <c>ecdh</c> is present in the - list with the <c>public_keys</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + list returned by <seealso marker="crypto#supports-1">crypto:supports(public_keys)</seealso>. </p> <p>The Edward curves <c>x25519</c> and <c>x448</c> are supported with OpenSSL 1.1.1 or later if not disabled by configuration. </p> - <p>The actual supported named curves could be checked by examining the list with the - <c>curves</c> tag in the return value of - <seealso marker="crypto#supports-0">crypto:supports()</seealso>. + <p>The actual supported named curves could be checked by examining the + list returned by <seealso marker="crypto#supports-1">crypto:supports(curves)</seealso>. </p> </section> diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml index d1d1252f29..3973cf3f9f 100644 --- a/lib/crypto/doc/src/crypto.xml +++ b/lib/crypto/doc/src/crypto.xml @@ -302,6 +302,12 @@ </datatype> <datatype> + <name name="cmac_cipher_algorithm"/> + <desc> + </desc> + </datatype> + + <datatype> <name name="rsa_digest_type"/> <desc> </desc> @@ -324,6 +330,11 @@ <name name="sha2"/> <name name="sha3"/> <name name="blake2"/> + <desc> + </desc> + </datatype> + + <datatype> <name name="compatibility_only_hash"/> <desc> <p>The <c>compatibility_only_hash()</c> algorithms are recommended only for compatibility with existing applications.</p> @@ -575,10 +586,11 @@ <datatype_title>Internal data types</datatype_title> <datatype> - <name name="stream_state"/> - <name name="hmac_state"/> - <name name="hash_state"/> <name name="crypto_state"/> + <name name="hash_state"/> + <name name="hmac_state"/> + <name name="mac_state"/> + <name name="stream_state"/> <desc> <p>Contexts with an internal state that should not be manipulated but passed between function calls. </p> @@ -783,6 +795,187 @@ </desc> </func> + <func> + <name name="mac" arity="3" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Short for <seealso marker="#mac-4">mac(Type, undefined, Key, Data)</seealso>. + </p> + </desc> + </func> + + <func> + <name name="mac" arity="4" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Computes a MAC (Message Authentication Code) of type <c>Type</c> from <c>Data</c>. + </p> + + <p><c>SubType</c> depends on the MAC <c>Type</c>: + </p> + <list> + <item>For <c>hmac</c> it is a hash algorithm, see + <seealso marker="algorithm_details#hmac">Algorithm Details</seealso> in the User's Guide. + </item> + <item>For <c>cmac</c> it is a cipher suitable for cmac, see + <seealso marker="algorithm_details#cmac">Algorithm Details</seealso> in the User's Guide. + </item> + <item>For <c>poly1305</c> it should be set to <c>undefined</c> or the + <seealso marker="#mac_init-2">mac/2</seealso> function could be used instead, see + <seealso marker="algorithm_details#poly1305">Algorithm Details</seealso> in the User's Guide. + </item> + </list> + + <p><c>Key</c> is the authentication key with a length according to the + <c>Type</c> and <c>SubType</c>. + The key length could be found with the + <seealso marker="#hash_info-1">hash_info/1</seealso> (<c>hmac</c>) for and + <seealso marker="#cipher_info-1">cipher_info/1</seealso> (<c>cmac</c>) + functions. For <c>poly1305</c> the key length is 32 bytes. Note that + the cryptographic quality of the key is not checked. + </p> + + <p>The <c>Mac</c> result will have a default length depending on the <c>Type</c> and <c>SubType</c>. + To set a shorter length, use <seealso marker="#macN-4">macN/4</seealso> or + <seealso marker="#macN-5">macN/5</seealso> instead. + The default length is documented in + <seealso marker="algorithm_details#message-authentication-codes--macs-">Algorithm Details</seealso> + in the User's Guide. + </p> + </desc> + </func> + + <func> + <name name="macN" arity="4" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Short for <seealso marker="#macN-5">macN(Type, undefined, Key, Data, MacLength)</seealso>. + </p> + </desc> + </func> + + <func> + <name name="macN" arity="5" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Computes a MAC (Message Authentication Code) + as <seealso marker="#mac-3">mac/3</seealso> and <seealso marker="#mac-4">mac/4</seealso> but + <c>MacLength</c> will limit the size of the resultant <c>Mac</c> to + at most <c>MacLength</c> bytes. + Note that if <c>MacLength</c> is greater than the actual number of + bytes returned from the underlying hash, the returned hash will have + that shorter length instead. + </p> + <p>The max <c>MacLength</c> is documented in + <seealso marker="algorithm_details#message-authentication-codes--macs-">Algorithm Details</seealso> + in the User's Guide. + </p> + </desc> + </func> + + <func> + <name name="mac_init" arity="2" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Short for <seealso marker="#mac_init-3">mac_init(Type, undefined, Key)</seealso>. + </p> + </desc> + </func> + + <func> + <name name="mac_init" arity="3" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Initializes the context for streaming MAC operations. + </p> + <p><c>Type</c> determines which mac algorithm to use in the MAC operation. + </p> + + <p><c>SubType</c> depends on the MAC <c>Type</c>: + </p> + <list> + <item>For <c>hmac</c> it is a hash algorithm, see + <seealso marker="algorithm_details#hmac">Algorithm Details</seealso> in the User's Guide. + </item> + <item>For <c>cmac</c> it is a cipher suitable for cmac, see + <seealso marker="algorithm_details#cmac">Algorithm Details</seealso> in the User's Guide. + </item> + <item>For <c>poly1305</c> it should be set to <c>undefined</c> or the + <seealso marker="#mac_init-2">mac/2</seealso> function could be used instead, see + <seealso marker="algorithm_details#poly1305">Algorithm Details</seealso> in the User's Guide. + </item> + </list> + + <p><c>Key</c> is the authentication key with a length according to the + <c>Type</c> and <c>SubType</c>. + The key length could be found with the + <seealso marker="#hash_info-1">hash_info/1</seealso> (<c>hmac</c>) for and + <seealso marker="#cipher_info-1">cipher_info/1</seealso> (<c>cmac</c>) + functions. For <c>poly1305</c> the key length is 32 bytes. Note that + the cryptographic quality of the key is not checked. + </p> + + <p>The returned <c>State</c> should be used in one or more subsequent calls to + <seealso marker="#mac_update-2">mac_update/2</seealso>. + The MAC value is finally returned by calling + <seealso marker="#mac_final-1">mac_final/1</seealso> or + <seealso marker="#mac_finalN-2">mac_finalN/2</seealso>. + </p> + + <p>See <seealso marker="crypto:new_api#example-of-mac_init-mac_update-and-mac_final"> + examples in the User's Guide.</seealso> + </p> + </desc> + </func> + + <func> + <name name="mac_update" arity="2" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Updates the MAC represented by <c>State0</c> using the given <c>Data</c> which + could be of any length. + </p> + <p>The <c>State0</c> is the State value originally from a MAC init function, that is + <seealso marker="#mac_init-2">mac_init/2</seealso>, + <seealso marker="#mac_init-3">mac_init/3</seealso> or + a previous call of <c>mac_update/2</c>. + The value <c>State0</c> is returned unchanged by the function as <c>State</c>. + </p> + </desc> + </func> + + <func> + <name name="mac_final" arity="1" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Finalizes the MAC operation referenced by <c>State</c>. The <c>Mac</c> result will have + a default length depending on the <c>Type</c> and <c>SubType</c> in the + <seealso marker="#mac_init-3">mac_init/2,3</seealso> call. + To set a shorter length, use <seealso marker="#mac_finalN-2">mac_finalN/2</seealso> instead. + The default length is documented in + <seealso marker="algorithm_details#message-authentication-codes--macs-">Algorithm Details</seealso> + in the User's Guide. + </p> + </desc> + </func> + + <func> + <name name="mac_finalN" arity="2" since="OTP @OTP-13872@"/> + <fsummary></fsummary> + <desc> + <p>Finalizes the MAC operation referenced by <c>State</c>. + </p> + <p><c>Mac</c> will be a binary with at most <c>MacLength</c> bytes. + Note that if <c>MacLength</c> is greater than the actual number of + bytes returned from the underlying hash, the returned hash will have + that shorter length instead. + </p> + <p>The max <c>MacLength</c> is documented in + <seealso marker="algorithm_details#message-authentication-codes--macs-">Algorithm Details</seealso> + in the User's Guide. + </p> + </desc> + </func> </funcs> <section> @@ -886,75 +1079,6 @@ </func> <func> - <name name="hmac" arity="3" since="OTP R16B"/> - <name name="hmac" arity="4" since="OTP R16B"/> - <fsummary></fsummary> - <desc> - <p>Computes a HMAC of type <c>Type</c> from <c>Data</c> using - <c>Key</c> as the authentication key.</p> <p><c>MacLength</c> - will limit the size of the resultant <c>Mac</c>.</p> - </desc> - </func> - - <func> - <name name="hmac_init" arity="2" since="OTP R14B03"/> - <fsummary></fsummary> - <desc> - <p>Initializes the context for streaming HMAC operations. <c>Type</c> determines - which hash function to use in the HMAC operation. <c>Key</c> is the authentication - key. The key can be any length.</p> - </desc> - </func> - - <func> - <name name="hmac_update" arity="2" since="OTP R14B03"/> - <fsummary></fsummary> - <desc> - <p>Updates the HMAC represented by <c>Context</c> using the given <c>Data</c>. <c>Context</c> - must have been generated using an HMAC init function (such as - <seealso marker="#hmac_init-2">hmac_init</seealso>). <c>Data</c> can be any length. <c>NewContext</c> - must be passed into the next call to <c>hmac_update</c> - or to one of the functions <seealso marker="#hmac_final-1">hmac_final</seealso> and - <seealso marker="#hmac_final_n-2">hmac_final_n</seealso> - </p> - <warning><p>Do not use a <c>Context</c> as argument in more than one - call to hmac_update or hmac_final. The semantics of reusing old contexts - in any way is undefined and could even crash the VM in earlier releases. - The reason for this limitation is a lack of support in the underlying - libcrypto API.</p></warning> - </desc> - </func> - - <func> - <name name="hmac_final" arity="1" since="OTP R14B03"/> - <fsummary></fsummary> - <desc> - <p>Finalizes the HMAC operation referenced by <c>Context</c>. The size of the resultant MAC is - determined by the type of hash function used to generate it.</p> - </desc> - </func> - - <func> - <name name="hmac_final_n" arity="2" since="OTP R14B03"/> - <fsummary></fsummary> - <desc> - <p>Finalizes the HMAC operation referenced by <c>Context</c>. <c>HashLen</c> must be greater than - zero. <c>Mac</c> will be a binary with at most <c>HashLen</c> bytes. Note that if HashLen is greater than the actual number of bytes returned from the underlying hash, the returned hash will have fewer than <c>HashLen</c> bytes.</p> - </desc> - </func> - - <func> - <name name="cmac" arity="3" since="OTP 20.0"/> - <name name="cmac" arity="4" since="OTP 20.0"/> - <fsummary>Calculates the Cipher-based Message Authentication Code.</fsummary> - <desc> - <p>Computes a CMAC of type <c>Type</c> from <c>Data</c> using - <c>Key</c> as the authentication key.</p> <p><c>MacLength</c> - will limit the size of the resultant <c>Mac</c>.</p> - </desc> - </func> - - <func> <name name="info_fips" arity="0" since="OTP 20.0"/> <fsummary>Provides information about the FIPS operating status.</fsummary> <desc> @@ -1068,15 +1192,6 @@ </func> <func> - <name name="poly1305" arity="2" since="OTP 21.1"/> - <fsummary></fsummary> - <desc> - <p>Computes a POLY1305 message authentication code (<c>Mac</c>) from <c>Data</c> using - <c>Key</c> as the authentication key.</p> - </desc> - </func> - - <func> <name name="private_decrypt" arity="4" since="OTP R16B01"/> <fsummary>Decrypts CipherText using the private Key.</fsummary> <desc> @@ -1901,7 +2016,7 @@ FloatValue = rand:uniform(). % again <seealso marker="#stream_encrypt-2">stream_encrypt</seealso> and <seealso marker="#stream_decrypt-2">stream_decrypt</seealso></p> <p>For keylengths see the - <seealso marker="crypto:algorithm_details#stream-ciphers">User's Guide</seealso>. + <seealso marker="crypto:algorithm_details#ciphers">User's Guide</seealso>. </p> </desc> </func> @@ -1917,7 +2032,7 @@ FloatValue = rand:uniform(). % again <seealso marker="#stream_encrypt-2">stream_encrypt</seealso> and <seealso marker="#stream_decrypt-2">stream_decrypt</seealso>.</p> <p>For keylengths and iv-sizes see the - <seealso marker="crypto:algorithm_details#stream-ciphers">User's Guide</seealso>. + <seealso marker="crypto:algorithm_details#ciphers">User's Guide</seealso>. </p> </desc> </func> @@ -1961,6 +2076,115 @@ FloatValue = rand:uniform(). % again </desc> </func> + <func> + <name name="hmac" arity="3" since="OTP R16B"/> + <name name="hmac" arity="4" since="OTP R16B"/> + <fsummary></fsummary> + <desc> + <dont><p>Don't use this function for new programs! Use + <seealso marker="crypto#mac-4">mac/4</seealso> or + <seealso marker="crypto#macN-5">macN/5</seealso> in + <seealso marker="crypto:new_api">the new api</seealso>.</p> + </dont> + <p>Computes a HMAC of type <c>Type</c> from <c>Data</c> using + <c>Key</c> as the authentication key.</p> <p><c>MacLength</c> + will limit the size of the resultant <c>Mac</c>.</p> + </desc> + </func> + + <func> + <name name="hmac_init" arity="2" since="OTP R14B03"/> + <fsummary></fsummary> + <desc> + <dont><p>Don't use this function for new programs! Use + <seealso marker="crypto#mac_init-3">mac_init/3</seealso> in + <seealso marker="crypto:new_api">the new api</seealso>.</p> + </dont> + <p>Initializes the context for streaming HMAC operations. <c>Type</c> determines + which hash function to use in the HMAC operation. <c>Key</c> is the authentication + key. The key can be any length.</p> + </desc> + </func> + + <func> + <name name="hmac_update" arity="2" since="OTP R14B03"/> + <fsummary></fsummary> + <desc> + <dont><p>Don't use this function for new programs! Use + <seealso marker="crypto#mac_update-2">mac_update/2</seealso> in + <seealso marker="crypto:new_api">the new api</seealso>.</p> + </dont> + <p>Updates the HMAC represented by <c>Context</c> using the given <c>Data</c>. <c>Context</c> + must have been generated using an HMAC init function (such as + <seealso marker="#hmac_init-2">hmac_init</seealso>). <c>Data</c> can be any length. <c>NewContext</c> + must be passed into the next call to <c>hmac_update</c> + or to one of the functions <seealso marker="#hmac_final-1">hmac_final</seealso> and + <seealso marker="#hmac_final_n-2">hmac_final_n</seealso> + </p> + <warning><p>Do not use a <c>Context</c> as argument in more than one + call to hmac_update or hmac_final. The semantics of reusing old contexts + in any way is undefined and could even crash the VM in earlier releases. + The reason for this limitation is a lack of support in the underlying + libcrypto API.</p></warning> + </desc> + </func> + + <func> + <name name="hmac_final" arity="1" since="OTP R14B03"/> + <fsummary></fsummary> + <desc> + <dont><p>Don't use this function for new programs! Use + <seealso marker="crypto#mac_final-1">mac_final/1</seealso> in + <seealso marker="crypto:new_api">the new api</seealso>.</p> + </dont> + <p>Finalizes the HMAC operation referenced by <c>Context</c>. The size of the resultant MAC is + determined by the type of hash function used to generate it.</p> + </desc> + </func> + + <func> + <name name="hmac_final_n" arity="2" since="OTP R14B03"/> + <fsummary></fsummary> + <desc> + <dont><p>Don't use this function for new programs! Use + <seealso marker="crypto#mac_finalN-2">mac_finalN/2</seealso> in + <seealso marker="crypto:new_api">the new api</seealso>.</p> + </dont> + <p>Finalizes the HMAC operation referenced by <c>Context</c>. <c>HashLen</c> must be greater than + zero. <c>Mac</c> will be a binary with at most <c>HashLen</c> bytes. Note that if HashLen is greater than the actual number of bytes returned from the underlying hash, the returned hash will have fewer than <c>HashLen</c> bytes.</p> + </desc> + </func> + + <func> + <name name="cmac" arity="3" since="OTP 20.0"/> + <name name="cmac" arity="4" since="OTP 20.0"/> + <fsummary>Calculates the Cipher-based Message Authentication Code.</fsummary> + <desc> + <dont><p>Don't use this function for new programs! Use + <seealso marker="crypto#mac-4">mac/4</seealso> or + <seealso marker="crypto#macN-5">macN/5</seealso> in + <seealso marker="crypto:new_api">the new api</seealso>.</p> + </dont> + <p>Computes a CMAC of type <c>Type</c> from <c>Data</c> using + <c>Key</c> as the authentication key.</p> <p><c>MacLength</c> + will limit the size of the resultant <c>Mac</c>.</p> + </desc> + </func> + + <func> + <name name="poly1305" arity="2" since="OTP 21.1"/> + <fsummary></fsummary> + <desc> + <dont><p>Don't use this function for new programs! Use + <seealso marker="crypto#mac-3">mac/3</seealso> or + <seealso marker="crypto#macN-4">macN/4</seealso> in + <seealso marker="crypto:new_api">the new api</seealso>.</p> + </dont> + <p>Computes a POLY1305 message authentication code (<c>Mac</c>) from <c>Data</c> using + <c>Key</c> as the authentication key.</p> + </desc> + </func> + </funcs> diff --git a/lib/crypto/doc/src/new_api.xml b/lib/crypto/doc/src/new_api.xml index bd2334ac9f..aacf5e4f76 100644 --- a/lib/crypto/doc/src/new_api.xml +++ b/lib/crypto/doc/src/new_api.xml @@ -40,7 +40,7 @@ to maintain. </p> <p>It turned out that using the old api in the new way (more about that later), and still keep it - backwards compatible, was not possible. Specially as more precision in the error messages was wanted + backwards compatible, was not possible. Specially as more precision in the error messages is desired it could not be combined with the old standard. </p> <p>Therefore the old api (see next section) is kept for now but internally implemented with new primitives. @@ -49,7 +49,7 @@ <section> <title>The old API</title> - <p>The old functions - not recommended for new programs - are:</p> + <p>The old functions - not recommended for new programs - are for chipers:</p> <list> <item><seealso marker="crypto#block_encrypt-3">block_encrypt/3</seealso></item> <item><seealso marker="crypto#block_encrypt-4">block_encrypt/4</seealso></item> @@ -59,61 +59,101 @@ <item><seealso marker="crypto#stream_init-2">stream_init/3</seealso></item> <item><seealso marker="crypto#stream_encrypt-2">stream_encrypt/2</seealso></item> <item><seealso marker="crypto#stream_decrypt-2">stream_decrypt/2</seealso></item> + </list> + <p>for lists of supported algorithms:</p> + <list> <item><seealso marker="crypto#supports-0">supports/0</seealso></item> </list> + <p>and for MACs (Message Authentication Codes):</p> + <list> + <item><seealso marker="crypto#cmac-3">cmac/3</seealso></item> + <item><seealso marker="crypto#cmac-4">cmac/4</seealso></item> + <item><seealso marker="crypto#hmac-3">hmac/3</seealso></item> + <item><seealso marker="crypto#hmac-4">hmac/4</seealso></item> + <item><seealso marker="crypto#hmac_init-2">hmac_init/2</seealso></item> + <item><seealso marker="crypto#hmac_update-2">hmac_update/2</seealso></item> + <item><seealso marker="crypto#hmac_final-1">hmac_final/1</seealso></item> + <item><seealso marker="crypto#hmac_final_n-2">hmac_final_n/2</seealso></item> + <item><seealso marker="crypto#poly1305-2">poly1305/2</seealso></item> + </list> <p>They are not deprecated for now, but may be in a future release. </p> </section> <section> <title>The new API</title> - <p>The new functions for encrypting or decrypting one single binary are: - </p> - <list> - <item><seealso marker="crypto#crypto_one_time/4">crypto_one_time/4</seealso></item> - <item><seealso marker="crypto#crypto_one_time/5">crypto_one_time/5</seealso></item> - <item><seealso marker="crypto#crypto_one_time_aead/6">crypto_one_time_aead/6</seealso></item> - <item><seealso marker="crypto#crypto_one_time_aead/7">crypto_one_time_aead/7</seealso></item> - </list> - <p>In those functions the internal crypto state is first created and initialized - with the cipher type, the key and possibly other data. Then the single binary is encrypted - or decrypted, - the crypto state is de-allocated and the result of the crypto operation is returned. - </p> - <p>The <c>crypto_one_time_aead</c> functions are for the ciphers of mode <c>ccm</c> or - <c>gcm</c>, and for the cipher <c>chacha20-poly1305</c>. - </p> - <p>For repeated encryption or decryption of a text divided in parts, where the internal - crypto state is initialized once, and then many binaries are encrypted or decrypted with - the same state, the functions are: - </p> - <list> - <item><seealso marker="crypto#crypto_init/4">crypto_init/4</seealso></item> - <item><seealso marker="crypto#crypto_init/3">crypto_init/3</seealso></item> - <item><seealso marker="crypto#crypto_update/2">crypto_update/2</seealso></item> - </list> - <p>The <c>crypto_init</c> initialies an internal cipher state, and one or more calls of - <c>crypto_update</c> does the acual encryption or decryption. Note that AEAD ciphers - can't be handled this way due to their nature. - </p> - <p>For repeated encryption or decryption of a text divided in parts where the - same cipher and same key is used, but a new initialization vector (nounce) should be applied - for each part, the functions are: - </p> - <list> - <item><seealso marker="crypto#crypto_dyn_iv_init/3">crypto_dyn_iv_init/3</seealso></item> - <item><seealso marker="crypto#crypto_dyn_iv_update/3">crypto_dyn_iv_update/3</seealso></item> - </list> - <p>An example of where those functions are needed, is when handling the TLS protocol.</p> - <p>For information about available algorithms, use: - </p> - <list> - <item><seealso marker="crypto#supports-1">supports/1</seealso></item> - <item><seealso marker="crypto#hash_info-1">hash_info/1</seealso></item> - <item><seealso marker="crypto#cipher_info-1">cipher_info/1</seealso></item> - </list> + <section> + <title>Encryption and decryption</title> + <p>The new functions for encrypting or decrypting one single binary are: + </p> + <list> + <item><seealso marker="crypto#crypto_one_time/4">crypto_one_time/4</seealso></item> + <item><seealso marker="crypto#crypto_one_time/5">crypto_one_time/5</seealso></item> + <item><seealso marker="crypto#crypto_one_time_aead/6">crypto_one_time_aead/6</seealso></item> + <item><seealso marker="crypto#crypto_one_time_aead/7">crypto_one_time_aead/7</seealso></item> + </list> + <p>In those functions the internal crypto state is first created and initialized + with the cipher type, the key and possibly other data. Then the single binary is encrypted + or decrypted, + the crypto state is de-allocated and the result of the crypto operation is returned. + </p> + <p>The <c>crypto_one_time_aead</c> functions are for the ciphers of mode <c>ccm</c> or + <c>gcm</c>, and for the cipher <c>chacha20-poly1305</c>. + </p> + <p>For repeated encryption or decryption of a text divided in parts, where the internal + crypto state is initialized once, and then many binaries are encrypted or decrypted with + the same state, the functions are: + </p> + <list> + <item><seealso marker="crypto#crypto_init/4">crypto_init/4</seealso></item> + <item><seealso marker="crypto#crypto_init/3">crypto_init/3</seealso></item> + <item><seealso marker="crypto#crypto_update/2">crypto_update/2</seealso></item> + </list> + <p>The <c>crypto_init</c> initialies an internal cipher state, and one or more calls of + <c>crypto_update</c> does the acual encryption or decryption. Note that AEAD ciphers + can't be handled this way due to their nature. + </p> + <p>For repeated encryption or decryption of a text divided in parts where the + same cipher and same key is used, but a new initialization vector (nounce) should be applied + for each part, the functions are: + </p> + <list> + <item><seealso marker="crypto#crypto_dyn_iv_init/3">crypto_dyn_iv_init/3</seealso></item> + <item><seealso marker="crypto#crypto_dyn_iv_update/3">crypto_dyn_iv_update/3</seealso></item> + </list> + <p>An example of where those functions are needed, is when handling the TLS protocol.</p> + <p>For information about available algorithms, use: + </p> + <list> + <item><seealso marker="crypto#supports-1">supports/1</seealso></item> + <item><seealso marker="crypto#hash_info-1">hash_info/1</seealso></item> + <item><seealso marker="crypto#cipher_info-1">cipher_info/1</seealso></item> + </list> + </section> <section> + <title>MACs (Message Authentication Codes)</title> + <p>The new functions for calculating a MAC of a single piece of text are:</p> + <list> + <item><seealso marker="crypto#mac-3">mac/3</seealso></item> + <item><seealso marker="crypto#mac-4">mac/4</seealso></item> + <item><seealso marker="crypto#macN-4">macN/4</seealso></item> + <item><seealso marker="crypto#macN-5">macN/5</seealso></item> + </list> + <p>For calculating a MAC of a text divided in parts use:</p> + <list> + <item><seealso marker="crypto#mac_init-2">mac_init/2</seealso></item> + <item><seealso marker="crypto#mac_init-3">mac_init/3</seealso></item> + <item><seealso marker="crypto#mac_update-2">mac_update/2</seealso></item> + <item><seealso marker="crypto#mac_final-1">mac_final/1</seealso></item> + <item><seealso marker="crypto#mac_finalN-2">mac_finalN/2</seealso></item> + </list> + </section> + </section> + + <section> + <title>Examples of the new api</title> + <section> <title>Examples of crypto_init/4 and crypto_update/2</title> <p>The functions <seealso marker="crypto#crypto_init/4">crypto_init/4</seealso> and <seealso marker="crypto#crypto_update/2">crypto_update/2</seealso> are intended @@ -143,7 +183,7 @@ 8> crypto:crypto_update(StateDec, <<67,44,216,166,25,130,203>>). <<"First b">> 9> crypto:crypto_update(StateDec, <<5,66,6,162,16,79,94,115,234,197, - 94,253,16,144,151>>). + 94,253,16,144,151>>). <<"ytesSecond byte">> 10> crypto:crypto_update(StateDec, <<41>>). <<"s">> @@ -159,16 +199,16 @@ </p> <code type="erl"> encode(Crypto, Key, IV) -> - crypto_loop(crypto:crypto_init(Crypto, Key, IV, true)). + crypto_loop(crypto:crypto_init(Crypto, Key, IV, true)). crypto_loop(State) -> - receive - {Text, Requester} -> - Requester ! crypto:crypto_update(State, Text), - loop(State) - end. + receive + {Text, Requester} -> + Requester ! crypto:crypto_update(State, Text), + loop(State) + end. </code> - </section> + </section> <section> <title>Example of crypto_one_time/5</title> @@ -219,6 +259,35 @@ </p> </section> + <section> + <title>Example of mac_init mac_update and mac_final</title> + <code> + 1> Key = <<1:128>>. + <<0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1>> + 2> StateMac = crypto:mac_init(cmac, aes_128_cbc, Key). + #Ref<0.2424664121.2781478916.232610> + 3> crypto:mac_update(StateMac, <<"First bytes">>). + #Ref<0.2424664121.2781478916.232610> + 4> crypto:mac_update(StateMac, " "). + #Ref<0.2424664121.2781478916.232610> + 5> crypto:mac_update(StateMac, <<"last bytes">>). + #Ref<0.2424664121.2781478916.232610> + 6> crypto:mac_final(StateMac). + <<68,191,219,128,84,77,11,193,197,238,107,6,214,141,160, + 249>> + 7> + </code> + <p>and compare the result with a single calculation just for this example:</p> + <code> + 7> crypto:mac(cmac, aes_128_cbc, Key, "First bytes last bytes"). + <<68,191,219,128,84,77,11,193,197,238,107,6,214,141,160, + 249>> + 8> v(7) == v(6). + true + 9> + </code> + </section> + </section> <section> @@ -233,7 +302,7 @@ on the mode. An example is the ccm mode which has a variant called ccm8 where the so called tag has a length of eight bits. </p> - <p>The old names had by time lost any common naming which the new names now introduces. The new names include + <p>The old names had by time lost any common naming convention which the new names now introduces. The new names include the key length which improves the error checking in the lower levels of the crypto application. </p> diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index 8ffdde2b90..965697578d 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -28,9 +28,6 @@ -export([hash/2, hash_init/1, hash_update/2, hash_final/1]). -export([sign/4, sign/5, verify/5, verify/6]). -export([generate_key/2, generate_key/3, compute_key/4]). --export([hmac/3, hmac/4, hmac_init/2, hmac_update/2, hmac_final/1, hmac_final_n/2]). --export([cmac/3, cmac/4]). --export([poly1305/2]). -export([exor/2, strong_rand_bytes/1, mod_pow/3]). -export([rand_seed/0, rand_seed_alg/1, rand_seed_alg/2]). -export([rand_seed_s/0, rand_seed_alg_s/1, rand_seed_alg_s/2]). @@ -48,6 +45,9 @@ -export([rand_seed/1]). %% Old interface. Now implemented with the New interface +-export([hmac/3, hmac/4, hmac_init/2, hmac_update/2, hmac_final/1, hmac_final_n/2]). +-export([cmac/3, cmac/4]). +-export([poly1305/2]). -export([stream_init/2, stream_init/3, stream_encrypt/2, stream_decrypt/2, @@ -62,7 +62,9 @@ crypto_one_time_aead/6, crypto_one_time_aead/7, crypto_dyn_iv_init/3, crypto_dyn_iv_update/3, - supports/1 + supports/1, + mac/3, mac/4, macN/4, macN/5, + mac_init/2, mac_init/3, mac_update/2, mac_final/1, mac_finalN/2 ]). @@ -109,9 +111,10 @@ stream_state/0, hmac_state/0, hash_state/0, - crypto_state/0 + crypto_state/0, + mac_state/0 ]). - + %% Private. For tests. -export([packed_openssl_version/4, engine_methods_convert_to_bitmask/2, get_test_engine/0]). @@ -136,7 +139,7 @@ -type rsa_private() :: [key_integer()] . % [E, N, D] | [E, N, D, P1, P2, E1, E2, C] -type rsa_params() :: {ModulusSizeInBits::integer(), PublicExponent::key_integer()} . --type dss_public() :: [key_integer()] . % [P, Q, G, Y] +-type dss_public() :: [key_integer()] . % [P, Q, G, Y] -type dss_private() :: [key_integer()] . % [P, Q, G, X] -type ecdsa_public() :: key_integer() . @@ -282,7 +285,7 @@ %%% New cipher schema %%% -type cipher() :: cipher_no_iv() - | cipher_iv() + | cipher_iv() | cipher_aead() . -type cipher_no_iv() :: aes_128_ecb @@ -326,7 +329,7 @@ -type cipher_aead() :: aes_128_ccm | aes_192_ccm | aes_256_ccm - + | aes_128_gcm | aes_192_gcm | aes_256_gcm @@ -334,23 +337,6 @@ | chacha20_poly1305 . -%% -type retired_cipher_no_iv_aliases() :: aes_ecb . - -%% -type retired_cipher_iv_aliases() :: aes_cbc -%% | aes_cbc128 % aes_128_cbc -%% | aes_cbc256 % aes_256_cbc -%% | aes_cfb128 -%% | aes_cfb8 -%% | aes_ctr -%% | des3_cbc % des_ede3_cbc -%% | des_ede3 % des_ede3_cbc -%% | des_ede3_cbf % des_ede3_cfb -%% | des3_cbf % des_ede3_cfb -%% | des3_cfb . % des_ede3_cfb - -%% -type retired_cipher_aead_aliases() :: aes_ccm -%% | aes_gcm . - %%%---------------------------------------------------------------- %%% Old cipher scheme %%% @@ -365,7 +351,7 @@ -type stream_cipher() :: ctr_cipher() | chacha20 | rc4 . - + %%%---- -type cbc_cipher() :: aes_128_cbc @@ -374,7 +360,7 @@ | blowfish_cbc | des_cbc | des_ede3_cbc - | rc2_cbc + | rc2_cbc | retired_cbc_cipher_aliases() . -type retired_cbc_cipher_aliases() :: aes_cbc % aes_*_cbc @@ -382,7 +368,7 @@ | aes_cbc256 % aes_256_cbc | des3_cbc % des_ede3_cbc | des_ede3 . % des_ede3_cbc - + %%%---- -type cfb_cipher() :: aes_128_cfb128 | aes_192_cfb128 @@ -398,7 +384,7 @@ -type retired_cfb_cipher_aliases() :: aes_cfb8 % aes_*_cfb8 | aes_cfb128 % aes_*_cfb128 | des3_cbf % des_ede3_cfb, cfb misspelled - | des3_cfb % des_ede3_cfb + | des3_cfb % des_ede3_cfb | des_ede3_cbf .% cfb misspelled @@ -457,6 +443,19 @@ %%-------------------------------------------------------------------- +%% +%% Make the new descriptive_error() look like the old run_time_error() +%% +-define(COMPAT(CALL), + try begin CALL end + catch + error:{error, {_File,_Line}, _Reason} -> + error(badarg); + error:{E, {_File,_Line}, _Reason} when E==notsup ; E==badarg -> + error(E) + end). + +%%-------------------------------------------------------------------- -compile(no_native). -on_load(on_load/0). -define(CRYPTO_NIF_VSN,302). @@ -580,7 +579,7 @@ hash(Type, Data) -> -spec hash_init(Type) -> State when Type :: hash_algorithm(), State :: hash_state(). -hash_init(Type) -> +hash_init(Type) -> notsup_to_error(hash_init_nif(Type)). -spec hash_update(State, Data) -> NewState when State :: hash_state(), @@ -599,25 +598,139 @@ hash_final(Context) -> %%%================================================================ %%% %%% MACs (Message Authentication Codes) -%%% +%%% %%%================================================================ -%%%---- HMAC - -type hmac_hash_algorithm() :: sha1() | sha2() | sha3() | compatibility_only_hash(). -%%%---- hmac/3,4 +-type cmac_cipher_algorithm() :: aes_128_cbc | aes_192_cbc | aes_256_cbc | blowfish_cbc + | des_cbc | des_ede3_cbc | rc2_cbc + | aes_128_cfb128 | aes_192_cfb128 | aes_256_cfb128 + | aes_128_cfb8 | aes_192_cfb8 | aes_256_cfb8 + . + +%%%---------------------------------------------------------------- +%%% Calculate MAC for the whole text at once + +-spec mac(Type :: poly1305, Key, Data) -> Mac | descriptive_error() + when Key :: iodata(), + Data :: iodata(), + Mac :: binary(). + +mac(poly1305, Key, Data) -> mac(poly1305, undefined, Key, Data). + + +-spec mac(Type, SubType, Key, Data) -> Mac | descriptive_error() + when Type :: hmac | cmac | poly1305, + SubType :: hmac_hash_algorithm() | cmac_cipher_algorithm() | undefined, + Key :: iodata(), + Data :: iodata(), + Mac :: binary(). + +mac(Type, SubType, Key, Data) -> mac_nif(Type, SubType, Key, Data). + + + +-spec macN(Type :: poly1305, Key, Data, MacLength) -> Mac | descriptive_error() + when Key :: iodata(), + Data :: iodata(), + Mac :: binary(), + MacLength :: pos_integer(). + +macN(Type, Key, Data, MacLength) -> + macN(Type, undefined, Key, Data, MacLength). + + +-spec macN(Type, SubType, Key, Data, MacLength) -> Mac | descriptive_error() + when Type :: hmac | cmac | poly1305, + SubType :: hmac_hash_algorithm() | cmac_cipher_algorithm() | undefined, + Key :: iodata(), + Data :: iodata(), + Mac :: binary(), + MacLength :: pos_integer(). + +macN(Type, SubType, Key, Data, MacLength) -> + erlang:binary_part(mac(Type,SubType,Key,Data), 0, MacLength). + + +%%%---------------------------------------------------------------- +%%% Calculate the MAC by uppdating by pieces of the text + +-opaque mac_state() :: reference() . + +-spec mac_init(Type :: poly1305, Key) -> State | descriptive_error() + when Key :: iodata(), + State :: mac_state() . +mac_init(poly1305, Key) -> + mac_init_nif(poly1305, undefined, Key). + + +-spec mac_init(Type, SubType, Key) -> State | descriptive_error() + when Type :: hmac | cmac | poly1305, + SubType :: hmac_hash_algorithm() | cmac_cipher_algorithm() | undefined, + Key :: iodata(), + State :: mac_state() . +mac_init(Type, SubType, Key) -> + mac_init_nif(Type, SubType, Key). + + +-spec mac_update(State0, Data) -> State | descriptive_error() + when Data :: iodata(), + State0 :: mac_state(), + State :: mac_state(). +mac_update(Ref, Data) -> + mac_update_nif(Ref, Data). + + + +-spec mac_final(State) -> Mac | descriptive_error() + when State :: mac_state(), + Mac :: binary(). +mac_final(Ref) -> + mac_final_nif(Ref). + + +-spec mac_finalN(State, MacLength) -> Mac | descriptive_error() + when State :: mac_state(), + MacLength :: pos_integer(), + Mac :: binary(). +mac_finalN(Ref, MacLength) -> + erlang:binary_part(mac_final(Ref), 0, MacLength). + --spec hmac(Type, Key, Data) -> +%%%---------------------------------------------------------------- +%%% NIFs for the functions above + +mac_nif(_Type, _SubType, _Key, _Data) -> ?nif_stub. + +mac_init_nif(_Type, _SubType, _Key) -> ?nif_stub. +mac_update_nif(_Ref, _Data) -> ?nif_stub. +mac_final_nif(_Ref) -> ?nif_stub. + +%%%================================================================ +%%% +%%% The "Old API", kept for compatibility +%%% +%%%================================================================ + +%%%---------------------------------------------------------------- +%%%---------------------------------------------------------------- +%%% Message Authentication Codes, MAC +%%% + +%%%---- HMAC + +%%%---- hmac/3,4 + +-spec hmac(Type, Key, Data) -> Mac when Type :: hmac_hash_algorithm(), Key :: iodata(), Data :: iodata(), Mac :: binary() . hmac(Type, Key, Data) -> - Data1 = iolist_to_binary(Data), - hmac(Type, Key, Data1, undefined, erlang:byte_size(Data1), max_bytes()). + ?COMPAT(mac(hmac, Type, Key, Data)). --spec hmac(Type, Key, Data, MacLength) -> +-spec hmac(Type, Key, Data, MacLength) -> Mac when Type :: hmac_hash_algorithm(), Key :: iodata(), Data :: iodata(), @@ -625,45 +738,43 @@ hmac(Type, Key, Data) -> Mac :: binary() . hmac(Type, Key, Data, MacLength) -> - Data1 = iolist_to_binary(Data), - hmac(Type, Key, Data1, MacLength, erlang:byte_size(Data1), max_bytes()). + ?COMPAT(macN(hmac, Type, Key, Data, MacLength)). %%%---- hmac_init, hamc_update, hmac_final --opaque hmac_state() :: binary(). +-opaque hmac_state() :: mac_state(). % Was: binary(). -spec hmac_init(Type, Key) -> State when Type :: hmac_hash_algorithm(), Key :: iodata(), State :: hmac_state() . hmac_init(Type, Key) -> - notsup_to_error(hmac_init_nif(Type, Key)). + ?COMPAT(mac_init(hmac, Type, Key)). %%%---- hmac_update -spec hmac_update(State, Data) -> NewState when Data :: iodata(), State :: hmac_state(), NewState :: hmac_state(). -hmac_update(State, Data0) -> - Data = iolist_to_binary(Data0), - hmac_update(State, Data, erlang:byte_size(Data), max_bytes()). +hmac_update(State, Data) -> + ?COMPAT(mac_update(State, Data)). %%%---- hmac_final -spec hmac_final(State) -> Mac when State :: hmac_state(), Mac :: binary(). hmac_final(Context) -> - notsup_to_error(hmac_final_nif(Context)). + ?COMPAT(mac_final(Context)). -spec hmac_final_n(State, HashLen) -> Mac when State :: hmac_state(), HashLen :: integer(), Mac :: binary(). hmac_final_n(Context, HashLen) -> - notsup_to_error(hmac_final_nif(Context, HashLen)). + ?COMPAT(mac_finalN(Context, HashLen)). %%%---- CMAC --define(CMAC_CIPHER_ALGORITHM, cbc_cipher() | cfb_cipher() | blowfish_cbc | des_ede3 | rc2_cbc ). +-define(CMAC_CIPHER_ALGORITHM, cbc_cipher() | cfb_cipher() | blowfish_cbc | des_ede3 | rc2_cbc ). -spec cmac(Type, Key, Data) -> Mac when Type :: ?CMAC_CIPHER_ALGORITHM, @@ -671,42 +782,31 @@ hmac_final_n(Context, HashLen) -> Data :: iodata(), Mac :: binary(). cmac(Type, Key, Data) -> - notsup_to_error(cmac_nif(alias(Type), Key, Data)). + ?COMPAT(mac(cmac, alias(Type), Key, Data)). -spec cmac(Type, Key, Data, MacLength) -> Mac when Type :: ?CMAC_CIPHER_ALGORITHM, Key :: iodata(), Data :: iodata(), - MacLength :: integer(), + MacLength :: integer(), Mac :: binary(). cmac(Type, Key, Data, MacLength) -> - erlang:binary_part(cmac(alias(Type), Key, Data), 0, MacLength). + ?COMPAT(macN(cmac, alias(Type), Key, Data, MacLength)). %%%---- POLY1305 -spec poly1305(iodata(), iodata()) -> Mac when Mac :: binary(). poly1305(Key, Data) -> - poly1305_nif(Key, Data). + ?COMPAT(mac(poly1305, Key, Data)). -%%%================================================================ -%%% -%%% Encrypt/decrypt, The "Old API" -%%% -%%%================================================================ +%%%---------------------------------------------------------------- +%%%---------------------------------------------------------------- +%%% Ciphers --define(COMPAT(CALL), - try begin CALL end - catch - error:{error, {_File,_Line}, _Reason} -> - error(badarg); - error:{E, {_File,_Line}, _Reason} when E==notsup ; E==badarg -> - error(E) - end). %%%---- Cipher info -%%%---------------------------------------------------------------- -spec cipher_info(Type) -> Result | run_time_error() when Type :: cipher(), Result :: #{key_length := integer(), @@ -845,7 +945,7 @@ block_decrypt(Type, Key0, CryptoText) -> Key :: iodata(), IVec ::binary(), State :: stream_state() . -stream_init(Type, Key0, IVec) when is_binary(IVec) -> +stream_init(Type, Key0, IVec) when is_binary(IVec) -> Key = iolist_to_binary(Key0), Ref = ?COMPAT(ng_crypto_init_nif(alias(Type,Key), Key, iolist_to_binary(IVec), @@ -933,7 +1033,7 @@ next_iv(Type, Data, _Ivec) -> %%%---------------------------------------------------------------- %%% %%% Create and initialize a new state for encryption or decryption -%%% +%%% -spec crypto_init(Cipher, Key, EncryptFlag) -> State | descriptive_error() when Cipher :: cipher_no_iv(), @@ -971,12 +1071,12 @@ crypto_dyn_iv_init(Cipher, Key, EncryptFlag) -> %%% Encrypt/decrypt a sequence of bytes. The sum of the sizes %%% of all blocks must be an integer multiple of the crypto's %%% blocksize. -%%% +%%% -spec crypto_update(State, Data) -> Result | descriptive_error() - when State :: crypto_state(), - Data :: iodata(), - Result :: binary() . + when State :: crypto_state(), + Data :: iodata(), + Result :: binary() . crypto_update(State, Data0) -> case iolist_to_binary(Data0) of <<>> -> @@ -1005,7 +1105,7 @@ crypto_dyn_iv_update(State, Data0, IV) -> %%% %%% Encrypt/decrypt one set bytes. %%% The size must be an integer multiple of the crypto's blocksize. -%%% +%%% -spec crypto_one_time(Cipher, Key, Data, EncryptFlag) -> Result | descriptive_error() @@ -1015,8 +1115,15 @@ crypto_dyn_iv_update(State, Data0, IV) -> EncryptFlag :: boolean(), Result :: binary() . -crypto_one_time(Cipher, Key, Data, EncryptFlag) -> - crypto_one_time(Cipher, Key, <<>>, Data, EncryptFlag). +crypto_one_time(Cipher, Key, Data0, EncryptFlag) -> + case iolist_to_binary(Data0) of + <<>> -> + <<>>; % Known to fail on OpenSSL 0.9.8h + Data -> + ng_crypto_one_time_nif(Cipher, + iolist_to_binary(Key), <<>>, Data, + EncryptFlag) + end. -spec crypto_one_time(Cipher, Key, IV, Data, EncryptFlag) -> Result | descriptive_error() @@ -1121,7 +1228,7 @@ ng_crypto_one_time_nif(_Cipher, _Key, _IVec, _Data, _EncryptFlg) -> ?nif_stub. false -> Ciphers end). - + prepend_old_aliases(L0) -> L1 = ?if_also(des_ede3_cbc, L0, @@ -1465,7 +1572,7 @@ rand_seed_nif(_Seed) -> ?nif_stub. %%% Sign -spec sign(Algorithm, DigestType, Msg, Key) - -> Signature + -> Signature when Algorithm :: pk_sign_verify_algs(), DigestType :: rsa_digest_type() | dss_digest_type() @@ -1483,7 +1590,7 @@ sign(Algorithm, Type, Data, Key) -> -spec sign(Algorithm, DigestType, Msg, Key, Options) - -> Signature + -> Signature when Algorithm :: pk_sign_verify_algs(), DigestType :: rsa_digest_type() | dss_digest_type() @@ -1580,7 +1687,7 @@ sign_verify_compatibility(Algorithm0, Type0, _Digest) -> | rsa_x931_padding | rsa_no_padding. --type rsa_opt() :: {rsa_padding, rsa_padding()} +-type rsa_opt() :: {rsa_padding, rsa_padding()} | {signature_md, atom()} | {rsa_mgf1_md, sha} | {rsa_oaep_label, binary()} @@ -1653,7 +1760,7 @@ pkey_crypt_nif(_Algorithm, _In, _Key, _Options, _IsPrivate, _IsEncrypt) -> ?nif_ %%%================================================================ -spec generate_key(Type, Params) - -> {PublicKey, PrivKeyOut} + -> {PublicKey, PrivKeyOut} when Type :: dh | ecdh | rsa | srp, PublicKey :: dh_public() | ecdh_public() | rsa_public() | srp_public(), PrivKeyOut :: dh_private() | ecdh_private() | rsa_private() | {srp_public(),srp_private()}, @@ -1663,7 +1770,7 @@ generate_key(Type, Params) -> generate_key(Type, Params, undefined). -spec generate_key(Type, Params, PrivKeyIn) - -> {PublicKey, PrivKeyOut} + -> {PublicKey, PrivKeyOut} when Type :: dh | ecdh | rsa | srp, PublicKey :: dh_public() | ecdh_public() | rsa_public() | srp_public(), PrivKeyIn :: undefined | dh_private() | ecdh_private() | rsa_private() | {srp_public(),srp_private()}, @@ -1814,7 +1921,7 @@ mod_pow(Base, Exponent, Prime) -> %%%====================================================================== %%% %%% Engine functions -%%% +%%% %%%====================================================================== %%%---- Refering to keys stored in an engine: @@ -2121,7 +2228,7 @@ ensure_engine_unloaded(Engine) -> %%---------------------------------------------------------------------- %% Function: ensure_engine_unloaded/2 %%---------------------------------------------------------------------- --spec ensure_engine_unloaded(Engine, EngineMethods) -> +-spec ensure_engine_unloaded(Engine, EngineMethods) -> Result when Engine :: engine_ref(), EngineMethods :: [engine_method_type()], Result :: ok | {error, Reason::term()}. @@ -2203,7 +2310,7 @@ path2bin(Path) when is_list(Path) -> %%%================================================================ %%% %%% Internal functions -%%% +%%% %%%================================================================ max_bytes() -> @@ -2235,43 +2342,6 @@ hash_init_nif(_Hash) -> ?nif_stub. hash_update_nif(_State, _Data) -> ?nif_stub. hash_final_nif(_State) -> ?nif_stub. -%% HMAC -------------------------------------------------------------------- - -hmac(Type, Key, Data, MacSize, Size, MaxBytes) when Size =< MaxBytes -> - notsup_to_error( - case MacSize of - undefined -> hmac_nif(Type, Key, Data); - _ -> hmac_nif(Type, Key, Data, MacSize) - end); -hmac(Type, Key, Data, MacSize, Size, MaxBytes) -> - State0 = hmac_init(Type, Key), - State1 = hmac_update(State0, Data, Size, MaxBytes), - case MacSize of - undefined -> hmac_final(State1); - _ -> hmac_final_n(State1, MacSize) - end. - -hmac_update(State, Data, Size, MaxBytes) when Size =< MaxBytes -> - notsup_to_error(hmac_update_nif(State, Data)); -hmac_update(State0, Data, _, MaxBytes) -> - <<Increment:MaxBytes/binary, Rest/binary>> = Data, - State = notsup_to_error(hmac_update_nif(State0, Increment)), - hmac_update(State, Rest, erlang:byte_size(Rest), MaxBytes). - -hmac_nif(_Type, _Key, _Data) -> ?nif_stub. -hmac_nif(_Type, _Key, _Data, _MacSize) -> ?nif_stub. -hmac_init_nif(_Type, _Key) -> ?nif_stub. -hmac_update_nif(_Context, _Data) -> ?nif_stub. -hmac_final_nif(_Context) -> ?nif_stub. -hmac_final_nif(_Context, _MacSize) -> ?nif_stub. - -%% CMAC -cmac_nif(_Type, _Key, _Data) -> ?nif_stub. - -%% POLY1305 -poly1305_nif(_Key, _Data) -> ?nif_stub. - - %% CIPHERS -------------------------------------------------------------------- cipher_info_nif(_Type) -> ?nif_stub. diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index 56691223c4..0da70d5592 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -176,19 +176,19 @@ groups() -> ]}, {md4, [], [hash]}, - {md5, [], [hash, hmac]}, + {md5, [], [hash, hmac, hmac_update]}, {ripemd160, [], [hash]}, - {sha, [], [hash, hmac]}, - {sha224, [], [hash, hmac]}, - {sha256, [], [hash, hmac]}, - {sha384, [], [hash, hmac]}, - {sha512, [], [hash, hmac]}, - {sha3_224, [], [hash, hmac]}, - {sha3_256, [], [hash, hmac]}, - {sha3_384, [], [hash, hmac]}, - {sha3_512, [], [hash, hmac]}, - {blake2b, [], [hash, hmac]}, - {blake2s, [], [hash, hmac]}, + {sha, [], [hash, hmac, hmac_update]}, + {sha224, [], [hash, hmac, hmac_update]}, + {sha256, [], [hash, hmac, hmac_update]}, + {sha384, [], [hash, hmac, hmac_update]}, + {sha512, [], [hash, hmac, hmac_update]}, + {sha3_224, [], [hash, hmac, hmac_update]}, + {sha3_256, [], [hash, hmac, hmac_update]}, + {sha3_384, [], [hash, hmac, hmac_update]}, + {sha3_512, [], [hash, hmac, hmac_update]}, + {blake2b, [], [hash, hmac, hmac_update]}, + {blake2s, [], [hash, hmac, hmac_update]}, {no_blake2b, [], [no_hash, no_hmac]}, {no_blake2s, [], [no_hash, no_hmac]}, {rsa, [], [sign_verify, @@ -265,9 +265,9 @@ groups() -> %% New cipher nameing schema {des_ede3_cbc, [], [api_ng, api_ng_one_shot, api_ng_tls]}, {des_ede3_cfb, [], [api_ng, api_ng_one_shot, api_ng_tls]}, - {aes_128_cbc, [], [api_ng, api_ng_one_shot, api_ng_tls]}, + {aes_128_cbc, [], [api_ng, api_ng_one_shot, api_ng_tls, cmac]}, {aes_192_cbc, [], [api_ng, api_ng_one_shot, api_ng_tls]}, - {aes_256_cbc, [], [api_ng, api_ng_one_shot, api_ng_tls]}, + {aes_256_cbc, [], [api_ng, api_ng_one_shot, api_ng_tls, cmac]}, {aes_128_ctr, [], [api_ng, api_ng_one_shot, api_ng_tls]}, {aes_192_ctr, [], [api_ng, api_ng_one_shot, api_ng_tls]}, {aes_256_ctr, [], [api_ng, api_ng_one_shot, api_ng_tls]}, @@ -386,7 +386,7 @@ init_per_testcase(info, Config) -> init_per_testcase(cmac, Config) -> case is_supported(cmac) of true -> - Config; + configure_mac(cmac, proplists:get_value(type,Config), Config); false -> {skip, "CMAC is not supported"} end; @@ -405,6 +405,8 @@ init_per_testcase(generate, Config) -> end; _ -> Config end; +init_per_testcase(hmac, Config) -> + configure_mac(hmac, proplists:get_value(type,Config), Config); init_per_testcase(_Name,Config) -> Config. @@ -452,27 +454,41 @@ no_hash(Config) when is_list(Config) -> notsup(fun crypto:hash_init/1, [Type]). %%-------------------------------------------------------------------- hmac() -> - [{doc, "Test all different hmac functions"}]. + [{doc, "Test hmac function"}]. hmac(Config) when is_list(Config) -> - {Type, Keys, DataLE, Expected} = proplists:get_value(hmac, Config), - Data = lazy_eval(DataLE), - hmac(Type, Keys, Data, Expected), - hmac(Type, lists:map(fun iolistify/1, Keys), lists:map(fun iolistify/1, Data), Expected), - hmac_increment(Type). + Tuples = lazy_eval(proplists:get_value(hmac, Config)), + lists:foreach(fun hmac_check/1, Tuples), + lists:foreach(fun hmac_check/1, mac_listify(Tuples)). + %%-------------------------------------------------------------------- no_hmac() -> [{doc, "Test all disabled hmac functions"}]. no_hmac(Config) when is_list(Config) -> Type = ?config(type, Config), - notsup(fun crypto:hmac/3, [Type, <<"Key">>, <<"Hi There">>]), + notsup(fun crypto:hmac/3, [Type, <<"Key">>, <<"Hi There">>]). + +%%-------------------------------------------------------------------- +hmac_update() -> + [{doc, "Test all incremental hmac functions"}]. +hmac_update(Config) -> + Type = ?config(type, Config), + hmac_increment(Type). + +%%-------------------------------------------------------------------- +no_hmac_update() -> + [{doc, "Test all disabled incremental hmac functions"}]. +no_hmac_update(Config) -> + Type = ?config(type, Config), notsup(fun crypto:hmac_init/2, [Type, <<"Key">>]). + %%-------------------------------------------------------------------- cmac() -> [{doc, "Test all different cmac functions"}]. cmac(Config) when is_list(Config) -> Pairs = lazy_eval(proplists:get_value(cmac, Config)), lists:foreach(fun cmac_check/1, Pairs), - lists:foreach(fun cmac_check/1, cmac_iolistify(Pairs)). + lists:foreach(fun cmac_check/1, mac_listify(Pairs)). + %%-------------------------------------------------------------------- poly1305() -> [{doc, "Test poly1305 function"}]. @@ -957,33 +973,46 @@ hash_increment(State0, [Increment | Rest]) -> State = crypto:hash_update(State0, Increment), hash_increment(State, Rest). -hmac(_, [],[],[]) -> - ok; -hmac(sha = Type, [Key | Keys], [ <<"Test With Truncation">> = Data| Rest], [Expected | Expects]) -> - call_crypto_hmac([Type, Key, Data, 20], Type, Expected), - hmac(Type, Keys, Rest, Expects); -hmac(Type, [Key | Keys], [ <<"Test With Truncation">> = Data| Rest], [Expected | Expects]) -> - call_crypto_hmac([Type, Key, Data, 16], Type, Expected), - hmac(Type, Keys, Rest, Expects); -hmac(Type, [Key | Keys], [Data| Rest], [Expected | Expects]) -> - call_crypto_hmac([Type, Key, Data], Type, Expected), - hmac(Type, Keys, Rest, Expects). - -call_crypto_hmac(Args, Type, Expected) -> - try apply(crypto, hmac, Args) + +%%%---------------------------------------------------------------- +hmac_check({hmac, sha=Type, Key, <<"Test With Truncation">>=Data, Expected}) -> + do_hmac_check(Type, Key, Data, 20, Expected); +hmac_check({hmac, Type, Key, <<"Test With Truncation">>=Data, Expected}) -> + do_hmac_check(Type, Key, Data, 16, Expected); +hmac_check({hmac, Type, Key, Data, Expected}) -> + do_hmac_check(Type, Key, Data, Expected). + + +do_hmac_check(Type, Key, Data, Expected) -> + try crypto:hmac(Type, Key, Data) of Expected -> ok; Other -> - ct:fail({{crypto,hmac,Args}, {expected,Expected}, {got,Other}}) + ct:fail({{crypto,hmac,[Type,Key,Data]}, {expected,Expected}, {got,Other}}) catch error:notsup -> ct:fail("HMAC ~p not supported", [Type]); Class:Cause -> - ct:fail({{crypto,hmac,Args}, {expected,Expected}, {got,{Class,Cause}}}) + ct:fail({{crypto,hmac,[Type,Key,Data]}, {expected,Expected}, {got,{Class,Cause}}}) end. +do_hmac_check(Type, Key, Data, MacLength, Expected) -> + try crypto:hmac(Type, Key, Data, MacLength) + of + Expected -> + ok; + Other -> + ct:fail({{crypto,hmac,[Type,Key,Data,MacLength]}, {expected,Expected}, {got,Other}}) + catch + error:notsup -> + ct:fail("HMAC ~p not supported", [Type]); + Class:Cause -> + ct:fail({{crypto,hmac,[Type,Key,Data,MacLength]}, {expected,Expected}, {got,{Class,Cause}}}) + end. + +%%%---------------------------------------------------------------- hmac_increment(Type) -> Key = hmac_key(Type), Increments = hmac_inc(Type), @@ -1002,7 +1031,8 @@ hmac_increment(State0, [Increment | Rest]) -> State = crypto:hmac_update(State0, Increment), hmac_increment(State, Rest). -cmac_check({Type, Key, Text, CMac}) -> +%%%---------------------------------------------------------------- +cmac_check({cmac, Type, Key, Text, CMac}) -> ExpCMac = iolist_to_binary(CMac), case crypto:cmac(Type, Key, Text) of ExpCMac -> @@ -1010,7 +1040,7 @@ cmac_check({Type, Key, Text, CMac}) -> Other -> ct:fail({{crypto, cmac, [Type, Key, Text]}, {expected, ExpCMac}, {got, Other}}) end; -cmac_check({Type, Key, Text, Size, CMac}) -> +cmac_check({cmac, Type, Key, Text, Size, CMac}) -> ExpCMac = iolist_to_binary(CMac), case crypto:cmac(Type, Key, Text, Size) of ExpCMac -> @@ -1020,6 +1050,24 @@ cmac_check({Type, Key, Text, Size, CMac}) -> end. +mac_check({MacType, SubType, Key, Text, Mac}) -> + ExpMac = iolist_to_binary(Mac), + case crypto:mac(MacType, SubType, Key, Text) of + ExpMac -> + ok; + Other -> + ct:fail({{crypto, mac, [MacType, SubType, Key, Text]}, {expected, ExpMac}, {got, Other}}) + end; +mac_check({MacType, SubType, Key, Text, Size, Mac}) -> + ExpMac = iolist_to_binary(Mac), + case crypto:mac(MacType, SubType, Key, Text, Size) of + ExpMac -> + ok; + Other -> + ct:fail({{crypto, mac, [MacType, SubType, Key, Text]}, {expected, ExpMac}, {got, Other}}) + end. + + block_cipher({Type, Key, PlainText}) -> Plain = iolist_to_binary(PlainText), CipherText = crypto:block_encrypt(Type, Key, PlainText), @@ -1450,17 +1498,17 @@ decstr2int(S) -> is_supported(Group) -> lists:member(Group, lists:append([Algo || {_, Algo} <- crypto:supports()])). -cmac_iolistify(Blocks) -> - lists:map(fun do_cmac_iolistify/1, Blocks). +mac_listify(Blocks) -> + lists:map(fun do_mac_listify/1, Blocks). block_iolistify(Blocks) -> lists:map(fun do_block_iolistify/1, Blocks). stream_iolistify(Streams) -> lists:map(fun do_stream_iolistify/1, Streams). -do_cmac_iolistify({Type, Key, Text, CMac}) -> - {Type, iolistify(Key), iolistify(Text), CMac}; -do_cmac_iolistify({Type, Key, Text, Size, CMac}) -> - {Type, iolistify(Key), iolistify(Text), Size, CMac}. +do_mac_listify({MType, Type, Key, Text, CMac}) -> + {MType, Type, iolistify(Key), iolistify(Text), CMac}; +do_mac_listify({MType, Type, Key, Text, Size, CMac}) -> + {MType, Type, iolistify(Key), iolistify(Text), Size, CMac}. do_stream_iolistify({Type, Key, PlainText}) -> {Type, iolistify(Key), iolistify(PlainText)}; @@ -1694,10 +1742,7 @@ group_config(md4 = Type, Config) -> group_config(md5 = Type, Config) -> Msgs = rfc_1321_msgs(), Digests = rfc_1321_md5_digests(), - Keys = rfc_2202_md5_keys() ++ [long_hmac_key(md5)], - Data = rfc_2202_msgs() ++ [long_msg()], - Hmac = rfc_2202_hmac_md5() ++ [long_hmac(md5)], - [{hash, {Type, Msgs, Digests}}, {hmac, {Type, Keys, Data, Hmac}} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(ripemd160 = Type, Config) -> Msgs = ripemd160_msgs(), Digests = ripemd160_digests(), @@ -1705,56 +1750,41 @@ group_config(ripemd160 = Type, Config) -> group_config(sha = Type, Config) -> Msgs = [rfc_4634_test1(), rfc_4634_test2_1(),long_msg()], Digests = rfc_4634_sha_digests() ++ [long_sha_digest()], - Keys = rfc_2202_sha_keys() ++ [long_hmac_key(sha)], - Data = rfc_2202_msgs() ++ [long_msg()], - Hmac = rfc_2202_hmac_sha() ++ [long_hmac(sha)], - [{hash, {Type, Msgs, Digests}}, {hmac, {Type, Keys, Data, Hmac}} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(sha224 = Type, Config) -> Msgs = [rfc_4634_test1(), rfc_4634_test2_1()], Digests = rfc_4634_sha224_digests(), - Keys = rfc_4231_keys(), - Data = rfc_4231_msgs(), - Hmac = rfc4231_hmac_sha224(), - [{hash, {Type, Msgs, Digests}}, {hmac, {Type, Keys, Data, Hmac}} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(sha256 = Type, Config) -> Msgs = [rfc_4634_test1(), rfc_4634_test2_1(), long_msg()], Digests = rfc_4634_sha256_digests() ++ [long_sha256_digest()], - Keys = rfc_4231_keys() ++ [long_hmac_key(sha256)], - Data = rfc_4231_msgs() ++ [long_msg()], - Hmac = rfc4231_hmac_sha256() ++ [long_hmac(sha256)], - [{hash, {Type, Msgs, Digests}}, {hmac, {Type, Keys, Data, Hmac}} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(sha384 = Type, Config) -> Msgs = [rfc_4634_test1(), rfc_4634_test2(), long_msg()], Digests = rfc_4634_sha384_digests() ++ [long_sha384_digest()], - Keys = rfc_4231_keys() ++ [long_hmac_key(sha384)], - Data = rfc_4231_msgs() ++ [long_msg()], - Hmac = rfc4231_hmac_sha384() ++ [long_hmac(sha384)], - [{hash, {Type, Msgs, Digests}}, {hmac, {Type, Keys, Data, Hmac}} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(sha512 = Type, Config) -> Msgs = [rfc_4634_test1(), rfc_4634_test2(), long_msg()], Digests = rfc_4634_sha512_digests() ++ [long_sha512_digest()], - Keys = rfc_4231_keys() ++ [long_hmac_key(sha512)], - Data = rfc_4231_msgs() ++ [long_msg()], - Hmac = rfc4231_hmac_sha512() ++ [long_hmac(sha512)], - [{hash, {Type, Msgs, Digests}}, {hmac, {Type, Keys, Data, Hmac}} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(sha3_224 = Type, Config) -> {Msgs,Digests} = sha3_test_vectors(Type), - [{hash, {Type, Msgs, Digests}}, {hmac, hmac_sha3(Type)} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(sha3_256 = Type, Config) -> {Msgs,Digests} = sha3_test_vectors(Type), - [{hash, {Type, Msgs, Digests}}, {hmac, hmac_sha3(Type)} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(sha3_384 = Type, Config) -> {Msgs,Digests} = sha3_test_vectors(Type), - [{hash, {Type, Msgs, Digests}}, {hmac, hmac_sha3(Type)} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(sha3_512 = Type, Config) -> {Msgs,Digests} = sha3_test_vectors(Type), - [{hash, {Type, Msgs, Digests}}, {hmac, hmac_sha3(Type)} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(blake2b = Type, Config) -> {Msgs, Digests} = blake2_test_vectors(Type), - [{hash, {Type, Msgs, Digests}}, {hmac, blake2_hmac(Type)} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(blake2s = Type, Config) -> {Msgs, Digests} = blake2_test_vectors(Type), - [{hash, {Type, Msgs, Digests}}, {hmac, blake2_hmac(Type)} | Config]; + [{hash, {Type, Msgs, Digests}} | Config]; group_config(rsa, Config) -> Msg = rsa_plain(), Public = rsa_public(), @@ -1828,7 +1858,6 @@ group_config(Type, Config) when Type == ed25519 ; Type == ed448 -> group_config(srp, Config) -> GenerateCompute = [srp3(), srp6(), srp6a(), srp6a_smaller_prime()], [{generate_compute, GenerateCompute} | Config]; - group_config(ecdh, Config) -> Compute = ecdh(), Generate = ecc(), @@ -1836,19 +1865,6 @@ group_config(ecdh, Config) -> group_config(dh, Config) -> GenerateCompute = [dh()], [{generate_compute, GenerateCompute} | Config]; - -group_config(aes_cbc128 = Type, Config) -> - Block = fun() -> aes_cbc128(Config) end, - Pairs = fun() -> cmac_nist(Config, Type) end, - [{cipher, Block}, {cmac, Pairs} | Config]; -group_config(aes_cbc256 = Type, Config) -> - Block = fun() -> aes_cbc256(Config) end, - Pairs = fun() -> cmac_nist(Config, Type) end, - [{cipher, Block}, {cmac, Pairs} | Config]; -group_config(chacha20_poly1305, Config) -> - AEAD = chacha20_poly1305(Config), - [{cipher, AEAD} | Config]; - group_config(poly1305, Config) -> V = [%% {Key, Txt, Expect} {%% RFC7539 2.5.2 @@ -1864,6 +1880,76 @@ group_config(F, Config) -> [{cipher, TestVectors} | Config]. +configure_mac(MacType, SubType, Config) -> + case do_configure_mac(MacType, SubType, Config) of + undefined -> + {skip, io:format("No ~p test vectors for ~p", [MacType, SubType])}; + Pairs -> + [{MacType, Pairs} | Config] + end. + +do_configure_mac(hmac, Type, _Config) -> + case Type of + md5 -> + Keys = rfc_2202_md5_keys() ++ [long_hmac_key(md5)], + Data = rfc_2202_msgs() ++ [long_msg()], + Hmac = rfc_2202_hmac_md5() ++ [long_hmac(md5)], + zip3_special(hmac, Type, Keys, Data, Hmac); + sha -> + Keys = rfc_2202_sha_keys() ++ [long_hmac_key(sha)], + Data = rfc_2202_msgs() ++ [long_msg()], + Hmac = rfc_2202_hmac_sha() ++ [long_hmac(sha)], + zip3_special(hmac, Type, Keys, Data, Hmac); + sha224 -> + Keys = rfc_4231_keys(), + Data = rfc_4231_msgs(), + Hmac = rfc4231_hmac_sha224(), + zip3_special(hmac, Type, Keys, Data, Hmac); + sha256 -> + Keys = rfc_4231_keys() ++ [long_hmac_key(sha256)], + Data = rfc_4231_msgs() ++ [long_msg()], + Hmac = rfc4231_hmac_sha256() ++ [long_hmac(sha256)], + zip3_special(hmac, Type, Keys, Data, Hmac); + sha384 -> + Keys = rfc_4231_keys() ++ [long_hmac_key(sha384)], + Data = rfc_4231_msgs() ++ [long_msg()], + Hmac = rfc4231_hmac_sha384() ++ [long_hmac(sha384)], + zip3_special(hmac, Type, Keys, Data, Hmac); + sha512 -> + Keys = rfc_4231_keys() ++ [long_hmac_key(sha512)], + Data = rfc_4231_msgs() ++ [long_msg()], + Hmac = rfc4231_hmac_sha512() ++ [long_hmac(sha512)], + zip3_special(hmac, Type, Keys, Data, Hmac); + sha3_224 -> + hmac_sha3(Type); + sha3_256 -> + hmac_sha3(Type); + sha3_384 -> + hmac_sha3(Type); + sha3_512 -> + hmac_sha3(Type); + blake2b -> + blake2_hmac(Type); + blake2s -> + blake2_hmac(Type); + _ -> + undefined + end; +do_configure_mac(cmac, Cipher, Config) -> + case Cipher of + aes_128_cbc -> + fun() -> read_rsp(Config, Cipher, ["CMACGenAES128.rsp", "CMACVerAES128.rsp"]) end; + aes_256_cbc -> + fun() -> read_rsp(Config, Cipher, ["CMACGenAES256.rsp", "CMACVerAES256.rsp"]) end; + _ -> + undefined + end. + + +zip3_special(Type, SubType, As, Bs, Cs) -> + [{Type, SubType, A, B, C} + || {A,B,C} <- lists:zip3(As, Bs, Cs)]. + rsa_sign_verify_tests(Config, Msg, Public, Private, PublicS, PrivateS, OptsToTry) -> case ?config(fips, Config) of @@ -1981,10 +2067,8 @@ blake2_test_vectors(blake2s) -> ]}. blake2_hmac(Type) -> - {Ks, Ds, Hs} = lists:unzip3( - [ {hexstr2bin(K), hexstr2bin(D), H} - || {{K, D}, H} <- lists:zip(blake2_hmac_key_data(), blake2_hmac_hmac(Type)) ]), - {Type, Ks, Ds, Hs}. + [{hmac, Type, hexstr2bin(K), hexstr2bin(D), H} + || {{K, D}, H} <- lists:zip(blake2_hmac_key_data(), blake2_hmac_hmac(Type)) ]. blake2_hmac_key_data() -> [ {"0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b 0b0b0b0b", @@ -2083,12 +2167,8 @@ hmac_sha3(Type) -> sha3_384 -> 3; sha3_512 -> 4 end, - {Keys, Datas, Hmacs} = - lists:unzip3( - [{hexstr2bin(Key), hexstr2bin(Data), hexstr2bin(element(N,Hmacs))} - || {Key,Data,Hmacs} <- hmac_sha3_data()]), - {Type, Keys, Datas, Hmacs}. - + [{hmac, Type, hexstr2bin(Key), hexstr2bin(Data), hexstr2bin(element(N,Hmacs))} + || {Key,Data,Hmacs} <- hmac_sha3_data()]. hmac_sha3_data() -> [ @@ -3843,14 +3923,6 @@ ecc() -> end, TestCases). -cmac_nist(Config, aes_cbc128 = Type) -> - read_rsp(Config, Type, - ["CMACGenAES128.rsp", "CMACVerAES128.rsp"]); - -cmac_nist(Config, aes_cbc256 = Type) -> - read_rsp(Config, Type, - ["CMACGenAES256.rsp", "CMACVerAES256.rsp"]). - int_to_bin(X) when X < 0 -> int_to_bin_neg(X, []); int_to_bin(X) -> int_to_bin_pos(X, []). @@ -4068,12 +4140,11 @@ parse_rsp_cmac(Type, Key0, Msg0, Mlen0, Tlen, MAC0, Next, State, Acc) -> Mlen = binary_to_integer(Mlen0), <<Msg:Mlen/bytes, _/binary>> = hexstr2bin(Msg0), MAC = hexstr2bin(MAC0), - case binary_to_integer(Tlen) of 0 -> - parse_rsp(Type, Next, State, [{Type, Key, Msg, MAC}|Acc]); + parse_rsp(Type, Next, State, [{cmac, Type, Key, Msg, MAC}|Acc]); I -> - parse_rsp(Type, Next, State, [{Type, Key, Msg, I, MAC}|Acc]) + parse_rsp(Type, Next, State, [{cmac, Type, Key, Msg, I, MAC}|Acc]) end. api_errors_ecdh(Config) when is_list(Config) -> diff --git a/lib/dialyzer/doc/src/dialyzer.xml b/lib/dialyzer/doc/src/dialyzer.xml index 443de7b0dd..8dd814982d 100644 --- a/lib/dialyzer/doc/src/dialyzer.xml +++ b/lib/dialyzer/doc/src/dialyzer.xml @@ -537,7 +537,10 @@ Option :: {files, [Filename :: string()]} 'plt_check' | 'plt_remove'} | {warnings, [WarnOpts]} - | {get_warnings, bool()} + | {get_warnings, boolean()} + | {native, boolean()} + %% Defaults to false when invoked from Erlang + | {native_cache, boolean()} WarnOpts :: error_handling | no_behaviours diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index dd0a2bfd7d..0930f79840 100644 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -32,6 +32,22 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 4.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug that caused a crash when indenting a + Dialyzer warning mentioning more than one record field. + </p> + <p> + Own Id: OTP-15861 Aux Id: ERL-953 </p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 4.0</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/dialyzer/src/dialyzer.erl b/lib/dialyzer/src/dialyzer.erl index cfe5fa9b3f..d4fe064edd 100644 --- a/lib/dialyzer/src/dialyzer.erl +++ b/lib/dialyzer/src/dialyzer.erl @@ -642,11 +642,11 @@ c(Cerl, _I) -> field_diffs(Src, false) -> Src; field_diffs(Src, true) -> - Fields = string:split(Src, " and "), + Fields = string:split(Src, " and ", all), lists:join(" and ", [field_diff(Field) || Field <- Fields]). field_diff(Field) -> - [F | Ts] = string:split(Field, "::"), + [F | Ts] = string:split(Field, "::", all), F ++ " ::" ++ t(lists:flatten(lists:join("::", Ts)), true). rec_type("record "++Src, I) -> @@ -658,7 +658,7 @@ ps("pattern "++Src, I) -> ps("variable "++_=Src, _I) -> Src; ps("record field"++Rest, I) -> - [S, TypeStr] = string:split(Rest, "of type "), + [S, TypeStr] = string:split(Rest, "of type ", all), "record field" ++ S ++ "of type " ++ t(TypeStr, I). %% Scan and parse a type or a literal, and pretty-print it using erl_pp. diff --git a/lib/dialyzer/src/dialyzer.hrl b/lib/dialyzer/src/dialyzer.hrl index 4a12b9b671..e1821f10eb 100644 --- a/lib/dialyzer/src/dialyzer.hrl +++ b/lib/dialyzer/src/dialyzer.hrl @@ -160,7 +160,9 @@ indent_opt = ?INDENT_OPT :: iopt(), callgraph_file = "" :: file:filename(), check_plt = true :: boolean(), - solvers = [] :: [solver()]}). + solvers = [] :: [solver()], + native = maybe :: boolean() | 'maybe', + native_cache = true :: boolean()}). -record(contract, {contracts = [] :: [contract_pair()], args = [] :: [erl_types:erl_type()], diff --git a/lib/dialyzer/src/dialyzer_cl.erl b/lib/dialyzer/src/dialyzer_cl.erl index f887f661bd..403fcb6279 100644 --- a/lib/dialyzer/src/dialyzer_cl.erl +++ b/lib/dialyzer/src/dialyzer_cl.erl @@ -489,10 +489,20 @@ expand_dependent_modules_1([], Included, _ModDeps) -> -spec hipe_compile([file:filename()], #options{}) -> 'ok'. -hipe_compile(Files, #options{erlang_mode = ErlangMode} = Options) -> - NoNative = (get(dialyzer_options_native) =:= false), +hipe_compile(Files, #options{erlang_mode = ErlangMode, + native = Native, + native_cache = NativeCache} = Options) -> + NoNative = + case ErlangMode of + true -> + %% In Erlang mode, native compilation must be explicitly enabled + Native =/= true; + false -> + %% In CLI mode, perform native compilation unless disabled + Native =:= false + end, FewFiles = (length(Files) < ?MIN_FILES_FOR_NATIVE_COMPILE), - case NoNative orelse FewFiles orelse ErlangMode of + case NoNative orelse FewFiles of true -> ok; false -> case erlang:system_info(hipe_architecture) of @@ -508,8 +518,7 @@ hipe_compile(Files, #options{erlang_mode = ErlangMode} = Options) -> dialyzer_worker], report_native_comp(Options), {T1, _} = statistics(wall_clock), - Cache = (get(dialyzer_options_native_cache) =/= false), - native_compile(Mods, Cache), + native_compile(Mods, NativeCache), {T2, _} = statistics(wall_clock), report_elapsed_time(T1, T2, Options) end diff --git a/lib/dialyzer/src/dialyzer_cl_parse.erl b/lib/dialyzer/src/dialyzer_cl_parse.erl index 280cae81d5..cadc2116b0 100644 --- a/lib/dialyzer/src/dialyzer_cl_parse.erl +++ b/lib/dialyzer/src/dialyzer_cl_parse.erl @@ -316,7 +316,9 @@ common_options() -> {use_spec, get(dialyzer_options_use_contracts)}, {warnings, get(dialyzer_warnings)}, {check_plt, get(dialyzer_options_check_plt)}, - {solvers, get(dialyzer_solvers)}]. + {solvers, get(dialyzer_solvers)}, + {native, get(dialyzer_options_native)}, + {native_cache, get(dialyzer_options_native_cache)}]. %%----------------------------------------------------------------------- diff --git a/lib/dialyzer/src/dialyzer_options.erl b/lib/dialyzer/src/dialyzer_options.erl index 3b30036c1c..f88f4f8ea2 100644 --- a/lib/dialyzer/src/dialyzer_options.erl +++ b/lib/dialyzer/src/dialyzer_options.erl @@ -197,6 +197,10 @@ build_options([{OptionName, Value} = Term|Rest], Options) -> solvers -> assert_solvers(Value), build_options(Rest, Options#options{solvers = Value}); + native -> + build_options(Rest, Options#options{native = Value}); + native_cache -> + build_options(Rest, Options#options{native_cache = Value}); _ -> bad_option("Unknown dialyzer command line option", Term) end; diff --git a/lib/dialyzer/test/small_SUITE_data/results/union_paren b/lib/dialyzer/test/small_SUITE_data/results/union_paren index 3a3526df89..1766773f2d 100644 --- a/lib/dialyzer/test/small_SUITE_data/results/union_paren +++ b/lib/dialyzer/test/small_SUITE_data/results/union_paren @@ -1,7 +1,25 @@ -union_paren.erl:12: Function t2/0 has no local return -union_paren.erl:13: The call union_paren:t2(3.14) breaks the contract (integer() | atom()) -> integer() -union_paren.erl:19: Function t3/0 has no local return -union_paren.erl:20: The pattern 3.14 can never match the type atom() | integer() -union_paren.erl:5: Function t1/0 has no local return -union_paren.erl:6: The call union_paren:t1(3.14) breaks the contract ((A::integer()) | (B::atom())) -> integer() +union_paren.erl:20: Function r1/0 has no local return +union_paren.erl:21: Record construction #r1{f1::[4,...],f2::'undefined',f3::'undefined',f8::float()} violates the declared type of field f2::[atom() | pid() | integer()] and f3::[atom() | pid() | integer()] and f8::[atom() | pid() | integer()] +union_paren.erl:23: Function t1/0 has no local return +union_paren.erl:24: The call union_paren:t1(3.14) breaks the contract ((A::integer()) | (B::atom())) -> integer() +union_paren.erl:30: Function t2/0 has no local return +union_paren.erl:31: The call union_paren:t2(3.14) breaks the contract (integer() | atom()) -> integer() +union_paren.erl:37: Function t3/0 has no local return +union_paren.erl:38: The pattern 3.14 can never match the type atom() | integer() +union_paren.erl:44: Function c1/0 has no local return +union_paren.erl:45: The call union_paren:c1({'r0', 'a', 'undefined', 'undefined'}) breaks the contract (#r0{f1::integer() | pid()}) -> atom() +union_paren.erl:51: Function c2/0 has no local return +union_paren.erl:52: The call union_paren:c2({'r0', 'a', 'undefined', 'undefined'}) breaks the contract (#r0{f1::A::integer() | pid()}) -> atom() +union_paren.erl:58: Function c3/0 has no local return +union_paren.erl:59: The call union_paren:c3({'r0', 'a', 'undefined', 'undefined'}) breaks the contract (#r0{f1::(A::integer()) | (B::pid())}) -> atom() +union_paren.erl:65: Function c4/0 has no local return +union_paren.erl:66: The call union_paren:c4({'r0', 'a', 'undefined', 'undefined'}) breaks the contract (#r0{f1::X::(A::integer()) | (B::pid())}) -> atom() +union_paren.erl:72: Function c5/0 has no local return +union_paren.erl:73: The call union_paren:c5({'r1', ['a'], [1], ['a'], ['u']}) breaks the contract (#r1{f1::[integer()] | [pid()]}) -> atom() +union_paren.erl:79: Function c6/0 has no local return +union_paren.erl:80: The call union_paren:c6({'r1', ['a'], [1], ['a'], ['u']}) breaks the contract (#r1{f1::A::[integer()] | [pid()]}) -> atom() +union_paren.erl:86: Function c7/0 has no local return +union_paren.erl:87: The call union_paren:c7({'r1', ['a'], [1], ['a'], ['u']}) breaks the contract (#r1{f1::(A::[integer()]) | (B::[pid()])}) -> atom() +union_paren.erl:93: Function c8/0 has no local return +union_paren.erl:94: The call union_paren:c8({'r1', ['a'], [1], ['a'], ['u']}) breaks the contract (#r1{f1::X::(A::[integer()]) | (B::[pid()])}) -> atom() diff --git a/lib/dialyzer/test/small_SUITE_data/src/union_paren.erl b/lib/dialyzer/test/small_SUITE_data/src/union_paren.erl index 4691a57d98..65bda1876e 100644 --- a/lib/dialyzer/test/small_SUITE_data/src/union_paren.erl +++ b/lib/dialyzer/test/small_SUITE_data/src/union_paren.erl @@ -2,6 +2,24 @@ -compile(export_all). +-record(r0, + { + f1 = 4 :: atom () | integer() | pid(), + f2 :: atom() | integer() | pid(), + f3 :: A :: atom() | integer() | pid + }). + +-record(r1, + { + f1 = [4] :: [atom ()] | [integer()] | [pid()], + f2 :: [atom()] | [integer()] | [pid()], + f3 :: A :: [atom()] | [integer()] | [pid()], + f8 = [u] :: X :: [A :: atom()] | [B :: integer()] | (C :: [pid()]) + }). + +r1() -> + #r1{f8 = 3.14}. + t1() -> t1(3.14). @@ -22,3 +40,59 @@ t3() -> -spec t3(_) -> (I :: integer()) | (A :: atom()). t3(A) when is_atom(A) -> A; t3(I) when is_integer(I) -> I. + +c1() -> + c1(#r0{f1 = a}). + +-spec c1(#r0{f1 :: integer() | pid()}) -> atom(). +c1(_) -> + a. + +c2() -> + c2(#r0{f1 = a}). + +-spec c2(#r0{f1 :: A :: integer() | pid()}) -> atom(). +c2(_) -> + a. + +c3() -> + c3(#r0{f1 = a}). + +-spec c3(#r0{f1 :: (A :: integer()) | (B :: pid())}) -> atom(). +c3(_) -> + a. + +c4() -> + c4(#r0{f1 = a}). + +-spec c4(#r0{f1 :: X :: (A :: integer()) | (B :: pid())}) -> atom(). +c4(_) -> + a. + +c5() -> + c5(#r1{f1 = [a], f2 = [1], f3 = [a]}). + +-spec c5(#r1{f1 :: [integer()] | [pid()]}) -> atom(). +c5(_) -> + a. + +c6() -> + c6(#r1{f1 = [a], f2 = [1], f3 = [a]}). + +-spec c6(#r1{f1 :: A :: [integer()] | [pid()]}) -> atom(). +c6(_) -> + a. + +c7() -> + c7(#r1{f1 = [a], f2 = [1], f3 = [a]}). + +-spec c7(#r1{f1 :: (A :: [integer()]) | (B :: [pid()])}) -> atom(). +c7(_) -> + a. + +c8() -> + c8(#r1{f1 = [a], f2 = [1], f3 = [a]}). + +-spec c8(#r1{f1 :: X :: (A :: [integer()]) | (B :: [pid()])}) -> atom(). +c8(_) -> + a. diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index 95984c7c85..466bbfd0f2 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 4.0 +DIALYZER_VSN = 4.0.1 diff --git a/lib/inets/src/http_server/httpd_request.erl b/lib/inets/src/http_server/httpd_request.erl index 9d7538a13d..f3e24263b8 100644 --- a/lib/inets/src/http_server/httpd_request.erl +++ b/lib/inets/src/http_server/httpd_request.erl @@ -196,9 +196,9 @@ parse_headers(<<?CR,?LF,?LF,Body/binary>>, [], [], Current, Max, Options, Result parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], Current, Max, Options, Result); -parse_headers(<<?LF,?LF,Body/binary>>, [], [], Current, Max, Options, Result) -> +parse_headers(<<?LF,?LF,Body/binary>>, Header, Headers, Current, Max, Options, Result) -> %% If ?CR is is missing RFC2616 section-19.3 - parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], Current, Max, + parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, Header, Headers, Current, Max, Options, Result); parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], _, _, _, Result) -> diff --git a/lib/inets/test/http_format_SUITE.erl b/lib/inets/test/http_format_SUITE.erl index 0a5aed67d5..3ff3ed4e97 100644 --- a/lib/inets/test/http_format_SUITE.erl +++ b/lib/inets/test/http_format_SUITE.erl @@ -414,6 +414,19 @@ http_request(Config) when is_list(Config) -> {max_content_length, ?HTTP_MAX_CONTENT_LENGTH} ]], HttpHead2), + %% If ?CR is is missing RFC2616 section-19.3 + HttpHead3 = ["GET http://www.erlang.org HTTP/1.1", [?LF], + "Accept: text/html", [?LF, ?LF]], + {"GET", + "http://www.erlang.org", + "HTTP/1.1", + {#http_request_h{}, [{"accept","text/html"}]}, <<>>} = + parse(httpd_request, parse, [[{max_header, ?HTTP_MAX_HEADER_SIZE}, + {max_version, ?HTTP_MAX_VERSION_STRING}, + {max_method, ?HTTP_MAX_METHOD_STRING}, + {max_content_length, ?HTTP_MAX_CONTENT_LENGTH} + ]], HttpHead3), + %% Note the following body is not related to the headers above HttpBody = ["<HTML>\n<HEAD>\n<TITLE> dummy </TITLE>\n</HEAD>\n<BODY>\n", "<H1>dummy</H1>\n</BODY>\n</HTML>\n"], diff --git a/lib/kernel/doc/src/gen_udp.xml b/lib/kernel/doc/src/gen_udp.xml index d20fc1fdfd..6c0d072fed 100644 --- a/lib/kernel/doc/src/gen_udp.xml +++ b/lib/kernel/doc/src/gen_udp.xml @@ -213,12 +213,93 @@ </func> <func> - <name name="send" arity="4" since=""/> + <name name="send" arity="3" since="OTP @OTP-15747@"/> <fsummary>Send a packet.</fsummary> <desc> <p> - Sends a packet to the specified address and port. Argument - <c><anno>Address</anno></c> can be a hostname or a socket address. + Sends a packet to the specified <c><anno>Destination</anno></c>. + </p> + <p> + This function is equivalent to + <seealso marker="#send-4-AncData"><c>send(<anno>Socket</anno>, <anno>Destination</anno>, [], <anno>Packet</anno>)</c></seealso>. + </p> + </desc> + </func> + + <func> + <name name="send" arity="4" clause_i="1" since=""/> + <fsummary>Send a packet.</fsummary> + <desc> + <p> + Sends a packet to the specified <c><anno>Host</anno></c> + and <c><anno>Port</anno></c>. + </p> + <p> + This clause is equivalent to + <seealso marker="#send/5"><c>send(<anno>Socket</anno>, <anno>Host</anno>, <anno>Port</anno>, [], <anno>Packet</anno>)</c></seealso>. + </p> + </desc> + </func> + + <func> + <name name="send" arity="4" clause_i="2" anchor="send-4-AncData" since="OTP @OTP-15747@"/> + <fsummary>Send a packet.</fsummary> + <desc> + <p> + Sends a packet to the specified <c><anno>Destination</anno></c> + with ancillary data <c><anno>AncData</anno></c>. + </p> + <note> + <p> + The ancillary data <c><anno>AncData</anno></c> + contains options that for this single message + override the default options for the socket, + an operation that may not be supported on all platforms, + and if so return <c>{error, einval}</c>. + Using more than one of an ancillary data item type + may also not be supported. + <c><anno>AncData</anno> =:= []</c> is always supported. + </p> + </note> + </desc> + </func> + + <func> + <name name="send" arity="4" clause_i="3" since="OTP @OTP-15747@"/> + <fsummary>Send a packet.</fsummary> + <desc> + <p> + Sends a packet to the specified <c><anno>Destination</anno></c>. + Since <c><anno>Destination</anno></c> is complete, + <c><anno>PortZero</anno></c> is redundant and has to be <c>0</c>. + </p> + <p> + This is a legacy clause mostly for + <c><anno>Destination</anno> = {local, Binary}</c> + where <c><anno>PortZero</anno></c> is superfluous. + It is equivalent to + <seealso marker="#send-4-AncData"><c>send(<anno>Socket</anno>, <anno>Destination</anno>, [], <anno>Packet</anno>)</c></seealso>, the clause right above here. + </p> + </desc> + </func> + + <func> + <name name="send" arity="5" since="OTP @OTP-15747@"/> + <fsummary>Send a packet.</fsummary> + <desc> + <p> + Sends a packet to the specified <c><anno>Host</anno></c> + and <c><anno>Port</anno></c>, + with ancillary data <c><anno>AncData</anno></c>. + </p> + <p> + Argument <c><anno>Host</anno></c> can be + a hostname or a socket address, + and <c><anno>Port</anno></c> can be a port number + or a service name atom. + These are resolved into a <c>Destination</c> and after that + this function is equivalent to + <seealso marker="#send-4-AncData"><c>send(<anno>Socket</anno>, Destination, <anno>AncData</anno>, <anno>Packet</anno>)</c></seealso>, read there about ancillary data. </p> </desc> </func> diff --git a/lib/kernel/doc/src/inet.xml b/lib/kernel/doc/src/inet.xml index d4678ca5db..1011befca0 100644 --- a/lib/kernel/doc/src/inet.xml +++ b/lib/kernel/doc/src/inet.xml @@ -118,6 +118,42 @@ fe80::204:acff:fe17:bf38 <name name="port_number"/> </datatype> <datatype> + <name name="family_address" since="OTP @OTP-15747@"/> + <desc> + <p> + A general address format on the form <c>{Family, Destination}</c> + where <c>Family</c> is an atom such as <c>local</c> + and the format of <c>Destination</c> depends on <c>Family</c>, + and is a complete address + (for example an IP address including port number). + </p> + </desc> + </datatype> + <datatype> + <name name="inet_address" since="OTP @OTP-15747@"/> + <desc> + <warning> + <p> + This address format is for now experimental + and for completeness to make all address families have a + <c>{Family, Destination}</c> representation. + </p> + </warning> + </desc> + </datatype> + <datatype> + <name name="inet6_address" since="OTP @OTP-15747@"/> + <desc> + <warning> + <p> + This address format is for now experimental + and for completeness to make all address families have a + <c>{Family, Destination}</c> representation. + </p> + </warning> + </desc> + </datatype> + <datatype> <name name="local_address"/> <desc> <p> @@ -180,12 +216,16 @@ fe80::204:acff:fe17:bf38 <name name="ancillary_data"/> <desc> <p> - Ancillary data received with the data packet - or read with the socket option + Ancillary data received with the data packet, + read with the socket option <seealso marker="gen_tcp#type-pktoptions_value"> <c>pktoptions</c> </seealso> - from a TCP socket. + from a TCP socket, + or to set in a call to + <seealso marker="gen_udp#send-4-AncData"><c>gen_udp:send/4</c></seealso> + or + <seealso marker="gen_udp#send/5"><c>gen_udp:send/5</c></seealso>. </p> <p> The value(s) correspond to the currently active socket @@ -193,7 +233,9 @@ fe80::204:acff:fe17:bf38 <seealso marker="inet#option-recvtos"><c>recvtos</c></seealso>, <seealso marker="inet#option-recvtclass"><c>recvtclass</c></seealso> and - <seealso marker="inet#option-recvttl"><c>recvttl</c></seealso>. + <seealso marker="inet#option-recvttl"><c>recvttl</c></seealso>, + or for a single send operation the option(s) to override + the currently active socket option(s). </p> </desc> </datatype> diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 6f68a67174..4d31eeea3d 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -31,6 +31,24 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 6.4.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p><c>user</c>/<c>user_drv</c> could respond to io + requests before they had been processed, which could + cause data to be dropped if the emulator was halted soon + after a call to <c>io:format/2</c>, such as in an + escript.</p> + <p> + Own Id: OTP-15805</p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 6.4</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -129,6 +147,37 @@ </section> +<section><title>Kernel 6.3.1.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The possibility to send ancillary data, in particular the + TOS field, has been added to <c>gen_udp:send/4,5</c>.</p> + <p> + Own Id: OTP-15747 Aux Id: ERIERL-294 </p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 6.3.1.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix type spec for <c>seq_trace:set_token/2</c>.</p> + <p> + Own Id: OTP-15858 Aux Id: ERL-700 </p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 6.3.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -6027,4 +6076,3 @@ </section> </section> </chapter> - diff --git a/lib/kernel/src/gen_sctp.erl b/lib/kernel/src/gen_sctp.erl index d893d44079..a63df54ff9 100644 --- a/lib/kernel/src/gen_sctp.erl +++ b/lib/kernel/src/gen_sctp.erl @@ -217,24 +217,29 @@ peeloff(S, AssocId) when is_port(S), is_integer(AssocId) -> Error -> Error end. --spec connect(Socket, Addr, Port, Opts) -> {ok, Assoc} | {error, inet:posix()} when +-spec connect(Socket, Addr, Port, Opts) -> + {ok, #sctp_assoc_change{state :: 'comm_up'}} | + {error, #sctp_assoc_change{state :: 'cant_assoc'}} | + {error, inet:posix()} + when Socket :: sctp_socket(), Addr :: inet:ip_address() | inet:hostname(), Port :: inet:port_number(), - Opts :: [Opt :: option()], - Assoc :: #sctp_assoc_change{}. + Opts :: [Opt :: option()]. connect(S, Addr, Port, Opts) -> connect(S, Addr, Port, Opts, infinity). -spec connect(Socket, Addr, Port, Opts, Timeout) -> - {ok, Assoc} | {error, inet:posix()} when + {ok, #sctp_assoc_change{state :: 'comm_up'}} | + {error, #sctp_assoc_change{state :: 'cant_assoc'}} | + {error, inet:posix()} + when Socket :: sctp_socket(), Addr :: inet:ip_address() | inet:hostname(), Port :: inet:port_number(), Opts :: [Opt :: option()], - Timeout :: timeout(), - Assoc :: #sctp_assoc_change{}. + Timeout :: timeout(). connect(S, Addr, Port, Opts, Timeout) -> case do_connect(S, Addr, Port, Opts, Timeout, true) of diff --git a/lib/kernel/src/gen_udp.erl b/lib/kernel/src/gen_udp.erl index fad7b2f887..3001948209 100644 --- a/lib/kernel/src/gen_udp.erl +++ b/lib/kernel/src/gen_udp.erl @@ -20,7 +20,7 @@ -module(gen_udp). -export([open/1, open/2, close/1]). --export([send/2, send/4, recv/2, recv/3, connect/3]). +-export([send/2, send/3, send/4, send/5, recv/2, recv/3, connect/3]). -export([controlling_process/2]). -export([fdopen/2]). @@ -125,20 +125,80 @@ open(Port, Opts0) -> close(S) -> inet:udp_close(S). --spec send(Socket, Address, Port, Packet) -> ok | {error, Reason} when +-spec send(Socket, Destination, Packet) -> ok | {error, Reason} when Socket :: socket(), - Address :: inet:socket_address() | inet:hostname(), - Port :: inet:port_number(), + Destination :: {inet:ip_address(), inet:port_number()} | + inet:family_address(), + Packet :: iodata(), + Reason :: not_owner | inet:posix(). +%%% +send(Socket, Destination, Packet) -> + send(Socket, Destination, [], Packet). + +-spec send(Socket, Host, Port, Packet) -> ok | {error, Reason} when + Socket :: socket(), + Host :: inet:hostname() | inet:ip_address(), + Port :: inet:port_number() | atom(), + Packet :: iodata(), + Reason :: not_owner | inet:posix(); +%%% + (Socket, Destination, AncData, Packet) -> ok | {error, Reason} when + Socket :: socket(), + Destination :: {inet:ip_address(), inet:port_number()} | + inet:family_address(), + AncData :: inet:ancillary_data(), + Packet :: iodata(), + Reason :: not_owner | inet:posix(); +%%% + (Socket, Destination, PortZero, Packet) -> ok | {error, Reason} when + Socket :: socket(), + Destination :: {inet:ip_address(), inet:port_number()} | + inet:family_address(), + PortZero :: inet:port_number(), Packet :: iodata(), Reason :: not_owner | inet:posix(). +%%% +send(S, {_,_} = Destination, PortZero = AncData, Packet) when is_port(S) -> + %% Destination is {Family,Addr} | {IP,Port}, + %% so it is complete - argument PortZero is redundant + if + PortZero =:= 0 -> + case inet_db:lookup_socket(S) of + {ok, Mod} -> + Mod:send(S, Destination, [], Packet); + Error -> + Error + end; + is_integer(PortZero) -> + %% Redundant PortZero; must be 0 + {error, einval}; + is_list(AncData) -> + case inet_db:lookup_socket(S) of + {ok, Mod} -> + Mod:send(S, Destination, AncData, Packet); + Error -> + Error + end + end; +send(S, Host, Port, Packet) when is_port(S) -> + send(S, Host, Port, [], Packet). -send(S, Address, Port, Packet) when is_port(S) -> +-spec send(Socket, Host, Port, AncData, Packet) -> ok | {error, Reason} when + Socket :: socket(), + Host :: inet:hostname() | inet:ip_address() | inet:local_address(), + Port :: inet:port_number() | atom(), + AncData :: inet:ancillary_data(), + Packet :: iodata(), + Reason :: not_owner | inet:posix(). +%%% +send(S, Host, Port, AncData, Packet) + when is_port(S), is_list(AncData) -> case inet_db:lookup_socket(S) of {ok, Mod} -> - case Mod:getaddr(Address) of + case Mod:getaddr(Host) of {ok,IP} -> case Mod:getserv(Port) of - {ok,UP} -> Mod:send(S, IP, UP, Packet); + {ok,P} -> Mod:send(S, {IP,P}, AncData, Packet); {error,einval} -> exit(badarg); Error -> Error end; @@ -149,6 +209,7 @@ send(S, Address, Port, Packet) when is_port(S) -> Error end. +%% Connected send send(S, Packet) when is_port(S) -> case inet_db:lookup_socket(S) of {ok, Mod} -> diff --git a/lib/kernel/src/inet.erl b/lib/kernel/src/inet.erl index 9f22eb6aaa..7940903658 100644 --- a/lib/kernel/src/inet.erl +++ b/lib/kernel/src/inet.erl @@ -75,7 +75,8 @@ -export_type([address_family/0, socket_protocol/0, hostent/0, hostname/0, ip4_address/0, ip6_address/0, ip_address/0, port_number/0, - local_address/0, socket_address/0, returned_non_ip_address/0, + family_address/0, local_address/0, + socket_address/0, returned_non_ip_address/0, socket_setopt/0, socket_getopt/0, ancillary_data/0, posix/0, socket/0, stat_option/0]). %% imports @@ -100,11 +101,16 @@ 0..65535,0..65535,0..65535,0..65535}. -type ip_address() :: ip4_address() | ip6_address(). -type port_number() :: 0..65535. --type local_address() :: {local, File :: binary() | string()}. +-type family_address() :: inet_address() | inet6_address() | local_address(). +-type inet_address() :: + {'inet', {ip4_address() | 'any' | 'loopback', port_number()}}. +-type inet6_address() :: + {'inet6', {ip6_address() | 'any' | 'loopback', port_number()}}. +-type local_address() :: {'local', File :: binary() | string()}. -type returned_non_ip_address() :: - {local, binary()} | - {unspec, <<>>} | - {undefined, any()}. + {'local', binary()} | + {'unspec', <<>>} | + {'undefined', any()}. -type posix() :: 'eaddrinuse' | 'eaddrnotavail' | 'eafnosupport' | 'ealready' | 'econnaborted' | 'econnrefused' | 'econnreset' | diff --git a/lib/kernel/src/inet6_udp.erl b/lib/kernel/src/inet6_udp.erl index 71db0357cd..cb95a69798 100644 --- a/lib/kernel/src/inet6_udp.erl +++ b/lib/kernel/src/inet6_udp.erl @@ -65,16 +65,25 @@ open(Port, Opts) -> {ok, _} -> exit(badarg) end. -send(S, Addr = {A,B,C,D,E,F,G,H}, P, Data) - when ?ip6(A,B,C,D,E,F,G,H), ?port(P) -> - prim_inet:sendto(S, Addr, P, Data). +send(S, {A,B,C,D,E,F,G,H} = IP, Port, Data) + when ?ip6(A,B,C,D,E,F,G,H), ?port(Port) -> + prim_inet:sendto(S, {IP, Port}, [], Data); +send(S, {{A,B,C,D,E,F,G,H}, Port} = Addr, AncData, Data) + when ?ip6(A,B,C,D,E,F,G,H), ?port(Port), is_list(AncData) -> + prim_inet:sendto(S, Addr, AncData, Data); +send(S, {?FAMILY, {{A,B,C,D,E,F,G,H}, Port}} = Address, AncData, Data) + when ?ip6(A,B,C,D,E,F,G,H), ?port(Port), is_list(AncData) -> + prim_inet:sendto(S, Address, AncData, Data); +send(S, {?FAMILY, {loopback, Port}} = Address, AncData, Data) + when ?port(Port), is_list(AncData) -> + prim_inet:sendto(S, Address, AncData, Data). send(S, Data) -> - prim_inet:sendto(S, {0,0,0,0,0,0,0,0}, 0, Data). + prim_inet:sendto(S, {any, 0}, [], Data). -connect(S, Addr = {A,B,C,D,E,F,G,H}, P) - when ?ip6(A,B,C,D,E,F,G,H), ?port(P) -> - prim_inet:connect(S, Addr, P). +connect(S, Addr = {A,B,C,D,E,F,G,H}, Port) + when ?ip6(A,B,C,D,E,F,G,H), ?port(Port) -> + prim_inet:connect(S, Addr, Port). recv(S, Len) -> prim_inet:recvfrom(S, Len). diff --git a/lib/kernel/src/inet_udp.erl b/lib/kernel/src/inet_udp.erl index 1e624b9e90..083059a2dc 100644 --- a/lib/kernel/src/inet_udp.erl +++ b/lib/kernel/src/inet_udp.erl @@ -66,16 +66,25 @@ open(Port, Opts) -> {ok, _} -> exit(badarg) end. -send(S, {A,B,C,D} = Addr, P, Data) - when ?ip(A,B,C,D), ?port(P) -> - prim_inet:sendto(S, Addr, P, Data). +send(S, {A,B,C,D} = IP, Port, Data) + when ?ip(A,B,C,D), ?port(Port) -> + prim_inet:sendto(S, {IP, Port}, [], Data); +send(S, {{A,B,C,D}, Port} = Addr, AncData, Data) + when ?ip(A,B,C,D), ?port(Port), is_list(AncData) -> + prim_inet:sendto(S, Addr, AncData, Data); +send(S, {?FAMILY, {{A,B,C,D}, Port}} = Address, AncData, Data) + when ?ip(A,B,C,D), ?port(Port), is_list(AncData) -> + prim_inet:sendto(S, Address, AncData, Data); +send(S, {?FAMILY, {loopback, Port}} = Address, AncData, Data) + when ?port(Port), is_list(AncData) -> + prim_inet:sendto(S, Address, AncData, Data). send(S, Data) -> - prim_inet:sendto(S, {0,0,0,0}, 0, Data). + prim_inet:sendto(S, {any, 0}, [], Data). -connect(S, Addr = {A,B,C,D}, P) - when ?ip(A,B,C,D), ?port(P) -> - prim_inet:connect(S, Addr, P). +connect(S, Addr = {A,B,C,D}, Port) + when ?ip(A,B,C,D), ?port(Port) -> + prim_inet:connect(S, Addr, Port). recv(S, Len) -> prim_inet:recvfrom(S, Len). diff --git a/lib/kernel/src/kernel.appup.src b/lib/kernel/src/kernel.appup.src index cd0397a98c..95853a7a8f 100644 --- a/lib/kernel/src/kernel.appup.src +++ b/lib/kernel/src/kernel.appup.src @@ -38,7 +38,9 @@ {<<"^6\\.2\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^6\\.3$">>,[restart_new_emulator]}, {<<"^6\\.3\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^6\\.3\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], + {<<"^6\\.3\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^6\\.4$">>,[restart_new_emulator]}, + {<<"^6\\.4\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}], [{<<"^6\\.0$">>,[restart_new_emulator]}, {<<"^6\\.0\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^6\\.0\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -50,4 +52,6 @@ {<<"^6\\.2\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^6\\.3$">>,[restart_new_emulator]}, {<<"^6\\.3\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^6\\.3\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. + {<<"^6\\.3\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^6\\.4$">>,[restart_new_emulator]}, + {<<"^6\\.4\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}]}. diff --git a/lib/kernel/src/local_udp.erl b/lib/kernel/src/local_udp.erl index 481a8c4910..933e56228b 100644 --- a/lib/kernel/src/local_udp.erl +++ b/lib/kernel/src/local_udp.erl @@ -70,11 +70,13 @@ open(0, Opts) -> {ok, _} -> exit(badarg) end. -send(S, Addr = {?FAMILY,_}, 0, Data) -> - prim_inet:sendto(S, Addr, 0, Data). +send(S, {?FAMILY,_} = Addr, 0, Data) -> + prim_inet:sendto(S, Addr, [], Data); +send(S, {?FAMILY,_} = Addr, AncData, Data) when is_list(AncData) -> + prim_inet:sendto(S, Addr, AncData, Data). %% send(S, Data) -> - prim_inet:sendto(S, {?FAMILY,<<>>}, 0, Data). + prim_inet:sendto(S, {?FAMILY,<<>>}, [], Data). connect(S, Addr = {?FAMILY,_}, 0) -> prim_inet:connect(S, Addr, 0). diff --git a/lib/kernel/src/logger_std_h.erl b/lib/kernel/src/logger_std_h.erl index c8f1acfca4..2b078ef091 100644 --- a/lib/kernel/src/logger_std_h.erl +++ b/lib/kernel/src/logger_std_h.erl @@ -170,9 +170,11 @@ check_h_config(_Type,[]) -> ok. normalize_config(#{type:={file,File}}=HConfig) -> - HConfig#{type=>file,file=>File}; + normalize_config(HConfig#{type=>file,file=>File}); normalize_config(#{type:={file,File,Modes}}=HConfig) -> - HConfig#{type=>file,file=>File,modes=>Modes}; + normalize_config(HConfig#{type=>file,file=>File,modes=>Modes}); +normalize_config(#{file:=File}=HConfig) -> + HConfig#{file=>filename:absname(File)}; normalize_config(HConfig) -> HConfig. @@ -188,7 +190,7 @@ merge_default_config(Name,Type,HConfig) -> get_default_config(Name,file) -> #{type => file, - file => atom_to_list(Name), + file => filename:absname(atom_to_list(Name)), modes => [raw,append], file_check => 0, max_no_bytes => infinity, diff --git a/lib/kernel/src/user.erl b/lib/kernel/src/user.erl index 0c9e1ea303..5a3487a9ba 100644 --- a/lib/kernel/src/user.erl +++ b/lib/kernel/src/user.erl @@ -296,7 +296,8 @@ io_requests([], Stat, _) -> %% port. put_port(List, Port) -> - send_port(Port, {command, List}). + true = port_command(Port, List), + ok. %% send_port(Port, Command) diff --git a/lib/kernel/src/user_drv.erl b/lib/kernel/src/user_drv.erl index 69ff8e7971..644aa752b6 100644 --- a/lib/kernel/src/user_drv.erl +++ b/lib/kernel/src/user_drv.erl @@ -543,19 +543,14 @@ set_unicode_state(Iport, Bool) -> %% io_request(Request, InPort, OutPort) %% io_requests(Requests, InPort, OutPort) %% Note: InPort is unused. - -io_request(Request, Iport, Oport) -> - try io_command(Request) of - {command,_} = Command -> - Oport ! {self(),Command}, - ok; - {Command,Reply} -> - Oport ! {self(),Command}, - Reply - catch - {requests,Rs} -> - io_requests(Rs, Iport, Oport); - _ -> +io_request({requests,Rs}, Iport, Oport) -> + io_requests(Rs, Iport, Oport); +io_request(Request, _Iport, Oport) -> + case io_command(Request) of + {Data, Reply} -> + true = port_command(Oport, Data), + Reply; + unhandled -> ok end. @@ -575,19 +570,19 @@ put_int16(N, Tail) -> %% to the console before the vm stops when calling erlang:halt(integer()). -dialyzer({no_improper_lists, io_command/1}). io_command({put_chars_sync, unicode,Cs,Reply}) -> - {{command,[?OP_PUTC_SYNC|unicode:characters_to_binary(Cs,utf8)]},Reply}; + {[?OP_PUTC_SYNC|unicode:characters_to_binary(Cs,utf8)], Reply}; io_command({put_chars, unicode,Cs}) -> - {command,[?OP_PUTC|unicode:characters_to_binary(Cs,utf8)]}; + {[?OP_PUTC|unicode:characters_to_binary(Cs,utf8)], ok}; io_command({move_rel,N}) -> - {command,[?OP_MOVE|put_int16(N, [])]}; + {[?OP_MOVE|put_int16(N, [])], ok}; io_command({insert_chars,unicode,Cs}) -> - {command,[?OP_INSC|unicode:characters_to_binary(Cs,utf8)]}; + {[?OP_INSC|unicode:characters_to_binary(Cs,utf8)], ok}; io_command({delete_chars,N}) -> - {command,[?OP_DELC|put_int16(N, [])]}; + {[?OP_DELC|put_int16(N, [])], ok}; io_command(beep) -> - {command,[?OP_BEEP]}; -io_command(Else) -> - throw(Else). + {[?OP_BEEP], ok}; +io_command(_) -> + unhandled. %% gr_new() %% gr_get_num(Group, Index) diff --git a/lib/kernel/test/gen_tcp_misc_SUITE.erl b/lib/kernel/test/gen_tcp_misc_SUITE.erl index edf30448c4..421510f9d6 100644 --- a/lib/kernel/test/gen_tcp_misc_SUITE.erl +++ b/lib/kernel/test/gen_tcp_misc_SUITE.erl @@ -25,6 +25,7 @@ init_per_group/2,end_per_group/2, controlling_process/1, controlling_process_self/1, no_accept/1, close_with_pending_output/1, active_n/1, + active_n_closed/1, data_before_close/1, iter_max_socks/0, iter_max_socks/1, get_status/1, @@ -36,7 +37,8 @@ show_econnreset_passive/1, econnreset_after_sync_send/1, econnreset_after_async_send_active/1, econnreset_after_async_send_active_once/1, - econnreset_after_async_send_passive/1, linger_zero/1, + econnreset_after_async_send_passive/1, + linger_zero/1, linger_zero_sndbuf/1, default_options/1, http_bad_packet/1, busy_send/1, busy_disconnect_passive/1, busy_disconnect_active/1, fill_sendq/1, partial_recv_and_close/1, @@ -73,14 +75,15 @@ suite() -> all() -> [controlling_process, controlling_process_self, no_accept, close_with_pending_output, data_before_close, - iter_max_socks, passive_sockets, active_n, + iter_max_socks, passive_sockets, active_n, active_n_closed, accept_closed_by_other_process, otp_3924, closed_socket, shutdown_active, shutdown_passive, shutdown_pending, show_econnreset_active, show_econnreset_active_once, show_econnreset_passive, econnreset_after_sync_send, econnreset_after_async_send_active, econnreset_after_async_send_active_once, - econnreset_after_async_send_passive, linger_zero, + econnreset_after_async_send_passive, + linger_zero, linger_zero_sndbuf, default_options, http_bad_packet, busy_send, busy_disconnect_passive, busy_disconnect_active, fill_sendq, partial_recv_and_close, @@ -1356,7 +1359,42 @@ linger_zero(Config) when is_list(Config) -> ok = gen_tcp:close(Client), ok = ct:sleep(1), undefined = erlang:port_info(Client, connected), - {error, econnreset} = gen_tcp:recv(S, PayloadSize). + {error, econnreset} = gen_tcp:recv(S, PayloadSize), + ok. + + +linger_zero_sndbuf(Config) when is_list(Config) -> + %% All the econnreset tests will prove that {linger, {true, 0}} aborts + %% a connection when the driver queue is empty. We will test here + %% that it also works when the driver queue is not empty + %% and the linger zero option is set on the listen socket. + {OS, _} = os:type(), + {ok, Listen} = + gen_tcp:listen(0, [{active, false}, + {recbuf, 4096}, + {show_econnreset, true}, + {linger, {true, 0}}]), + {ok, Port} = inet:port(Listen), + {ok, Client} = + gen_tcp:connect(localhost, Port, + [{active, false}, + {sndbuf, 4096}]), + {ok, Server} = gen_tcp:accept(Listen), + ok = gen_tcp:close(Listen), + PayloadSize = 1024 * 1024, + Payload = binary:copy(<<"0123456789ABCDEF">>, 256 * 1024), % 1 MB + ok = gen_tcp:send(Server, Payload), + case erlang:port_info(Server, queue_size) of + {queue_size, N} when N > 0 -> ok; + {queue_size, 0} when OS =:= win32 -> ok; + {queue_size, 0} = T -> ct:fail(T) + end, + {ok, [{linger, {true, 0}}]} = inet:getopts(Server, [linger]), + ok = gen_tcp:close(Server), + ok = ct:sleep(1), + undefined = erlang:port_info(Server, connected), + {error, closed} = gen_tcp:recv(Client, PayloadSize), + ok. %% Thanks to Luke Gorrie. Tests for a very specific problem with @@ -1984,7 +2022,7 @@ recvtclass(_Config) -> recvtos_ok({unix,openbsd}, OSVer) -> not semver_lt(OSVer, {6,4,0}); recvtos_ok({unix,darwin}, OSVer) -> not semver_lt(OSVer, {19,0,0}); %% Using the option returns einval, so it is not implemented. -recvtos_ok({unix,freebsd}, OSVer) -> not semver_lt(OSVer, {11,2,0}); +recvtos_ok({unix,freebsd}, OSVer) -> not semver_lt(OSVer, {12,1,0}); recvtos_ok({unix,sunos}, OSVer) -> not semver_lt(OSVer, {5,12,0}); %% Does not return any value - not implemented for pktoptions recvtos_ok({unix,linux}, OSVer) -> not semver_lt(OSVer, {3,1,0}); @@ -1996,7 +2034,7 @@ recvtos_ok(_, _) -> false. recvttl_ok({unix,openbsd}, OSVer) -> not semver_lt(OSVer, {6,4,0}); recvttl_ok({unix,darwin}, OSVer) -> not semver_lt(OSVer, {19,0,0}); %% Using the option returns einval, so it is not implemented. -recvttl_ok({unix,freebsd}, OSVer) -> not semver_lt(OSVer, {11,2,0}); +recvttl_ok({unix,freebsd}, OSVer) -> not semver_lt(OSVer, {12,1,0}); recvttl_ok({unix,sunos}, OSVer) -> not semver_lt(OSVer, {5,12,0}); %% Does not return any value - not implemented for pktoptions recvttl_ok({unix,linux}, OSVer) -> not semver_lt(OSVer, {2,7,0}); @@ -2009,7 +2047,7 @@ recvtclass_ok({unix,openbsd}, OSVer) -> not semver_lt(OSVer, {6,4,0}); recvtclass_ok({unix,darwin}, OSVer) -> not semver_lt(OSVer, {19,0,0}); recvtclass_ok({unix,sunos}, OSVer) -> not semver_lt(OSVer, {5,12,0}); %% Using the option returns einval, so it is not implemented. -recvtclass_ok({unix,freebsd}, OSVer) -> not semver_lt(OSVer, {11,2,0}); +recvtclass_ok({unix,freebsd}, OSVer) -> not semver_lt(OSVer, {12,1,0}); %% Does not return any value - not implemented for pktoptions recvtclass_ok({unix,linux}, OSVer) -> not semver_lt(OSVer, {3,1,0}); %% @@ -2582,7 +2620,51 @@ active_once_closed(Config) when is_list(Config) -> ok = inet:setopts(A,[{active,once}]), ok = receive {tcp_closed, A} -> ok after 1000 -> error end end)(). - + +%% Check that active n and tcp_close messages behave as expected. +active_n_closed(Config) when is_list(Config) -> + {ok, L} = gen_tcp:listen(0, [binary, {active, false}]), + + P = self(), + + {ok,Port} = inet:port(L), + + spawn_link(fun() -> + Payload = <<0:50000/unit:8>>, + Cnt = 10000, + P ! {size,Cnt * byte_size(Payload)}, + {ok, S} = gen_tcp:connect("localhost", Port, [binary, {active, false}]), + _ = [gen_tcp:send(S, Payload) || _ <- lists:seq(1, Cnt)], + gen_tcp:close(S) + end), + + receive {size,SendSize} -> SendSize end, + {ok, S} = gen_tcp:accept(L), + inet:setopts(S, [{active, 10}]), + RecvSize = + (fun Server(Size) -> + receive + {tcp, S, Bin} -> + Server(byte_size(Bin) + Size); + {tcp_closed, S} -> + Size; + {tcp_passive, S} -> + inet:setopts(S, [{active, 10}]), + Server(Size); + Msg -> + io:format("~p~n", [Msg]), + Server(Size) + end + end)(0), + + gen_tcp:close(L), + + if SendSize =:= RecvSize -> + ok; + true -> + ct:fail("Send and Recv size not equal: ~p ~p",[SendSize, RecvSize]) + end. + %% Test the send_timeout socket option. send_timeout(Config) when is_list(Config) -> Dir = filename:dirname(code:which(?MODULE)), diff --git a/lib/kernel/test/gen_udp_SUITE.erl b/lib/kernel/test/gen_udp_SUITE.erl index af9985de45..730886865c 100644 --- a/lib/kernel/test/gen_udp_SUITE.erl +++ b/lib/kernel/test/gen_udp_SUITE.erl @@ -37,6 +37,7 @@ buffer_size/1, binary_passive_recv/1, max_buffer_size/1, bad_address/1, read_packets/1, open_fd/1, connect/1, implicit_inet6/1, recvtos/1, recvtosttl/1, recvttl/1, recvtclass/1, + sendtos/1, sendtosttl/1, sendttl/1, sendtclass/1, local_basic/1, local_unbound/1, local_fdopen/1, local_fdopen_unbound/1, local_abstract/1]). @@ -49,6 +50,7 @@ all() -> bad_address, read_packets, open_fd, connect, implicit_inet6, active_n, recvtos, recvtosttl, recvttl, recvtclass, + sendtos, sendtosttl, sendttl, sendtclass, {group, local}]. groups() -> @@ -312,7 +314,6 @@ read_packets(Config) when is_list(Config) -> {ok,R} = gen_udp:open(0, [{read_packets,N1}]), {ok,RP} = inet:port(R), {ok,Node} = start_node(gen_udp_SUITE_read_packets), - Die = make_ref(), %% {V1, Trace1} = read_packets_test(R, RP, Msgs, Node), {ok,[{read_packets,N1}]} = inet:getopts(R, [read_packets]), @@ -324,7 +325,7 @@ read_packets(Config) when is_list(Config) -> stop_node(Node), ct:log("N1=~p, V1=~p vs N2=~p, V2=~p",[N1,V1,N2,V2]), - dump_terms(Config, "trace1.terms", Trace2), + dump_terms(Config, "trace1.terms", Trace1), dump_terms(Config, "trace2.terms", Trace2), %% Because of the inherit racy-ness of the feature it is @@ -348,15 +349,6 @@ dump_terms(Config, Name, Terms) -> file:write_file(FName, term_to_binary(Terms)), ct:log("Logged terms to ~s",[FName]). -infinite_loop(Die) -> - receive - Die -> - ok - after - 0 -> - infinite_loop(Die) - end. - read_packets_test(R, RP, Msgs, Node) -> Receiver = self(), Tracer = @@ -577,19 +569,19 @@ active_n(Config) when is_list(Config) -> recvtos(_Config) -> test_recv_opts( - inet, [{recvtos,tos,96}], + inet, [{recvtos,tos,96}], false, fun recvtos_ok/2). recvtosttl(_Config) -> test_recv_opts( - inet, [{recvtos,tos,96},{recvttl,ttl,33}], + inet, [{recvtos,tos,96},{recvttl,ttl,33}], false, fun (OSType, OSVer) -> recvtos_ok(OSType, OSVer) andalso recvttl_ok(OSType, OSVer) end). recvttl(_Config) -> test_recv_opts( - inet, [{recvttl,ttl,33}], + inet, [{recvttl,ttl,33}], false, fun recvttl_ok/2). recvtclass(_Config) -> @@ -601,15 +593,48 @@ recvtclass(_Config) -> of [_] -> test_recv_opts( - inet6, [{recvtclass,tclass,224}], + inet6, [{recvtclass,tclass,224}], false, fun recvtclass_ok/2); [] -> {skip,ipv6_not_supported,IFs} end. + +sendtos(_Config) -> + test_recv_opts( + inet, [{recvtos,tos,96}], true, + fun sendtos_ok/2). + +sendtosttl(_Config) -> + test_recv_opts( + inet, [{recvtos,tos,96},{recvttl,ttl,33}], true, + fun (OSType, OSVer) -> + sendtos_ok(OSType, OSVer) andalso sendttl_ok(OSType, OSVer) + end). + +sendttl(_Config) -> + test_recv_opts( + inet, [{recvttl,ttl,33}], true, + fun sendttl_ok/2). + +sendtclass(_Config) -> + {ok,IFs} = inet:getifaddrs(), + case + [Name || + {Name,Opts} <- IFs, + lists:member({addr,{0,0,0,0,0,0,0,1}}, Opts)] + of + [_] -> + test_recv_opts( + inet6, [{recvtclass,tclass,224}], true, + fun sendtclass_ok/2); + [] -> + {skip,ipv6_not_supported,IFs} + end. + %% These version numbers are just above the highest noted in daily tests %% where the test fails for a plausible reason, that is the lowest -%% where we can expect that the test mighe succeed, so +%% where we can expect that the test might succeed, so %% skip on platforms lower than this. %% %% On newer versions it might be fixed, but we'll see about that @@ -628,16 +653,55 @@ recvtos_ok({unix,sunos}, OSVer) -> not semver_lt(OSVer, {5,12,0}); recvtos_ok({unix,_}, _) -> true; recvtos_ok(_, _) -> false. +%% Option has no effect +recvttl_ok({unix,sunos}, OSVer) -> not semver_lt(OSVer, {5,12,0}); +%% recvttl_ok({unix,_}, _) -> true; recvttl_ok(_, _) -> false. %% Using the option returns einval, so it is not implemented. recvtclass_ok({unix,darwin}, OSVer) -> not semver_lt(OSVer, {9,9,0}); recvtclass_ok({unix,linux}, OSVer) -> not semver_lt(OSVer, {2,6,11}); +%% Option has no effect +recvtclass_ok({unix,sunos}, OSVer) -> not semver_lt(OSVer, {5,12,0}); %% recvtclass_ok({unix,_}, _) -> true; recvtclass_ok(_, _) -> false. + +%% To send ancillary data seems to require much higher version numbers +%% than receiving it... +%% + +%% Using the option returns einval, so it is not implemented. +sendtos_ok({unix,darwin}, OSVer) -> not semver_lt(OSVer, {19,0,0}); +sendtos_ok({unix,openbsd}, OSVer) -> not semver_lt(OSVer, {6,5,0}); +sendtos_ok({unix,sunos}, OSVer) -> not semver_lt(OSVer, {5,12,0}); +sendtos_ok({unix,linux}, OSVer) -> not semver_lt(OSVer, {4,0,0}); +sendtos_ok({unix,freebsd}, OSVer) -> not semver_lt(OSVer, {12,1,0}); +%% +sendtos_ok({unix,_}, _) -> true; +sendtos_ok(_, _) -> false. + +%% Using the option returns einval, so it is not implemented. +sendttl_ok({unix,darwin}, OSVer) -> not semver_lt(OSVer, {19,0,0}); +sendttl_ok({unix,linux}, OSVer) -> not semver_lt(OSVer, {4,0,0}); +%% Using the option returns enoprotoopt, so it is not implemented. +sendttl_ok({unix,freebsd}, OSVer) -> not semver_lt(OSVer, {12,1,0}); +%% Option has no effect +sendttl_ok({unix,openbsd}, OSVer) -> not semver_lt(OSVer, {6,5,0}); +%% +sendttl_ok({unix,_}, _) -> true; +sendttl_ok(_, _) -> false. + +%% Using the option returns einval, so it is not implemented. +sendtclass_ok({unix,darwin}, OSVer) -> not semver_lt(OSVer, {9,9,0}); +sendtclass_ok({unix,linux}, OSVer) -> not semver_lt(OSVer, {2,6,11}); +%% +sendtclass_ok({unix,_}, _) -> true; +sendtclass_ok(_, _) -> false. + + semver_lt({X1,Y1,Z1}, {X2,Y2,Z2}) -> if X1 > X2 -> false; @@ -650,18 +714,18 @@ semver_lt({X1,Y1,Z1}, {X2,Y2,Z2}) -> end; semver_lt(_, {_,_,_}) -> false. -test_recv_opts(Family, Spec, OSFilter) -> +test_recv_opts(Family, Spec, TestSend, OSFilter) -> OSType = os:type(), OSVer = os:version(), case OSFilter(OSType, OSVer) of true -> io:format("Os: ~p, ~p~n", [OSType,OSVer]), - test_recv_opts(Family, Spec, OSType, OSVer); + test_recv_opts(Family, Spec, TestSend, OSType, OSVer); false -> {skip,{not_supported_for_os_version,{OSType,OSVer}}} end. %% -test_recv_opts(Family, Spec, _OSType, _OSVer) -> +test_recv_opts(Family, Spec, TestSend, _OSType, _OSVer) -> Timeout = 5000, RecvOpts = [RecvOpt || {RecvOpt,_,_} <- Spec], TrueRecvOpts = [{RecvOpt,true} || {RecvOpt,_,_} <- Spec], @@ -686,16 +750,33 @@ test_recv_opts(Family, Spec, _OSType, _OSVer) -> ok = inet:setopts(S1, TrueRecvOpts_OptsVals), {ok,TrueRecvOpts_OptsVals} = inet:getopts(S1, RecvOpts ++ Opts), %% + %% S1 now has true receive options and set option values + %% {ok,S2} = gen_udp:open(0, [Family,binary,{active,true}|FalseRecvOpts]), {ok,P2} = inet:port(S2), {ok,FalseRecvOpts_OptsVals2} = inet:getopts(S2, RecvOpts ++ Opts), OptsVals2 = FalseRecvOpts_OptsVals2 -- FalseRecvOpts, %% - ok = gen_udp:send(S2, Addr, P1, <<"abcde">>), + %% S2 now has false receive options and default option values, + %% OptsVals2 contains the default option values + %% + ok = gen_udp:send(S2, {Addr,P1}, <<"abcde">>), ok = gen_udp:send(S1, Addr, P2, <<"fghij">>), + TestSend andalso + begin + ok = gen_udp:send(S2, Addr, P1, OptsVals, <<"ABCDE">>), + ok = gen_udp:send(S2, {Addr,P1}, OptsVals, <<"12345">>) + end, {ok,{_,P2,OptsVals3,<<"abcde">>}} = gen_udp:recv(S1, 0, Timeout), verify_sets_eq(OptsVals3, OptsVals2), + TestSend andalso + begin + {ok,{_,P2,OptsVals0,<<"ABCDE">>}} = gen_udp:recv(S1, 0, Timeout), + {ok,{_,P2,OptsVals1,<<"12345">>}} = gen_udp:recv(S1, 0, Timeout), + verify_sets_eq(OptsVals0, OptsVals), + verify_sets_eq(OptsVals1, OptsVals) + end, receive {udp,S2,_,P1,<<"fghij">>} -> ok; @@ -710,8 +791,16 @@ test_recv_opts(Family, Spec, _OSType, _OSVer) -> ok = inet:setopts(S2, TrueRecvOpts), {ok,TrueRecvOpts} = inet:getopts(S2, RecvOpts), %% - ok = gen_udp:send(S2, Addr, P1, <<"klmno">>), - ok = gen_udp:send(S1, Addr, P2, <<"pqrst">>), + %% S1 now has false receive options and set option values + %% + %% S2 now has true receive options and default option values + %% + ok = gen_udp:send(S2, {Addr,P1}, [], <<"klmno">>), + ok = gen_udp:send(S1, {Family,{loopback,P2}}, <<"pqrst">>), + TestSend andalso + begin + ok = gen_udp:send(S1, {Family,{loopback,P2}}, OptsVals2, <<"PQRST">>) + end, {ok,{_,P2,<<"klmno">>}} = gen_udp:recv(S1, 0, Timeout), receive {udp,S2,_,P1,OptsVals4,<<"pqrst">>} -> @@ -721,9 +810,18 @@ test_recv_opts(Family, Spec, _OSType, _OSVer) -> after Timeout -> exit(timeout) end, + TestSend andalso + receive + {udp,S2,_,P1,OptsVals5,<<"PQRST">>} -> + verify_sets_eq(OptsVals5, OptsVals2); + Other3 -> + exit({unexpected,Other3}) + after Timeout -> + exit(timeout) + end, ok = gen_udp:close(S1), ok = gen_udp:close(S2), -%% exit({{OSType,OSVer},success}), % In search for the truth +%%% exit({{_OSType,_OSVer},success}), % In search for the truth ok. verify_sets_eq(L1, L2) -> @@ -877,6 +975,10 @@ connect(Config) when is_list(Config) -> implicit_inet6(Config) when is_list(Config) -> Host = ok(inet:gethostname()), case inet:getaddr(Host, inet6) of + {ok,{16#fe80,0,0,0,_,_,_,_} = Addr} -> + {skip, + "Got link local IPv6 address: " + ++inet:ntoa(Addr)}; {ok,Addr} -> implicit_inet6(Host, Addr); {error,Reason} -> @@ -927,11 +1029,12 @@ ok({ok,V}) -> V; ok(NotOk) -> try throw(not_ok) catch - throw:Thrown:Stacktrace -> - erlang:raise( - error, {Thrown, NotOk}, tl(Stacktrace)) + throw:not_ok:Stacktrace -> + raise_error({not_ok, NotOk}, tl(Stacktrace)) end. +raise_error(Reason, Stacktrace) -> + erlang:raise(error, Reason, Stacktrace). local_filename(Tag) -> "/tmp/" ?MODULE_STRING "_" ++ os:getpid() ++ "_" ++ atom_to_list(Tag). diff --git a/lib/kernel/test/logger_std_h_SUITE.erl b/lib/kernel/test/logger_std_h_SUITE.erl index 16ab0e97fc..2b2d509860 100644 --- a/lib/kernel/test/logger_std_h_SUITE.erl +++ b/lib/kernel/test/logger_std_h_SUITE.erl @@ -132,6 +132,7 @@ all() -> bad_input, reconfig, file_opts, + relative_file_path, sync, write_failure, sync_failure, @@ -693,6 +694,54 @@ file_opts(Config) -> file_opts(cleanup, _Config) -> logger:remove_handler(?MODULE). +relative_file_path(_Config) -> + {ok,Dir} = file:get_cwd(), + AbsName1 = filename:join(Dir,?MODULE), + ok = logger:add_handler(?MODULE, + logger_std_h, + #{config => #{type=>file}, + filter_default=>log, + filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), + formatter=>{?MODULE,self()}}), + #{cb_state := #{handler_state := #{file:=AbsName1}}} = + logger_olp:info(h_proc_name()), + {ok,#{config := #{file:=AbsName1}}} = + logger:get_handler_config(?MODULE), + ok = logger:remove_handler(?MODULE), + + RelName2 = filename:join(atom_to_list(?FUNCTION_NAME), + lists:concat([?FUNCTION_NAME,".log"])), + AbsName2 = filename:join(Dir,RelName2), + ok = logger:add_handler(?MODULE, + logger_std_h, + #{config => #{file => RelName2}, + filter_default=>log, + filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), + formatter=>{?MODULE,self()}}), + #{cb_state := #{handler_state := #{file:=AbsName2}}} = + logger_olp:info(h_proc_name()), + {ok,#{config := #{file:=AbsName2}}} = + logger:get_handler_config(?MODULE), + logger:notice(M1=?msg,?domain), + ?check(M1), + B1 = ?bin(M1), + try_read_file(AbsName2, {ok,B1}, filesync_rep_int()), + + ok = file:set_cwd(".."), + logger:notice(M2=?msg,?domain), + ?check(M2), + B20 = ?bin(M2), + B2 = <<B1/binary,B20/binary>>, + try_read_file(AbsName2, {ok,B2}, filesync_rep_int()), + + {error,_} = logger:update_handler_config(?MODULE,config,#{file=>RelName2}), + ok = logger:update_handler_config(?MODULE,config,#{file=>AbsName2}), + ok = file:set_cwd(Dir), + ok = logger:update_handler_config(?MODULE,config,#{file=>RelName2}), + ok. +relative_file_path(cleanup,_Config) -> + logger:remove_handler(?MODULE). + sync(Config) -> Dir = ?config(priv_dir,Config), diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index 765e890157..e5188aa9b5 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 6.4 +KERNEL_VSN = 6.4.1 diff --git a/lib/public_key/asn1/CMSAesRsaesOaep.asn1 b/lib/public_key/asn1/CMSAesRsaesOaep.asn1 new file mode 100644 index 0000000000..ca8c7b7f92 --- /dev/null +++ b/lib/public_key/asn1/CMSAesRsaesOaep.asn1 @@ -0,0 +1,39 @@ +CMSAesRsaesOaep {iso(1) member-body(2) us(840) rsadsi(113549) + pkcs(1) pkcs-9(9) smime(16) modules(0) id-mod-cms-aes(19) } + + +DEFINITIONS IMPLICIT TAGS ::= +BEGIN + +-- EXPORTS ALL -- +IMPORTS + -- PKIX + AlgorithmIdentifier + FROM PKIX1Explicit88 {iso(1) identified-organization(3) dod(6) + internet(1) security(5) mechanisms(5) pkix(7) id-mod(0) + id-pkix1-explicit(18)}; + +-- AES information object identifiers -- + +aes OBJECT IDENTIFIER ::= { joint-iso-itu-t(2) country(16) us(840) + organization(1) gov(101) csor(3) nistAlgorithms(4) 1 } + +-- AES using CBC-chaining mode for key sizes of 128, 192, 256 + +id-aes128-CBC OBJECT IDENTIFIER ::= { aes 2 } +id-aes192-CBC OBJECT IDENTIFIER ::= { aes 22 } +id-aes256-CBC OBJECT IDENTIFIER ::= { aes 42 } + +-- AES-IV is a the parameter for all the above object identifiers. + +AES-IV ::= OCTET STRING (SIZE(16)) + + +-- AES Key Wrap Algorithm Identifiers - Parameter is absent + +id-aes128-wrap OBJECT IDENTIFIER ::= { aes 5 } +id-aes192-wrap OBJECT IDENTIFIER ::= { aes 25 } +id-aes256-wrap OBJECT IDENTIFIER ::= { aes 45 } + + +END diff --git a/lib/public_key/asn1/Makefile b/lib/public_key/asn1/Makefile index a920ea87ea..10952106c6 100644 --- a/lib/public_key/asn1/Makefile +++ b/lib/public_key/asn1/Makefile @@ -42,7 +42,7 @@ RELSYSDIR = $(RELEASE_PATH)/lib/public_key-$(VSN) ASN_TOP = OTP-PUB-KEY PKCS-FRAME ASN_MODULES = PKIX1Explicit88 PKIX1Implicit88 PKIX1Algorithms88 \ PKIXAttributeCertificate PKCS-1 PKCS-3 PKCS-7 PKCS-8 PKCS-10 PKCS5v2-0 OTP-PKIX \ - InformationFramework RFC5639 + InformationFramework RFC5639 CMSAesRsaesOaep ASN_ASNS = $(ASN_MODULES:%=%.asn1) ASN_ERLS = $(ASN_TOP:%=%.erl) ASN_HRLS = $(ASN_TOP:%=%.hrl) diff --git a/lib/public_key/asn1/OTP-PUB-KEY.set.asn b/lib/public_key/asn1/OTP-PUB-KEY.set.asn index b3f3ccdb77..7ab1684ff3 100644 --- a/lib/public_key/asn1/OTP-PUB-KEY.set.asn +++ b/lib/public_key/asn1/OTP-PUB-KEY.set.asn @@ -10,3 +10,5 @@ ECPrivateKey.asn1 PKCS-7.asn1 PKCS-10.asn1 RFC5639.asn1 +CMSAesRsaesOaep.asn1 + diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index d13c9a520a..57d9898661 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -87,6 +87,21 @@ </section> +<section><title>Public_Key 1.6.6.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Support Pasword based encryption with AES</p> + <p> + Own Id: OTP-15870 Aux Id: ERL-952 </p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 1.6.6</title> <section><title>Improvements and New Features</title> @@ -1247,4 +1262,3 @@ </chapter> - diff --git a/lib/public_key/doc/src/public_key_app.xml b/lib/public_key/doc/src/public_key_app.xml index 923a9f1dfb..5f2c50711a 100644 --- a/lib/public_key/doc/src/public_key_app.xml +++ b/lib/public_key/doc/src/public_key_app.xml @@ -51,6 +51,9 @@ Diffie-Hellman Key Agreement Standard </item> <item>Supports <url href="http://www.ietf.org/rfc/rfc2898.txt"> PKCS-5</url> - Password-Based Cryptography Standard </item> + <item>Supports <url href="http://www.ietf.org/rfc/fc3565.txt"> AES </url> - + Use of the Advanced Encryption Standard (AES) Algorithm in Cryptographic Message Syntax (CMS) + </item> <item>Supports <url href="http://www.ietf.org/rfc/rfc5208.txt"> PKCS-8</url> - Private-Key Information Syntax Standard</item> <item>Supports <url href="http://www.ietf.org/rfc/rfc5967.txt"> PKCS-10</url> - diff --git a/lib/public_key/src/pubkey_pbe.erl b/lib/public_key/src/pubkey_pbe.erl index 38b5c93521..6003bf21d0 100644 --- a/lib/public_key/src/pubkey_pbe.erl +++ b/lib/public_key/src/pubkey_pbe.erl @@ -26,9 +26,7 @@ -export([encode/4, decode/4, decrypt_parameters/1, encrypt_parameters/1]). -export([pbdkdf1/4, pbdkdf2/7]). --define(DEFAULT_SHA_MAC_KEYLEN, 20). -define(ASN1_OCTET_STR_TAG, 4). --define(IV_LEN, 8). %%==================================================================== %% Internal application API @@ -41,14 +39,23 @@ %%-------------------------------------------------------------------- encode(Data, Password, "DES-CBC" = Cipher, KeyDevParams) -> {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), - crypto:block_encrypt(des_cbc, Key, IV, pbe_pad(Data, KeyDevParams)); + crypto:block_encrypt(des_cbc, Key, IV, pbe_pad(Data, block_size(des_cbc))); encode(Data, Password, "DES-EDE3-CBC" = Cipher, KeyDevParams) -> {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), <<Key1:8/binary, Key2:8/binary, Key3:8/binary>> = Key, - crypto:block_encrypt(des3_cbc, [Key1, Key2, Key3], IV, pbe_pad(Data)); + crypto:block_encrypt(des3_cbc, [Key1, Key2, Key3], IV, pbe_pad(Data, block_size(des_3ede))); encode(Data, Password, "RC2-CBC" = Cipher, KeyDevParams) -> {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), - crypto:block_encrypt(rc2_cbc, Key, IV, pbe_pad(Data, KeyDevParams)). + crypto:block_encrypt(rc2_cbc, Key, IV, pbe_pad(Data, block_size(rc2_cbc))); +encode(Data, Password, "AES-128-CBC" = Cipher, KeyDevParams) -> + {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), + crypto:block_encrypt(aes_128_cbc, Key, IV, pbe_pad(Data, block_size(aes_128_cbc))); +encode(Data, Password, "AES-192-CBC" = Cipher, KeyDevParams) -> + {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), + crypto:block_encrypt(aes_192_cbc, Key, IV, pbe_pad(Data, block_size(aes_192_cbc))); +encode(Data, Password, "AES-256-CBC"= Cipher, KeyDevParams) -> + {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), + crypto:block_encrypt(aes_256_cbc, Key, IV, pbe_pad(Data, block_size(aes_256_cbc))). %%-------------------------------------------------------------------- -spec decode(binary(), string(), string(), term()) -> binary(). @@ -67,11 +74,13 @@ decode(Data, Password,"RC2-CBC"= Cipher, KeyDevParams) -> crypto:block_decrypt(rc2_cbc, Key, IV, Data); decode(Data, Password,"AES-128-CBC"= Cipher, KeyDevParams) -> {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), - crypto:block_decrypt(aes_cbc128, Key, IV, Data); -decode(Data, Password,"AES-256-CBC"= Cipher, KeyDevParams) -> + crypto:block_decrypt(aes_128_cbc, Key, IV, Data); +decode(Data, Password,"AES-192-CBC"= Cipher, KeyDevParams) -> + {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), + crypto:block_decrypt(aes_192_cbc, Key, IV, Data); +decode(Data, Password,"AES-256-CBC"= Cipher, KeyDevParams) -> {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), - crypto:block_decrypt(aes_cbc256, Key, IV, Data). - + crypto:block_decrypt(aes_256_cbc, Key, IV, Data). %%-------------------------------------------------------------------- -spec pbdkdf1(iodata(), iodata(), integer(), atom()) -> binary(). @@ -150,17 +159,15 @@ do_pbdkdf1(Prev, Count, Acc, Hash) -> Result = crypto:hash(Hash, Prev), do_pbdkdf1(Result, Count-1 , <<Result/binary, Acc/binary>>, Hash). -iv(#'PBES2-params_encryptionScheme'{algorithm = Algo, - parameters = ASN1IV}) - when (Algo == ?'desCBC') or - (Algo == ?'des-EDE3-CBC') -> - <<?ASN1_OCTET_STR_TAG, ?IV_LEN, IV:?IV_LEN/binary>> = decode_handle_open_type_wrapper(ASN1IV), - IV; iv(#'PBES2-params_encryptionScheme'{algorithm = ?'rc2CBC', parameters = ASN1IV}) -> {ok, #'RC2-CBC-Parameter'{iv = IV}} = 'PKCS-FRAME':decode('RC2-CBC-Parameter', decode_handle_open_type_wrapper(ASN1IV)), - iolist_to_binary(IV). + iolist_to_binary(IV); +iv(#'PBES2-params_encryptionScheme'{algorithm = _Algo, + parameters = ASN1IV}) -> + <<?ASN1_OCTET_STR_TAG, Len:8/unsigned-big-integer, IV:Len/binary>> = decode_handle_open_type_wrapper(ASN1IV), + IV. blocks(1, N, Index, Password, Salt, Count, Prf, PrfHash, PrfLen, Acc) -> <<XorSum:N/binary, _/binary>> = xor_sum(Password, Salt, Count, Index, Prf, PrfHash, PrfLen), @@ -217,17 +224,9 @@ pbe1_oid("RC2-CBC", md5) -> pbe1_oid("DES-CBC", md5) -> ?'pbeWithMD5AndDES-CBC'. -pbe_pad(Data, {#'PBEParameter'{}, _}) -> - pbe_pad(Data); -pbe_pad(Data, #'PBES2-params'{}) -> - pbe_pad(Data); -pbe_pad(Data, _) -> -pbe_pad(Data).%% Data. - - -pbe_pad(Data) -> - N = 8 - (erlang:byte_size(Data) rem 8), - Pad = list_to_binary(lists:duplicate(N, N)), +pbe_pad(Data, BlockSize) -> + N = BlockSize - (erlang:byte_size(Data) rem BlockSize), + Pad = binary:copy(<<N>>, N), <<Data/binary, Pad/binary>>. key_derivation_params(#'PBES2-params'{keyDerivationFunc = KeyDerivationFunc, @@ -249,11 +248,27 @@ key_derivation_params(#'PBES2-params'{keyDerivationFunc = KeyDerivationFunc, pseudo_random_function(#'PBKDF2-params_prf'{algorithm = {_,_, _,'id-hmacWithSHA1'}}) -> {fun crypto:hmac/4, sha, pseudo_output_length(?'id-hmacWithSHA1')}; -pseudo_random_function(#'PBKDF2-params_prf'{algorithm = ?'id-hmacWithSHA1'}) -> - {fun crypto:hmac/4, sha, pseudo_output_length(?'id-hmacWithSHA1')}. +pseudo_random_function(#'PBKDF2-params_prf'{algorithm = ?'id-hmacWithSHA1' = Algo}) -> + {fun crypto:hmac/4, sha, pseudo_output_length(Algo)}; +pseudo_random_function(#'PBKDF2-params_prf'{algorithm = ?'id-hmacWithSHA224'= Algo}) -> + {fun crypto:hmac/4, sha224, pseudo_output_length(Algo)}; +pseudo_random_function(#'PBKDF2-params_prf'{algorithm = ?'id-hmacWithSHA256' = Algo}) -> + {fun crypto:hmac/4, sha256, pseudo_output_length(Algo)}; +pseudo_random_function(#'PBKDF2-params_prf'{algorithm = ?'id-hmacWithSHA384' = Algo}) -> + {fun crypto:hmac/4, sha384, pseudo_output_length(Algo)}; +pseudo_random_function(#'PBKDF2-params_prf'{algorithm = ?'id-hmacWithSHA512' = Algo}) -> + {fun crypto:hmac/4, sha512, pseudo_output_length(Algo)}. pseudo_output_length(?'id-hmacWithSHA1') -> - ?DEFAULT_SHA_MAC_KEYLEN. + 20; %%160/8 +pseudo_output_length(?'id-hmacWithSHA224') -> + 28; %%%224/8 +pseudo_output_length(?'id-hmacWithSHA256') -> + 32; %%256/8 +pseudo_output_length(?'id-hmacWithSHA384') -> + 48; %%384/8 +pseudo_output_length(?'id-hmacWithSHA512') -> + 64. %%512/8 derived_key_length(_, Len) when is_integer(Len) -> Len; @@ -266,11 +281,33 @@ derived_key_length(Cipher,_) when (Cipher == ?'rc2CBC') or derived_key_length(Cipher,_) when (Cipher == ?'des-EDE3-CBC') or (Cipher == "DES-EDE3-CBC") -> 24; -derived_key_length(Cipher,_) when (Cipher == "AES-128-CBC") -> + +derived_key_length(Cipher,_) when (Cipher == "AES-128-CBC"); + (Cipher == ?'id-aes128-CBC') -> 16; -derived_key_length(Cipher,_) when (Cipher == "AES-256-CBC") -> +derived_key_length(Cipher,_) when (Cipher == "AES-192-CBC"); + (Cipher == ?'id-aes192-CBC') -> + 24; + +derived_key_length(Cipher,_) when (Cipher == "AES-256-CBC"); + (Cipher == ?'id-aes256-CBC') -> 32. +block_size(Cipher) when Cipher == rc2_cbc; + Cipher == des_cbc; + Cipher == des_3ede -> + 8; +block_size(Cipher) when Cipher == aes_128_cbc; + Cipher == aes_192_cbc; + Cipher == aes_256_cbc -> + 16. + +cipher(#'PBES2-params_encryptionScheme'{algorithm = ?'id-aes128-CBC'}) -> + "AES-128-CBC"; +cipher(#'PBES2-params_encryptionScheme'{algorithm = ?'id-aes192-CBC'}) -> + "AES-192-CBC"; +cipher(#'PBES2-params_encryptionScheme'{algorithm = ?'id-aes256-CBC'}) -> + "AES-256-CBC"; cipher(#'PBES2-params_encryptionScheme'{algorithm = ?'desCBC'}) -> "DES-CBC"; cipher(#'PBES2-params_encryptionScheme'{algorithm = ?'des-EDE3-CBC'}) -> diff --git a/lib/public_key/test/pbe_SUITE.erl b/lib/public_key/test/pbe_SUITE.erl index 1136267411..61db282dfa 100644 --- a/lib/public_key/test/pbe_SUITE.erl +++ b/lib/public_key/test/pbe_SUITE.erl @@ -206,7 +206,10 @@ pbes2() -> [{doc,"Tests encode/decode EncryptedPrivateKeyInfo encrypted with different ciphers using PBES2"}]. pbes2(Config) when is_list(Config) -> decode_encode_key_file("pbes2_des_cbc_enc_key.pem", "password", "DES-CBC", Config), - decode_encode_key_file("pbes2_des_ede3_cbc_enc_key.pem", "password", "DES-EDE3-CBC", Config), + decode_encode_key_file("pbes2_des_ede3_cbc_enc_key.pem", "password", "DES-EDE3-CBC", Config), + decode_encode_key_file("pbes2_aes_128_enc_key.pem", "password", "AES-128-CBC", Config), + decode_encode_key_file("pbes2_aes_192_enc_key.pem", "password", "AES-192-CBC", Config), + decode_encode_key_file("pbes2_aes_256_enc_key.pem", "password", "AES-256-CBC", Config), case lists:member(rc2_cbc, proplists:get_value(ciphers, crypto:supports())) of true -> decode_encode_key_file("pbes2_rc2_cbc_enc_key.pem", "password", "RC2-CBC", Config); @@ -239,7 +242,6 @@ decode_encode_key_file(File, Password, Cipher, Config) -> {ok, PemKey} = file:read_file(filename:join(Datadir, File)), PemEntry = public_key:pem_decode(PemKey), - ct:pal("Pem entry: ~p" , [PemEntry]), [{Asn1Type, _, {Cipher,_} = CipherInfo} = PubEntry] = PemEntry, #'RSAPrivateKey'{} = KeyInfo = public_key:pem_entry_decode(PubEntry, Password), PemKey1 = public_key:pem_encode([public_key:pem_entry_encode(Asn1Type, KeyInfo, {CipherInfo, Password})]), diff --git a/lib/public_key/test/pbe_SUITE_data/pbes2_aes_128_enc_key.pem b/lib/public_key/test/pbe_SUITE_data/pbes2_aes_128_enc_key.pem new file mode 100644 index 0000000000..5702119ad6 --- /dev/null +++ b/lib/public_key/test/pbe_SUITE_data/pbes2_aes_128_enc_key.pem @@ -0,0 +1,30 @@ +-----BEGIN ENCRYPTED PRIVATE KEY----- +MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQIWrPgmqJqNpICAggA +MAwGCCqGSIb3DQIJBQAwHQYJYIZIAWUDBAECBBA/bbIMYqQMUDxMk9ifPR7ABIIE +0Drfqke1/ccFxk786hTh36yjVo48Xx7B3Scb92KtmyQpNaR6GbR+jhP9cxIcvmGN +YroCB896VJSIx8PraqGgIJ1hblZXyfLanB0mUnZvaaQ4xp3UJT53a0yOm5Lfd+fB +0TyaoEzca2jA5EVVh3yH6gzNsvQJRw6cQP5CAptLjiUv2jrwVGnO8x8X4egJDLZS +Sb8B5AW8h1sGsyKEEFto6gpBjVqnVn5veMoI/Cfs9qDr071+dhbps/m6pseKKp0z +8qeFM7+9Y4npD1VYg2gqOFi19QAI3gwq6tC8grOzRA8dPFUgpV9eMToVsI2OFQc1 +xnFZEV7NZVymh5HjKM1jwFy6es+5TFoMtRu6vDxKS6Y13lIlZ4oQSh8aXtG5Ylt2 +CqsKNHyDbZUpvKe/k19TBmVXQBCYFuN733jI9/4JBtpygnxwt1aXCvq/PFFGsTS4 +p1JOQvr/jaD7b4JO6IMXH1kSVxiMXKXNG7wPUNr6OWJvc7OqdclsZa7ibEx4L52x +DuFmsxQo4a3iibhbcjr436OmR5Uw2UAstB5qxWfMhkt+e7rRhCOh/3O7SAYEpt+f +Zr2VFXdGme4kR6uMCzgGiSh0qCseQXpJUZVufn/Go9r+601OJTJIQ9a2VoqlMR8o +Dd14D0gBXXaZkY60Mh8iXR/MjKDuv0KBUyBzfcpk3fLmv0PhGSkbn6j+q1jZbogm +EhI0AL5s2EoofuBdvgdusBhCrrwCMonprqR7BuaKPD0GEw5utnT5ovcUg/sjMJox +10100QwAzQScU4iG/xic/TsN+ZMumhUcYs003MsZkRLvCEFxZurEMx7819CqfhIc +NGd7ETTBSwoNf5pXRTHaTbW6pPiIeWunLUUVsRcNoBtL/cXmg+mu1zdsD7nD51mJ +vG9A7LPW7XVl2Jv2NgQoKkHYO7cVozmcz6AE2z1q+XN4LGto8JEZktb6E7UIyXXg +Ls4Tv0sn5TLgtaJ31w4+9iybNiGoVYOc4h0s5DoNR4ivcZ6n/Qnf8PTrNzejEJY6 +R/UnDbc24u0palGc1kei99d0BYodnq4OlAj7M7ML0GncftInhgA0Dp81YG5PujMa +irhvwtnD5Xysfh1YrroAEN7Qxc8+2JlpgNSFlFFkMgfibc6jvTX6/C6MaFz8hiOq +W43ZBEzjMIs23ZrJKOJGsuTdHSob+VbvqIMgS2PeGb/6g3/GjdipCbynNhX3zUOM +3j/lpZOiAwE/Bftr5FOSfTFpnyorIIeyWgROEZTTL4eSYvnBjzf+tUdXY7ltxJie +q0rpQ42X7+B4gTo8Qj/xC7LXSCldERK57cCwwITvjcHwxPyOiJ9BMI1HlRQ/Fo3C +lPYIst1xjJ67qrTm6mWkor2hUOZcg4MOOzXWuijWRGJ/Wz0H+GKWtoE2X536D6sy +a4Nwwj09oFY4Fph/SUNwy0MLpTSzikpUx6mxjbs3Odvo6tWWVcicp/dCWYCqLpGU +3axEb/qlsaRNtKJg9O3Fq7hh1BTyLNGB2ET5wSKtlSD0bDeF15bBvkHB3z2/lDls +YQ2hEHMjeSEZZyGTPqEHwtBuUwiWBBXwOIhT8nfYXbHWR0CLBLth2+E/JCaO9hD2 +V277arqNFa8nugZMwS+ragi6vbgIX4BiS/rnfYXgqaxD +-----END ENCRYPTED PRIVATE KEY----- diff --git a/lib/public_key/test/pbe_SUITE_data/pbes2_aes_192_enc_key.pem b/lib/public_key/test/pbe_SUITE_data/pbes2_aes_192_enc_key.pem new file mode 100644 index 0000000000..ee82e9f667 --- /dev/null +++ b/lib/public_key/test/pbe_SUITE_data/pbes2_aes_192_enc_key.pem @@ -0,0 +1,30 @@ +-----BEGIN ENCRYPTED PRIVATE KEY----- +MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQIcqBCM7v+ZlkCAggA +MAwGCCqGSIb3DQIJBQAwHQYJYIZIAWUDBAEWBBD93r4IWBhvry+cdfwIDOKeBIIE +0DXM8S70sMsUmwxRZQtKwGfYddEWIc9lrEdsgEEuonF6NrseRq7QdXnBSPwq5f0O +ofMZ/0OCun3Qg1ls1EdsyKdijSOq27ZhHCnmWi1Rw1ApJIAq5i/jY8U17+lUakvG +VtcsuRzlKmFxbBW44kLK7vK6xiA76HPx0I4ZXcdywR0pbLT1ubbhbQ9djLnBiYkT +odszGTyxNceEse1Hu/RhFK17tnwov0fdioKY2i9F7qfq8lYLPrusEKTY7tOVjFOh +bXeCry1BL0KTt65JVGR9xQCI0qokEU0QrCgD6skq7Vx2C/Ho1sW6h8FBFVIm6ozO +bEUtVk3Xgs5yieetha1GxJAang1VxAPemnXfOmVapoSgSv1BQyDdnk3067Sfkh64 +A5yf44BUjvJsSd/ViCVmCryoXU7KOMAdFkyRSiDDLQus6bZGEhc6f+VEikG+TZ2L +xxY4OucE2Bz67S6ycyOUpXKo0+FW0juE6NTJdlYSXWOvfciZKA83h6yAej6MfUEu +4orIvnCTVO7i3+hHybnSgftj42jrqqZzeXll8rkGHg4syrKRVaDD6qfJjgAHBJkJ +pZT4zZwuJ1puWfBykI25S4mKUnk0erq4N5jpGqdm7U14fWBWCjZN85jY4WgZZOJx +kBNO2NbmZKzZEzRGyMJ563z4l7MNfzZBHv+FeBNkX146J4ZhMbT8IXPGV9peNWqu +mY2B9RhN4hlDrd3Hfz5uiiF3UGrFkDcsPRBHWGqQ20YpuOQNno7iL8N0FWauERw1 +dvxAGVwFfUznR3wc/eyGcnRhqQhlYPspukh0IVIyEbre3yVFSG/41GQYQfg08XYd +LYiiDUu1i515/GeDvYN5VcnZ4nMhPgqfxW4rEUZjI86p++bqwqGy8eOCivkzGV3A +IFWQwlvKKzU7tSdi3uHUq5v7xQsJrALdf67JVjCCGfUZa17O41vmm58L/vKhhL2Y +mLz/H004DPsB+CtWoLwqZ8Jmb1EHwqNbna3tGHn3n63j2cV7gykZFa/zXeuBbbJ/ +t4ZIojIEzwAVKA9Xzcl3wyGCRr62WJPEcOqe4kBYREuKd22juPEm9RQgciIIj0tP +eJVpD0QarGGzERsaq7pheAiWisO+Q4cLjF8Mb3/r89abnd4AQk6meabFJIE2dXWp +LZy3I6FkNQ7L7LxNOILhnaWzWGdOBVwHeAAxfbLOzM22ewj7oUwBCRpsBJ8zl2PL +VhUjX6N26YoiR9gE1RBaVrwRkYLmkyGvrowCDoZVPxvJqbfIESQE42zGB9DbEPNp +WXCnzAg5cIjNC31We274yLE7dpNPVRXPJCRhtp7noorWVzDdKB+dFvg08bIir6Vj +1gxy8DvuZE1Gq9vqx38V7Cy2MrSpsgapw5mli4n5cMafE7Ty3j5pBJFF2f3jUn6B +7MjCrKp1d8v6MEy18J/Ugu1Lytb92LMcNtWBKmqyCSxekrUB9/FC2hWqOpdwRI6q +QMWkwshjyEhmlr2PAkBPM4uVzUFc9lBw1GzOUChkr9jiINdbsUSRJrwZ32Nc3gRY +yKzWbEELPSgRcXwXgH3QqZukvmk2tBMTIxilXqKTLmd7t/AEnIhkbqC0pfnyChyU +YlFkme0RpAXpgbDJgv+Vk+1/1s6gyaNSzT4s2Q340WIO +-----END ENCRYPTED PRIVATE KEY----- diff --git a/lib/public_key/test/pbe_SUITE_data/pbes2_aes_256_enc_key.pem b/lib/public_key/test/pbe_SUITE_data/pbes2_aes_256_enc_key.pem new file mode 100644 index 0000000000..050337aead --- /dev/null +++ b/lib/public_key/test/pbe_SUITE_data/pbes2_aes_256_enc_key.pem @@ -0,0 +1,30 @@ +-----BEGIN ENCRYPTED PRIVATE KEY----- +MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQI4MxgpDiHxQcCAggA +MAwGCCqGSIb3DQIJBQAwHQYJYIZIAWUDBAEqBBA2g/L8XmlK2axDkeYJCltnBIIE +0C3+NQ93DzEK/9qicy1sj0Vag1M7AeJjTGGpatETCxM+eHjk4kNNeDeMV5+EmCSu +Db4P48uvHOBGGCcqdjnQovfQsAh81GWxgF3yqpd4OKn2RubMLO4/Qu+zGtt/XRKz +T0pyHHBu6hyPSOhad2SIjKWuaHepwxGYaejLP83sy6yhm0sEmyBUn4nGSTOROcqR +wd7EbwU2PYUcrRGGxtChU7MUNt48wBO50Xmri1ssPPtZV6MHio4IoIz4hqzCjvAc +VE1BqAvNIJ7icpdnL8Jqq0lfwEmGjFCkAjgov5fNW9I1b44jE2Tv5LM2urMH8InQ +9qNjTHozYQhHAk9nX4cmMgHsIhkOd7Z2M+nz8Hd1tj9DmBNOr5XbfyctgVntaMB4 +GGnThuNlX8d5giOKOcaNPMpLU1jtfDcb73mEhwCYcdo1PM0rjrYZ7qetjXJW/oHs +Nl/hIZIRpMuCRVuXHml4G+ziKbMnXUN8sbtvgkQatYFHFQOhAqZeyzWp8SlDcfqb +Zt0LlZVJEhKUYzZgKoe7SmR1rXTTCfYeB75PddyYwVgf/IkT6HJ/y1apGOP6/UJ8 +7UV6zssQA35gMsYDT36sH2hAQvA/cOFxSxrip0gm0xXOeFF0gbyZWbFqk0aULaeF +rbBoMe28akxdE4eD06b+TP2NguUGP72l3TPOlG4PQVScweMw9L3oPXOVj4Vbbd0y +DenNvRHlWIwOh/y7ADTHSWq9CE45QDBvFaTcn43JQWD8xCmhAhI/9H+fhAQUhABm +P5QoJLE2IGo8A+Gi7rfgYQb3fCgqcn8azsRJzozhE+oXxMvxEESejYTtm26FNmLg +ONTWysF9BiaKHt2IXwRX97691wZqv5wJEaxeeJxfVQ6MlAHoEDXe49VxGN4zFXuq +Yb71JdQDgM94jwc/PoUwFH2ALSkIciiKwU0xfFpptycl4qWpy9m7QTIKw0DjgCfg +MuySPRGM5jn3yVg72ux2Qf9MKNEybWjZ+Se9MJ1IZmZK5eOo6L2JsFCc0nRn908E +vn4gAgUfMxyCZ1ygXfxINVAixR+6KPHsz1QTIxTZkrlnXRsuEu1ZfBSHzmXESvJo +3I9PkP/Iekg1FBpB5xxd7mXwCj17EWqYXWsLnfd8SblMjRYd64q7hfx0oU/MJ1wi +KadkGcyAGVRyleJRBR0LleYj/2sDihrRQY4zu5UtzSMFMH0XWjSWk5+ZQb+z3iDc +Ud4GHcHiuTMH+i03ApZGWLN9v93za/15fsnZogstgJkaHxizTz5JuCkRf15xd8+O +EH77Tsfizjp+h2NF/wcr4OSD0i+H0mwZWajpZ3UmSeJ0BFK6ODEbmVycrInpHo3n +zyMJnEDTJXL3HUwZSLjO5e5cNaB+75tdHrj2yJtRLuaJFr02b0EO1MUYfuUuqlK4 +7mg7FkBsimW+CXkoLRjHYK88ibT3G+rZ/STf4S/jxiRjBi06FAql3H02K5i1umgB +0BaaQei0Z8wQxMeTEnGzL+OcJeqDA1ZRFeXe7DNGsX1jeTYKPHA/Dr2IdZqyiCr2 +xh6e7RJuUe4D2liXW8LlMdwhN/7xSinA031PgBmb8XzSRmfdHhytFkA8PiM5T2ew +NR3qXBJ/G7BuRa/t26RuKI3BMVoBQPhGx80ds10uJjxq +-----END ENCRYPTED PRIVATE KEY----- diff --git a/lib/ssh/src/Makefile b/lib/ssh/src/Makefile index 6d64a45112..9627b70eeb 100644 --- a/lib/ssh/src/Makefile +++ b/lib/ssh/src/Makefile @@ -99,7 +99,7 @@ APP_TARGET= $(EBIN)/$(APP_FILE) APPUP_SRC= $(APPUP_FILE).src APPUP_TARGET= $(EBIN)/$(APPUP_FILE) -INTERNAL_HRL_FILES = ssh_auth.hrl ssh_connect.hrl ssh_transport.hrl ssh.hrl ssh_userauth.hrl ssh_xfer.hrl +INTERNAL_HRL_FILES = ssh_auth.hrl ssh_connect.hrl ssh_transport.hrl ssh.hrl ssh_xfer.hrl # ---------------------------------------------------- # FLAGS diff --git a/lib/ssh/src/ssh.hrl b/lib/ssh/src/ssh.hrl index 04453e6ef0..54e98ee10e 100644 --- a/lib/ssh/src/ssh.hrl +++ b/lib/ssh/src/ssh.hrl @@ -68,6 +68,25 @@ -define(string(X), ?string_utf8(X)). -define(binary(X), << ?STRING(X) >>). +-define('2bin'(X), (if is_binary(X) -> X; + is_list(X) -> list_to_binary(X); + X==undefined -> <<>> + end) ). + +%% encoding macros +-define('E...'(X), ?'2bin'(X)/binary ). +-define(Eboolean(X), ?BOOLEAN(case X of + true -> ?TRUE; + false -> ?FALSE + end) ). +-define(Ebyte(X), ?BYTE(X) ). +-define(Euint32(X), ?UINT32(X) ). +-define(Estring(X), ?STRING(?'2bin'(X)) ). +-define(Estring_utf8(X), ?string_utf8(X)/binary ). +-define(Ename_list(X), ?STRING(ssh_bits:name_list(X)) ). +-define(Empint(X), (ssh_bits:mpint(X))/binary ). +-define(Ebinary(X), ?STRING(X) ). + %% Cipher details -define(SSH_CIPHER_NONE, 0). -define(SSH_CIPHER_3DES, 3). diff --git a/lib/ssh/src/ssh_message.erl b/lib/ssh/src/ssh_message.erl index d95e58c1bb..7c86a81108 100644 --- a/lib/ssh/src/ssh_message.erl +++ b/lib/ssh/src/ssh_message.erl @@ -34,24 +34,6 @@ -export([dbg_trace/3]). --define('2bin'(X), (if is_binary(X) -> X; - is_list(X) -> list_to_binary(X); - X==undefined -> <<>> - end) ). - --define('E...'(X), ?'2bin'(X)/binary ). --define(Eboolean(X), ?BOOLEAN(case X of - true -> ?TRUE; - false -> ?FALSE - end) ). --define(Ebyte(X), ?BYTE(X) ). --define(Euint32(X), ?UINT32(X) ). --define(Estring(X), ?STRING(?'2bin'(X)) ). --define(Estring_utf8(X), ?string_utf8(X)/binary ). --define(Ename_list(X), ?STRING(ssh_bits:name_list(X)) ). --define(Empint(X), (ssh_bits:mpint(X))/binary ). --define(Ebinary(X), ?STRING(X) ). - ucl(B) -> try unicode:characters_to_list(B) of L when is_list(L) -> L; diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index eaab13433a..a85926354e 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -61,14 +61,6 @@ -export([pack/3, adjust_algs_for_peer_version/2]). -export([decompress/2, decrypt_blocks/3, is_valid_mac/3 ]). % FIXME: remove --define(Estring(X), ?STRING((if is_binary(X) -> X; - is_list(X) -> list_to_binary(X); - X==undefined -> <<>> - end))). --define(Empint(X), (ssh_bits:mpint(X))/binary ). --define(Ebinary(X), ?STRING(X) ). --define(Euint32(X), ?UINT32(X) ). - %%%---------------------------------------------------------------------------- %%% %%% There is a difference between supported and default algorithms. The diff --git a/lib/ssh/src/ssh_userauth.hrl b/lib/ssh/src/ssh_userauth.hrl deleted file mode 100644 index 2cfc1f0f83..0000000000 --- a/lib/ssh/src/ssh_userauth.hrl +++ /dev/null @@ -1,78 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. -%% -%% Licensed under the Apache License, Version 2.0 (the "License"); -%% you may not use this file except in compliance with the License. -%% You may obtain a copy of the License at -%% -%% http://www.apache.org/licenses/LICENSE-2.0 -%% -%% Unless required by applicable law or agreed to in writing, software -%% distributed under the License is distributed on an "AS IS" BASIS, -%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -%% See the License for the specific language governing permissions and -%% limitations under the License. -%% -%% %CopyrightEnd% -%% - -%% - -%%% Description: user authentication protocol - --define(SSH_MSG_USERAUTH_REQUEST, 50). --define(SSH_MSG_USERAUTH_FAILURE, 51). --define(SSH_MSG_USERAUTH_SUCCESS, 52). --define(SSH_MSG_USERAUTH_BANNER, 53). --define(SSH_MSG_USERAUTH_PK_OK, 60). --define(SSH_MSG_USERAUTH_PASSWD_CHANGEREQ, 60). --define(SSH_MSG_USERAUTH_INFO_REQUEST, 60). --define(SSH_MSG_USERAUTH_INFO_RESPONSE, 61). - --record(ssh_msg_userauth_request, - { - user, %% string - service, %% string - method, %% string "publickey", "password" - data %% opaque - }). - --record(ssh_msg_userauth_failure, - { - authentications, %% string - partial_success %% boolean - }). - --record(ssh_msg_userauth_success, - { - }). - --record(ssh_msg_userauth_banner, - { - message, %% string - language %% string - }). - --record(ssh_msg_userauth_passwd_changereq, - { - prompt, %% string - languge %% string - }). - --record(ssh_msg_userauth_pk_ok, - { - algorithm_name, % string - key_blob % string - }). - --record(ssh_msg_userauth_info_request, - {name, - instruction, - language_tag, - num_prompts, - data}). --record(ssh_msg_userauth_info_response, - {num_responses, - data}). diff --git a/lib/ssh/test/ssh_bench_SUITE.erl b/lib/ssh/test/ssh_bench_SUITE.erl index 880c519a5e..5ff7a71c45 100644 --- a/lib/ssh/test/ssh_bench_SUITE.erl +++ b/lib/ssh/test/ssh_bench_SUITE.erl @@ -26,7 +26,7 @@ -include_lib("ssh/src/ssh.hrl"). -include_lib("ssh/src/ssh_transport.hrl"). -include_lib("ssh/src/ssh_connect.hrl"). --include_lib("ssh/src/ssh_userauth.hrl"). +-include_lib("ssh/src/ssh_auth.hrl"). %%%================================================================ %%% diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 01cd630668..f320b4c006 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -27,6 +27,66 @@ </header> <p>This document describes the changes made to the SSL application.</p> +<section><title>SSL 9.3.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Correct handshake handling, might cause strange symptoms + such as ASN.1 certificate decoding issues.</p> + <p> + Own Id: OTP-15879 Aux Id: ERL-968 </p> + </item> + <item> + <p> + Fix handling of the signature_algorithms_cert extension + in the ClientHello handshake message.</p> + <p> + Own Id: OTP-15887 Aux Id: ERL-973 </p> + </item> + <item> + <p> + Handle new ClientHello extensions when handshake is + paused by the {handshake, hello} ssl option.</p> + <p> + Own Id: OTP-15888 Aux Id: ERL-975 </p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 9.3.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Returned "alert error string" is now same as logged alert + string</p> + <p> + Own Id: OTP-15844</p> + </item> + <item> + <p> + Fix returned extension map fields to follow the + documentation.</p> + <p> + Own Id: OTP-15862 Aux Id: ERL-951 </p> + </item> + <item> + <p> + Avoid DTLS crash due to missing gen_server return value + in DTLS packet demux process.</p> + <p> + Own Id: OTP-15864 Aux Id: ERL-962 </p> + </item> + </list> + </section> + +</section> + <section><title>SSL 9.3.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -119,6 +179,38 @@ </section> +<section><title>SSL 9.2.3.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Correct handshake handling, might cause strange symptoms + such as ASN.1 certificate decoding issues.</p> + <p> + Own Id: OTP-15879 Aux Id: ERL-968 </p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 9.2.3.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Returned "alert error string" is now same as logged alert + string</p> + <p> + Own Id: OTP-15844</p> + </item> + </list> + </section> + +</section> + <section><title>SSL 9.2.3.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssl/doc/src/standards_compliance.xml b/lib/ssl/doc/src/standards_compliance.xml index 3bd86178c8..3a472d4776 100644 --- a/lib/ssl/doc/src/standards_compliance.xml +++ b/lib/ssl/doc/src/standards_compliance.xml @@ -340,8 +340,8 @@ <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">application_layer_protocol_negotiation (RFC7301)</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>22.1</em></cell> </row> <row> <cell align="left" valign="middle"></cell> @@ -479,8 +479,8 @@ <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">application_layer_protocol_negotiation (RFC7301)</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>22.1</em></cell> </row> <row> <cell align="left" valign="middle"></cell> diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index 6928d7a93d..b220691e79 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -836,9 +836,12 @@ initial_flight_state(_) -> next_dtls_record(Data, StateName, #state{protocol_buffers = #protocol_buffers{ dtls_record_buffer = Buf0, dtls_cipher_texts = CT0} = Buffers, + connection_env = #connection_env{negotiated_version = Version}, + static_env = #static_env{data_tag = DataTag}, ssl_options = SslOpts} = State0) -> case dtls_record:get_dtls_records(Data, - acceptable_record_versions(StateName, State0), + {DataTag, StateName, Version, + [dtls_record:protocol_version(Vsn) || Vsn <- ?ALL_AVAILABLE_DATAGRAM_VERSIONS]}, Buf0, SslOpts) of {Records, Buf1} -> CT1 = CT0 ++ Records, @@ -849,10 +852,6 @@ next_dtls_record(Data, StateName, #state{protocol_buffers = #protocol_buffers{ Alert end. -acceptable_record_versions(hello, _) -> - [dtls_record:protocol_version(Vsn) || Vsn <- ?ALL_AVAILABLE_DATAGRAM_VERSIONS]; -acceptable_record_versions(_, #state{connection_env = #connection_env{negotiated_version = Version}}) -> - [Version]. dtls_handshake_events(Packets) -> lists:map(fun(Packet) -> diff --git a/lib/ssl/src/dtls_packet_demux.erl b/lib/ssl/src/dtls_packet_demux.erl index c6431b55a9..94b350eaa5 100644 --- a/lib/ssl/src/dtls_packet_demux.erl +++ b/lib/ssl/src/dtls_packet_demux.erl @@ -154,9 +154,9 @@ handle_info({Transport, Socket, IP, InPortNo, _} = Msg, #state{listener = Socket handle_info({PassiveTag, Socket}, #state{active_n = N, listener = Socket, - transport = {_,_,_, udp_error, PassiveTag}}) -> - next_datagram(Socket, N); - + transport = {_, _, _, _, PassiveTag}} = State) -> + next_datagram(Socket, N), + {noreply, State}; %% UDP socket does not have a connection and should not receive an econnreset %% This does however happens on some windows versions. Just ignoring it %% appears to make things work as expected! diff --git a/lib/ssl/src/dtls_record.erl b/lib/ssl/src/dtls_record.erl index a4846f42c5..8b8db7b2de 100644 --- a/lib/ssl/src/dtls_record.erl +++ b/lib/ssl/src/dtls_record.erl @@ -162,26 +162,16 @@ current_connection_state_epoch(#{current_write := #{epoch := Epoch}}, Epoch. %%-------------------------------------------------------------------- --spec get_dtls_records(binary(), [ssl_record:ssl_version()], binary(), +-spec get_dtls_records(binary(), {atom(), atom(), ssl_record:ssl_version(), [ssl_record:ssl_version()]}, binary(), #ssl_options{}) -> {[binary()], binary()} | #alert{}. %% %% Description: Given old buffer and new data from UDP/SCTP, packs up a records %% and returns it as a list of tls_compressed binaries also returns leftover %% data %%-------------------------------------------------------------------- -get_dtls_records(Data, Versions, Buffer, SslOpts) -> +get_dtls_records(Data, Vinfo, Buffer, SslOpts) -> BinData = list_to_binary([Buffer, Data]), - case erlang:byte_size(BinData) of - N when N >= 3 -> - case assert_version(BinData, Versions) of - true -> - get_dtls_records_aux(BinData, [], SslOpts); - false -> - ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) - end; - _ -> - get_dtls_records_aux(BinData, [], SslOpts) - end. + get_dtls_records_aux(Vinfo, BinData, [], SslOpts). %%==================================================================== %% Encoding DTLS records @@ -405,52 +395,49 @@ initial_connection_state(ConnectionEnd, BeastMitigation) -> client_verify_data => undefined, server_verify_data => undefined }. -assert_version(<<?BYTE(_), ?BYTE(MajVer), ?BYTE(MinVer), _/binary>>, Versions) -> - is_acceptable_version({MajVer, MinVer}, Versions). -get_dtls_records_aux(<<?BYTE(?APPLICATION_DATA),?BYTE(MajVer),?BYTE(MinVer), - ?UINT16(Epoch), ?UINT48(SequenceNumber), - ?UINT16(Length), Data:Length/binary, Rest/binary>> = RawDTLSRecord, - Acc, SslOpts) -> - ssl_logger:debug(SslOpts#ssl_options.log_level, inbound, 'record', [RawDTLSRecord]), - get_dtls_records_aux(Rest, [#ssl_tls{type = ?APPLICATION_DATA, - version = {MajVer, MinVer}, - epoch = Epoch, sequence_number = SequenceNumber, - fragment = Data} | Acc], SslOpts); -get_dtls_records_aux(<<?BYTE(?HANDSHAKE),?BYTE(MajVer),?BYTE(MinVer), - ?UINT16(Epoch), ?UINT48(SequenceNumber), - ?UINT16(Length), - Data:Length/binary, Rest/binary>> = RawDTLSRecord, - Acc, SslOpts) when MajVer >= 128 -> - ssl_logger:debug(SslOpts#ssl_options.log_level, inbound, 'record', [RawDTLSRecord]), - get_dtls_records_aux(Rest, [#ssl_tls{type = ?HANDSHAKE, - version = {MajVer, MinVer}, - epoch = Epoch, sequence_number = SequenceNumber, - fragment = Data} | Acc], SslOpts); -get_dtls_records_aux(<<?BYTE(?ALERT),?BYTE(MajVer),?BYTE(MinVer), - ?UINT16(Epoch), ?UINT48(SequenceNumber), - ?UINT16(Length), Data:Length/binary, - Rest/binary>> = RawDTLSRecord, Acc, SslOpts) -> +get_dtls_records_aux({DataTag, StateName, _, Versions} = Vinfo, <<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer), + ?UINT16(Epoch), ?UINT48(SequenceNumber), + ?UINT16(Length), Data:Length/binary, Rest/binary>> = RawDTLSRecord, + Acc, SslOpts) when ((StateName == hello) orelse + ((StateName == certify) andalso (DataTag == udp)) orelse + ((StateName == abbreviated) andalso(DataTag == udp))) + andalso + ((Type == ?HANDSHAKE) orelse + (Type == ?ALERT)) -> ssl_logger:debug(SslOpts#ssl_options.log_level, inbound, 'record', [RawDTLSRecord]), - get_dtls_records_aux(Rest, [#ssl_tls{type = ?ALERT, - version = {MajVer, MinVer}, - epoch = Epoch, sequence_number = SequenceNumber, - fragment = Data} | Acc], SslOpts); -get_dtls_records_aux(<<?BYTE(?CHANGE_CIPHER_SPEC),?BYTE(MajVer),?BYTE(MinVer), + case is_acceptable_version({MajVer, MinVer}, Versions) of + true -> + get_dtls_records_aux(Vinfo, Rest, [#ssl_tls{type = Type, + version = {MajVer, MinVer}, + epoch = Epoch, sequence_number = SequenceNumber, + fragment = Data} | Acc], SslOpts); + false -> + ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) + end; +get_dtls_records_aux({_, _, Version, _} = Vinfo, <<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Epoch), ?UINT48(SequenceNumber), ?UINT16(Length), Data:Length/binary, Rest/binary>> = RawDTLSRecord, - Acc, SslOpts) -> + Acc, SslOpts) when (Type == ?APPLICATION_DATA) orelse + (Type == ?HANDSHAKE) orelse + (Type == ?ALERT) orelse + (Type == ?CHANGE_CIPHER_SPEC) -> ssl_logger:debug(SslOpts#ssl_options.log_level, inbound, 'record', [RawDTLSRecord]), - get_dtls_records_aux(Rest, [#ssl_tls{type = ?CHANGE_CIPHER_SPEC, - version = {MajVer, MinVer}, - epoch = Epoch, sequence_number = SequenceNumber, - fragment = Data} | Acc], SslOpts); -get_dtls_records_aux(<<?BYTE(_), ?BYTE(_MajVer), ?BYTE(_MinVer), + case {MajVer, MinVer} of + Version -> + get_dtls_records_aux(Vinfo, Rest, [#ssl_tls{type = Type, + version = {MajVer, MinVer}, + epoch = Epoch, sequence_number = SequenceNumber, + fragment = Data} | Acc], SslOpts); + _ -> + ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) + end; +get_dtls_records_aux(_, <<?BYTE(_), ?BYTE(_MajVer), ?BYTE(_MinVer), ?UINT16(Length), _/binary>>, _Acc, _) when Length > ?MAX_CIPHER_TEXT_LENGTH -> ?ALERT_REC(?FATAL, ?RECORD_OVERFLOW); -get_dtls_records_aux(Data, Acc, _) -> +get_dtls_records_aux(_, Data, Acc, _) -> case size(Data) =< ?MAX_CIPHER_TEXT_LENGTH + ?INITIAL_BYTES of true -> {lists:reverse(Acc), Data}; diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index f99c0e2b7a..20b1e85ceb 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -127,7 +127,8 @@ error_alert/0, tls_alert/0, srp_param_type/0, - named_curve/0]). + named_curve/0, + sign_scheme/0]). %% ------------------------------------------------------------------------------------------------------- @@ -193,7 +194,7 @@ | rsa_pss_pss_sha384 | rsa_pss_pss_sha512 | rsa_pkcs1_sha1 - | ecdsa_sha1. + | ecdsa_sha1. % exported -type kex_algo() :: rsa | dhe_rsa | dhe_dss | @@ -1912,7 +1913,7 @@ validate_option(Opt, Value) -> throw({error, {options, {Opt, Value}}}). handle_cb_info({V1, V2, V3, V4}, {_,_,_,_,_}) -> - {V1,V2,V3,V4, list_to_atom(atom_to_list(V2) ++ "passive")}; + {V1,V2,V3,V4, list_to_atom(atom_to_list(V2) ++ "_passive")}; handle_cb_info(CbInfo, _) -> CbInfo. diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index 21db887bb5..4da50d2af8 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -923,6 +923,12 @@ signature_scheme(rsa_pss_pss_sha384) -> ?RSA_PSS_PSS_SHA384; signature_scheme(rsa_pss_pss_sha512) -> ?RSA_PSS_PSS_SHA512; signature_scheme(rsa_pkcs1_sha1) -> ?RSA_PKCS1_SHA1; signature_scheme(ecdsa_sha1) -> ?ECDSA_SHA1; +%% Handling legacy signature algorithms +signature_scheme({Hash0, Sign0}) -> + Hash = hash_algorithm(Hash0), + Sign = sign_algorithm(Sign0), + <<?UINT16(SigAlg)>> = <<?BYTE(Hash),?BYTE(Sign)>>, + SigAlg; signature_scheme(?RSA_PKCS1_SHA256) -> rsa_pkcs1_sha256; signature_scheme(?RSA_PKCS1_SHA384) -> rsa_pkcs1_sha384; signature_scheme(?RSA_PKCS1_SHA512) -> rsa_pkcs1_sha512; @@ -962,7 +968,9 @@ scheme_to_components(rsa_pss_pss_sha256) -> {sha256, rsa_pss_pss, undefined}; scheme_to_components(rsa_pss_pss_sha384) -> {sha384, rsa_pss_pss, undefined}; scheme_to_components(rsa_pss_pss_sha512) -> {sha512, rsa_pss_pss, undefined}; scheme_to_components(rsa_pkcs1_sha1) -> {sha1, rsa_pkcs1, undefined}; -scheme_to_components(ecdsa_sha1) -> {sha1, ecdsa, undefined}. +scheme_to_components(ecdsa_sha1) -> {sha1, ecdsa, undefined}; +%% Handling legacy signature algorithms +scheme_to_components({Hash,Sign}) -> {Hash, Sign, undefined}. %% TODO: Add support for EC and RSA-SSA signatures diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index 7047c01657..cc4d60389e 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -124,7 +124,7 @@ handshake(#sslsocket{pid = [Pid|_]} = Socket, Timeout) -> connected -> {ok, Socket}; {ok, Ext} -> - {ok, Socket, Ext}; + {ok, Socket, no_records(Ext)}; Error -> Error end. @@ -709,6 +709,7 @@ handle_session(#server_hello{cipher_suite = CipherSuite, {ExpectNPN, Protocol} = case Protocol0 of undefined -> + {false, CurrentProtocol}; _ -> {ProtoExt =:= npn, Protocol0} @@ -3000,3 +3001,8 @@ new_emulated([], EmOpts) -> EmOpts; new_emulated(NewEmOpts, _) -> NewEmOpts. + +no_records(Extensions) -> + maps:map(fun(_, Value) -> + ssl_handshake:extension_value(Value) + end, Extensions). diff --git a/lib/ssl/src/ssl_connection.hrl b/lib/ssl/src/ssl_connection.hrl index ff7207a8ce..844368c761 100644 --- a/lib/ssl/src/ssl_connection.hrl +++ b/lib/ssl/src/ssl_connection.hrl @@ -66,6 +66,7 @@ sni_hostname = undefined, expecting_next_protocol_negotiation = false ::boolean(), next_protocol = undefined :: undefined | binary(), + alpn = undefined, %% Used in TLS 1.3 negotiated_protocol, hashsign_algorithm = {undefined, undefined}, cert_hashsign_algorithm = {undefined, undefined}, @@ -76,7 +77,7 @@ srp_params :: #srp_user{} | secret_printout() | 'undefined', public_key_info :: ssl_handshake:public_key_info() | 'undefined', premaster_secret :: binary() | secret_printout() | 'undefined', - server_psk_identity :: binary() | 'undefined' % server psk identity hint + server_psk_identity :: binary() | 'undefined' % server psk identity hint }). -record(connection_env, { diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index b51ba0fa2d..c6698bc74a 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -76,7 +76,8 @@ handle_client_hello_extensions/9, %% Returns server hello extensions handle_server_hello_extensions/9, select_curve/2, select_curve/3, select_hashsign/4, select_hashsign/5, - select_hashsign_algs/3, empty_extensions/2, add_server_share/3 + select_hashsign_algs/3, empty_extensions/2, add_server_share/3, + add_alpn/2, add_selected_version/1, decode_alpn/1 ]). -export([get_cert_params/1, @@ -363,7 +364,7 @@ certify(#certificate{asn1_certificates = ASN1Certs}, CertDbHandle, CertDbRef, CertDbHandle, CertDbRef) end catch - error:{badmatch,{asn1, Asn1Reason}} -> + error:{badmatch,{error, {asn1, Asn1Reason}}} -> %% ASN-1 decode of certificate somehow failed ?ALERT_REC(?FATAL, ?CERTIFICATE_UNKNOWN, {failed_to_decode_certificate, Asn1Reason}); error:OtherReason -> @@ -1165,6 +1166,13 @@ add_server_share(hello_retry_request, Extensions, Extensions#{key_share => #key_share_hello_retry_request{ selected_group = Group}}. +add_alpn(Extensions, ALPN0) -> + ALPN = encode_alpn([ALPN0], false), + Extensions#{alpn => ALPN}. + +add_selected_version(Extensions) -> + SupportedVersions = #server_hello_selected_version{selected_version = {3,4}}, + Extensions#{server_hello_selected_version => SupportedVersions}. kse_remove_private_key(#key_share_entry{ group = Group, @@ -1186,10 +1194,7 @@ signature_algs_ext(undefined) -> signature_algs_ext(SignatureSchemes0) -> %% The SSL option signature_algs contains both hash-sign algorithms (tuples) and %% signature schemes (atoms) if TLS 1.3 is configured. - %% Filter out all hash-sign tuples when creating the signature_algs extension. - %% (TLS 1.3 specific record type) - SignatureSchemes = lists:filter(fun is_atom/1, SignatureSchemes0), - #signature_algorithms{signature_scheme_list = SignatureSchemes}. + #signature_algorithms{signature_scheme_list = SignatureSchemes0}. signature_algs_cert(undefined) -> undefined; @@ -1474,7 +1479,16 @@ extension_value(#next_protocol_negotiation{extension_data = Data}) -> extension_value(#srp{username = Name}) -> Name; extension_value(#renegotiation_info{renegotiated_connection = Data}) -> - Data. + Data; +extension_value(#signature_algorithms{signature_scheme_list = Schemes}) -> + Schemes; +extension_value(#signature_algorithms_cert{signature_scheme_list = Schemes}) -> + Schemes; +extension_value(#key_share_client_hello{client_shares = ClientShares}) -> + ClientShares; +extension_value(#client_hello_versions{versions = Versions}) -> + Versions. + %%-------------------------------------------------------------------- %%% Internal functions diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index 2651fc09bd..323d9e3284 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -259,7 +259,7 @@ next_event(StateName, Record, State) -> next_event(StateName, no_record, State0, Actions) -> case next_record(StateName, State0) of {no_record, State} -> - {next_state, StateName, State, Actions}; + ssl_connection:hibernate_after(StateName, State, Actions); {Record, State} -> next_event(StateName, Record, State, Actions) end; @@ -317,8 +317,7 @@ handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, fragment = Data}, _ -> HsEnv = State#state.handshake_env, {next_state, StateName, - State#state{protocol_buffers = Buffers, - handshake_env = + State#state{handshake_env = HsEnv#handshake_env{unprocessed_handshake_events = unprocessed_events(Events)}}, Events} end diff --git a/lib/ssl/src/tls_connection_1_3.erl b/lib/ssl/src/tls_connection_1_3.erl index 821b7000cc..117e4f059d 100644 --- a/lib/ssl/src/tls_connection_1_3.erl +++ b/lib/ssl/src/tls_connection_1_3.erl @@ -228,8 +228,8 @@ wait_cert_cr(internal, #change_cipher_spec{}, State, _Module) -> tls_connection:next_event(?FUNCTION_NAME, no_record, State); wait_cert_cr(internal, #certificate_1_3{} = Certificate, State0, _Module) -> case tls_handshake_1_3:do_wait_cert_cr(Certificate, State0) of - #alert{} = Alert -> - ssl_connection:handle_own_alert(Alert, {3,4}, wait_cert_cr, State0); + {#alert{} = Alert, State} -> + ssl_connection:handle_own_alert(Alert, {3,4}, wait_cert_cr, State); {State1, NextState} -> tls_connection:next_event(NextState, no_record, State1) end; diff --git a/lib/ssl/src/tls_handshake_1_3.erl b/lib/ssl/src/tls_handshake_1_3.erl index 12ab2015aa..4de51c9a35 100644 --- a/lib/ssl/src/tls_handshake_1_3.erl +++ b/lib/ssl/src/tls_handshake_1_3.erl @@ -39,8 +39,7 @@ %% Create handshake messages -export([certificate/5, certificate_verify/4, - encrypted_extensions/0, - server_hello/4]). + encrypted_extensions/0]). -export([do_start/2, do_negotiated/2, @@ -62,10 +61,10 @@ %% Create handshake messages %%==================================================================== -server_hello(MsgType, SessionId, KeyShare, ConnectionStates) -> +server_hello(MsgType, SessionId, KeyShare, ConnectionStates, ALPN) -> #{security_parameters := SecParams} = ssl_record:pending_connection_state(ConnectionStates, read), - Extensions = server_hello_extensions(MsgType, KeyShare), + Extensions = server_hello_extensions(MsgType, KeyShare, ALPN), #server_hello{server_version = {3,3}, %% legacy_version cipher_suite = SecParams#security_parameters.cipher_suite, compression_method = 0, %% legacy attribute @@ -74,10 +73,26 @@ server_hello(MsgType, SessionId, KeyShare, ConnectionStates) -> extensions = Extensions }. -server_hello_extensions(MsgType, KeyShare) -> +%% The server's extensions MUST contain "supported_versions". +%% Additionally, it SHOULD contain the minimal set of extensions +%% necessary for the client to generate a correct ClientHello pair. As +%% with the ServerHello, a HelloRetryRequest MUST NOT contain any +%% extensions that were not first offered by the client in its +%% ClientHello, with the exception of optionally the "cookie" (see +%% Section 4.2.2) extension. +server_hello_extensions(hello_retry_request = MsgType, KeyShare, _) -> SupportedVersions = #server_hello_selected_version{selected_version = {3,4}}, Extensions = #{server_hello_selected_version => SupportedVersions}, - ssl_handshake:add_server_share(MsgType, Extensions, KeyShare). + ssl_handshake:add_server_share(MsgType, Extensions, KeyShare); +server_hello_extensions(MsgType, KeyShare, undefined) -> + SupportedVersions = #server_hello_selected_version{selected_version = {3,4}}, + Extensions = #{server_hello_selected_version => SupportedVersions}, + ssl_handshake:add_server_share(MsgType, Extensions, KeyShare); +server_hello_extensions(MsgType, KeyShare, ALPN0) -> + Extensions0 = ssl_handshake:add_selected_version(#{}), %% {3,4} (TLS 1.3) + Extensions1 = ssl_handshake:add_alpn(Extensions0, ALPN0), + ssl_handshake:add_server_share(MsgType, Extensions1, KeyShare). + server_hello_random(server_hello, #security_parameters{server_random = Random}) -> Random; @@ -469,7 +484,8 @@ do_start(#client_hello{cipher_suites = ClientCiphers, #state{connection_states = _ConnectionStates0, ssl_options = #ssl_options{ciphers = ServerCiphers, signature_algs = ServerSignAlgs, - supported_groups = ServerGroups0}, + supported_groups = ServerGroups0, + alpn_preferred_protocols = ALPNPreferredProtocols}, session = #session{own_certificate = Cert}} = State0) -> ClientGroups0 = maps:get(elliptic_curves, Extensions, undefined), ClientGroups = get_supported_groups(ClientGroups0), @@ -478,6 +494,9 @@ do_start(#client_hello{cipher_suites = ClientCiphers, ClientShares0 = maps:get(key_share, Extensions, undefined), ClientShares = get_key_shares(ClientShares0), + ClientALPN0 = maps:get(alpn, Extensions, undefined), + ClientALPN = ssl_handshake:decode_alpn(ClientALPN0), + ClientSignAlgs = get_signature_scheme_list( maps:get(signature_algs, Extensions, undefined)), ClientSignAlgsCert = get_signature_scheme_list( @@ -486,6 +505,9 @@ do_start(#client_hello{cipher_suites = ClientCiphers, {Ref,Maybe} = maybe(), try + %% Handle ALPN extension if ALPN is configured + ALPNProtocol = Maybe(handle_alpn(ALPNPreferredProtocols, ClientALPN)), + %% If the server does not select a PSK, then the server independently selects a %% cipher suite, an (EC)DHE group and key share for key establishment, %% and a signature algorithm/certificate pair to authenticate itself to @@ -511,8 +533,14 @@ do_start(#client_hello{cipher_suites = ClientCiphers, %% Generate server_share KeyShare = ssl_cipher:generate_server_share(Group), - State1 = update_start_state(State0, Cipher, KeyShare, SessionId, - Group, SelectedSignAlg, ClientPubKey), + State1 = update_start_state(State0, + #{cipher => Cipher, + key_share => KeyShare, + session_id => SessionId, + group => Group, + sign_alg => SelectedSignAlg, + peer_public_key => ClientPubKey, + alpn => ALPNProtocol}), %% 4.1.4. Hello Retry Request %% @@ -522,10 +550,7 @@ do_start(#client_hello{cipher_suites = ClientCiphers, %% the handshake. Maybe(send_hello_retry_request(State1, ClientPubKey, KeyShare, SessionId)) - %% TODO: - %% - session handling - %% - handle extensions: ALPN - %% (do not handle: NPN, srp, renegotiation_info, ec_point_formats) + %% TODO: session handling catch {Ref, {insufficient_security, no_suitable_groups}} -> @@ -537,7 +562,9 @@ do_start(#client_hello{cipher_suites = ClientCiphers, {Ref, {insufficient_security, no_suitable_signature_algorithm}} -> ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, "No suitable signature algorithm"); {Ref, {insufficient_security, no_suitable_public_key}} -> - ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, no_suitable_public_key) + ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, no_suitable_public_key); + {Ref, no_application_protocol} -> + ?ALERT_REC(?FATAL, ?NO_APPLICATION_PROTOCOL) end; %% TLS Client do_start(#server_hello{cipher_suite = SelectedCipherSuite, @@ -588,8 +615,11 @@ do_start(#server_hello{cipher_suite = SelectedCipherSuite, HelloVersion = tls_record:hello_version(SslOpts#ssl_options.versions), %% Update state - State1 = update_start_state(State0, SelectedCipherSuite, ClientKeyShare, SessionId, - SelectedGroup, undefined, undefined), + State1 = update_start_state(State0, + #{cipher => SelectedCipherSuite, + key_share => ClientKeyShare, + session_id => SessionId, + group => SelectedGroup}), %% Replace ClientHello1 with a special synthetic handshake message State2 = replace_ch1_with_message_hash(State1), @@ -625,7 +655,8 @@ do_negotiated(start_handshake, dh_public_value = ClientPublicKey}, ssl_options = #ssl_options{} = SslOpts, key_share = KeyShare, - handshake_env = #handshake_env{tls_handshake_history = _HHistory0}, + handshake_env = #handshake_env{tls_handshake_history = _HHistory0, + alpn = ALPN}, connection_env = #connection_env{private_key = CertPrivateKey}, static_env = #static_env{ cert_db = CertDbHandle, @@ -640,7 +671,7 @@ do_negotiated(start_handshake, try %% Create server_hello %% Extensions: supported_versions, key_share, (pre_shared_key) - ServerHello = server_hello(server_hello, SessionId, KeyShare, ConnectionStates0), + ServerHello = server_hello(server_hello, SessionId, KeyShare, ConnectionStates0, ALPN), {State1, _} = tls_connection:send_handshake(ServerHello, State0), @@ -702,6 +733,8 @@ do_wait_cert(#certificate_1_3{} = Certificate, State0) -> {?ALERT_REC(?FATAL, ?INTERNAL_ERROR, Reason), State}; {Ref, {{handshake_failure, Reason}, State}} -> {?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, Reason), State}; + {Ref, {#alert{} = Alert, State}} -> + {Alert, State}; {#alert{} = Alert, State} -> {Alert, State} end. @@ -801,8 +834,12 @@ do_wait_sh(#server_hello{cipher_suite = SelectedCipherSuite, {_, ClientPrivateKey} = get_client_private_key([SelectedGroup], ClientKeyShare), %% Update state - State1 = update_start_state(State0, SelectedCipherSuite, ClientKeyShare0, SessionId, - SelectedGroup, undefined, ServerPublicKey), + State1 = update_start_state(State0, + #{cipher => SelectedCipherSuite, + key_share => ClientKeyShare0, + session_id => SessionId, + group => SelectedGroup, + peer_public_key => ServerPublicKey}), State2 = calculate_handshake_secrets(ServerPublicKey, ClientPrivateKey, SelectedGroup, State1), @@ -858,7 +895,9 @@ do_wait_cert_cr(#certificate_1_3{} = Certificate, State0) -> {Ref, {{internal_error, Reason}, _State}} -> ?ALERT_REC(?FATAL, ?INTERNAL_ERROR, Reason); {Ref, {{handshake_failure, Reason}, _State}} -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, Reason) + ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, Reason); + {Ref, {#alert{} = Alert, State}} -> + {Alert, State} end; do_wait_cert_cr(#certificate_request_1_3{} = CertificateRequest, State0) -> {Ref,Maybe} = maybe(), @@ -984,9 +1023,10 @@ compare_verify_data(_, _) -> {error, decrypt_error}. -send_hello_retry_request(#state{connection_states = ConnectionStates0} = State0, +send_hello_retry_request(#state{connection_states = ConnectionStates0, + handshake_env = #handshake_env{alpn = ALPN}} = State0, no_suitable_key, KeyShare, SessionId) -> - ServerHello = server_hello(hello_retry_request, SessionId, KeyShare, ConnectionStates0), + ServerHello = server_hello(hello_retry_request, SessionId, KeyShare, ConnectionStates0, ALPN), {State1, _} = tls_connection:send_handshake(ServerHello, State0), %% Update handshake history @@ -1076,13 +1116,11 @@ process_certificate(#certificate_1_3{certificate_list = Certs0}, State = store_peer_cert(State0, PeerCert, PublicKeyInfo), {ok, {State, wait_cv}}; {error, Reason} -> - State1 = calculate_traffic_secrets(State0), - State = ssl_record:step_encryption_state(State1), + State = update_encryption_state(Role, State0), {error, {Reason, State}}; - #alert{} = Alert -> - State1 = calculate_traffic_secrets(State0), - State = ssl_record:step_encryption_state(State1), - {Alert, State} + {ok, #alert{} = Alert} -> + State = update_encryption_state(Role, State0), + {error, {Alert, State}} end; false -> State1 = calculate_traffic_secrets(State0), @@ -1106,6 +1144,17 @@ is_supported_signature_algorithm([BinCert|_], SignAlgs0) -> lists:member(Scheme, SignAlgs). +%% Sets correct encryption state when sending Alerts in shared states that use different secrets. +%% - If client: use handshake secrets. +%% - If server: use traffic secrets as by this time the client's state machine +%% already stepped into the 'connection' state. +update_encryption_state(server, State0) -> + State1 = calculate_traffic_secrets(State0), + ssl_record:step_encryption_state(State1); +update_encryption_state(client, State) -> + State. + + validate_certificate_chain(Certs, CertDbHandle, CertDbRef, SslOptions, CRLDbHandle, Role, Host) -> ServerName = ssl_handshake:server_name(SslOptions#ssl_options.server_name_indication, Host, Role), [PeerCert | ChainCerts ] = Certs, @@ -1126,9 +1175,9 @@ validate_certificate_chain(Certs, CertDbHandle, CertDbRef, SslOptions, CRLDbHand {ok, {PublicKeyInfo,_}} -> {ok, {PeerCert, PublicKeyInfo}}; {error, Reason} -> - ssl_handshake:handle_path_validation_error(Reason, PeerCert, ChainCerts, - SslOptions, Options, - CertDbHandle, CertDbRef) + {ok, ssl_handshake:handle_path_validation_error(Reason, PeerCert, ChainCerts, + SslOptions, Options, + CertDbHandle, CertDbRef)} end catch error:{badmatch,{asn1, Asn1Reason}} -> @@ -1337,11 +1386,24 @@ update_connection_state(ConnectionState = #{security_parameters := SecurityParam cipher_state => cipher_init(Key, IV, FinishedKey)}. +update_start_state(State, Map) -> + Cipher = maps:get(cipher, Map, undefined), + KeyShare = maps:get(key_share, Map, undefined), + SessionId = maps:get(session_id, Map, undefined), + Group = maps:get(group, Map, undefined), + SelectedSignAlg = maps:get(sign_alg, Map, undefined), + PeerPublicKey = maps:get(peer_public_key, Map, undefined), + ALPNProtocol = maps:get(alpn, Map, undefined), + update_start_state(State, Cipher, KeyShare, SessionId, + Group, SelectedSignAlg, PeerPublicKey, + ALPNProtocol). +%% update_start_state(#state{connection_states = ConnectionStates0, + handshake_env = #handshake_env{} = HsEnv, connection_env = CEnv, session = Session} = State, Cipher, KeyShare, SessionId, - Group, SelectedSignAlg, ClientPubKey) -> + Group, SelectedSignAlg, PeerPublicKey, ALPNProtocol) -> #{security_parameters := SecParamsR0} = PendingRead = maps:get(pending_read, ConnectionStates0), #{security_parameters := SecParamsW0} = PendingWrite = @@ -1352,11 +1414,12 @@ update_start_state(#state{connection_states = ConnectionStates0, ConnectionStates0#{pending_read => PendingRead#{security_parameters => SecParamsR}, pending_write => PendingWrite#{security_parameters => SecParamsW}}, State#state{connection_states = ConnectionStates, + handshake_env = HsEnv#handshake_env{alpn = ALPNProtocol}, key_share = KeyShare, session = Session#session{session_id = SessionId, ecc = Group, sign_alg = SelectedSignAlg, - dh_public_value = ClientPubKey, + dh_public_value = PeerPublicKey, cipher_suite = Cipher}, connection_env = CEnv#connection_env{negotiated_version = {3,4}}}. @@ -1628,19 +1691,28 @@ get_server_public_key({key_share_entry, Group, PublicKey}) -> {Group, PublicKey}. -%% get_client_public_key(Group, ClientShares) -> -%% case lists:keysearch(Group, 2, ClientShares) of -%% {value, {_, _, ClientPublicKey}} -> -%% ClientPublicKey; -%% false -> -%% %% 4.1.4. Hello Retry Request -%% %% -%% %% The server will send this message in response to a ClientHello -%% %% message if it is able to find an acceptable set of parameters but the -%% %% ClientHello does not contain sufficient information to proceed with -%% %% the handshake. -%% no_suitable_key -%% end. +%% RFC 7301 - Application-Layer Protocol Negotiation Extension +%% It is expected that a server will have a list of protocols that it +%% supports, in preference order, and will only select a protocol if the +%% client supports it. In that case, the server SHOULD select the most +%% highly preferred protocol that it supports and that is also +%% advertised by the client. In the event that the server supports no +%% protocols that the client advertises, then the server SHALL respond +%% with a fatal "no_application_protocol" alert. +handle_alpn(undefined, _) -> + {ok, undefined}; +handle_alpn([], _) -> + {error, no_application_protocol}; +handle_alpn([_|_], undefined) -> + {ok, undefined}; +handle_alpn([ServerProtocol|T], ClientProtocols) -> + case lists:member(ServerProtocol, ClientProtocols) of + true -> + {ok, ServerProtocol}; + false -> + handle_alpn(T, ClientProtocols) + end. + select_cipher_suite([], _) -> {error, no_suitable_cipher}; diff --git a/lib/ssl/src/tls_v1.erl b/lib/ssl/src/tls_v1.erl index 27cd5765e5..f7c8c770ae 100644 --- a/lib/ssl/src/tls_v1.erl +++ b/lib/ssl/src/tls_v1.erl @@ -606,8 +606,26 @@ signature_schemes(Version, SignatureSchemes) when is_tuple(Version) Acc end; %% Special clause for filtering out the legacy hash-sign tuples. - (_ , Acc) -> - Acc + ({Hash, dsa = Sign} = Alg, Acc) -> + case proplists:get_bool(dss, PubKeys) + andalso proplists:get_bool(Hash, Hashes) + andalso is_pair(Hash, Sign, Hashes) + of + true -> + [Alg | Acc]; + false -> + Acc + end; + ({Hash, Sign} = Alg, Acc) -> + case proplists:get_bool(Sign, PubKeys) + andalso proplists:get_bool(Hash, Hashes) + andalso is_pair(Hash, Sign, Hashes) + of + true -> + [Alg | Acc]; + false -> + Acc + end end, Supported = lists:foldl(Fun, [], SignatureSchemes), lists:reverse(Supported); diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 8cb98e7fa6..0cdfea0ac2 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -164,6 +164,7 @@ api_tests() -> prf, socket_options, active_n, + internal_active_1, cipher_suites, handshake_continue, handshake_continue_timeout, @@ -243,11 +244,13 @@ rizzo_tests() -> %% For testing TLS 1.3 features and possible regressions tls13_test_group() -> - [tls13_enable_client_side, + [handshake_continue_tls13_client, + tls13_enable_client_side, tls13_enable_server_side, tls_record_1_3_encode_decode, tls13_finished_verify_data, tls13_1_RTT_handshake, + tls12_ssl_server_tls13_ssl_client, tls13_basic_ssl_server_openssl_client, tls13_basic_ssl_server_ssl_client, tls13_basic_openssl_server_ssl_client, @@ -271,7 +274,11 @@ tls13_test_group() -> tls13_unsupported_sign_algo_client_auth_ssl_server_ssl_client, tls13_unsupported_sign_algo_cert_client_auth_ssl_server_openssl_client, tls13_unsupported_sign_algo_cert_client_auth_ssl_server_ssl_client, - tls13_connection_information]. + tls13_connection_information, + tls13_ssl_server_with_alpn_ssl_client, + tls13_ssl_server_with_alpn_ssl_client_empty_alpn, + tls13_ssl_server_with_alpn_ssl_client_bad_alpn, + tls13_ssl_server_with_alpn_ssl_client_alpn]. %%-------------------------------------------------------------------- init_per_suite(Config0) -> @@ -500,6 +507,15 @@ init_per_testcase(accept_pool, Config) -> ssl_test_lib:ct_log_supported_protocol_versions(Config), Config end; + +init_per_testcase(internal_active_1, Config) -> + ssl:stop(), + application:load(ssl), + application:set_env(ssl, internal_active_n, 1), + ssl:start(), + ct:timetrap({seconds, 5}), + Config; + init_per_testcase(controller_dies, Config) -> ct:timetrap({seconds, 10}), Config; @@ -522,6 +538,10 @@ end_per_testcase(reuse_session_expired, Config) -> application:unset_env(ssl, session_delay_cleanup_time), end_per_testcase(default_action, Config); +end_per_testcase(internal_active_n, Config) -> + application:unset_env(ssl, internal_active_n), + end_per_testcase(default_action, Config); + end_per_testcase(Case, Config) when Case == protocol_versions; Case == empty_protocol_versions-> application:unset_env(ssl, protocol_versions), @@ -669,6 +689,43 @@ handshake_continue(Config) when is_list(Config) -> ssl_test_lib:close(Server), ssl_test_lib:close(Client). + +%%-------------------------------------------------------------------- +handshake_continue_tls13_client() -> + [{doc, "Test API function ssl:handshake_continue/3"}]. +handshake_continue_tls13_client(Config) when is_list(Config) -> + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), + ClientOpts = [{versions, ['tlsv1.2','tlsv1.3']}|ClientOpts0], + + ClientOptsHello0 = ssl_test_lib:ssl_options([{handshake, hello}], Config), + ClientOptsHello = [{versions, ['tlsv1.2','tlsv1.3']}|ClientOptsHello0], + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ssl_test_lib:ssl_options([{reuseaddr, true}, {handshake, hello}], + Config)}, + {continue_options, proplists:delete(reuseaddr, ServerOpts)} + ]), + + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOptsHello}, + {continue_options, proplists:delete(reuseaddr, ClientOpts)}]), + + ssl_test_lib:check_result(Server, ok, Client, ok), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). + + %%------------------------------------------------------------------ handshake_continue_timeout() -> [{doc, "Test API function ssl:handshake_continue/3 with short timeout"}]. @@ -1987,6 +2044,10 @@ recv_active_once(Config) when is_list(Config) -> ssl_test_lib:close(Server), ssl_test_lib:close(Client). + + + + %%-------------------------------------------------------------------- recv_active_n() -> [{doc,"Test recv on active (n) socket"}]. @@ -2013,6 +2074,7 @@ recv_active_n(Config) when is_list(Config) -> ssl_test_lib:close(Server), ssl_test_lib:close(Client). + %%-------------------------------------------------------------------- %% Test case adapted from gen_tcp_misc_SUITE. active_n() -> @@ -2238,6 +2300,33 @@ upgrade_result(Socket) -> ok end. + +%%-------------------------------------------------------------------- +internal_active_1() -> + [{doc,"Test internal active 1 (behave as internal active once)"}]. + +internal_active_1(Config) when is_list(Config) -> + ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, [{active, true} | ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + Client = + ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, [{active, true} | ClientOpts]}]), + + ssl_test_lib:check_result(Server, ok, Client, ok), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). + %%-------------------------------------------------------------------- tls_upgrade_with_timeout() -> [{doc,"Test ssl_accept/3"}]. @@ -3587,7 +3676,7 @@ hibernate(Config) -> ssl_test_lib:check_result(Server, ok, Client, ok), - timer:sleep(1500), + ct:sleep(1500), {current_function, {erlang, hibernate, 3}} = process_info(Pid, current_function), @@ -3623,6 +3712,8 @@ hibernate_right_away(Config) -> [{port, Port1}, {options, [{hibernate_after, 0}|ClientOpts]}]), ssl_test_lib:check_result(Server1, ok, Client1, ok), + + ct:sleep(1000), %% Schedule out {current_function, {erlang, hibernate, 3}} = process_info(Pid1, current_function), @@ -5317,6 +5408,41 @@ tls13_finished_verify_data(_Config) -> FinishedKey = tls_v1:finished_key(BaseKey, sha256), VerifyData = tls_v1:finished_verify_data(FinishedKey, sha256, Messages). + +tls12_ssl_server_tls13_ssl_client() -> + [{doc,"Test basic connection between TLS 1.2 server and TLS 1.3 client"}]. + +tls12_ssl_server_tls13_ssl_client(Config) -> + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + %% Set versions + ServerOpts = [{versions, ['tlsv1.2']}|ServerOpts0], + ClientOpts = [{versions, ['tlsv1.2','tlsv1.3']}, + {signature_algs_cert, [ecdsa_secp384r1_sha384, + rsa_pss_rsae_sha256, + rsa_pkcs1_sha256, + {sha256,rsa},{sha256,dsa}]}|ClientOpts0], + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts}]), + + ssl_test_lib:check_result(Server, ok, Client, ok), + + ssl_test_lib:close(Server), + ssl_test_lib:close_port(Client). + + tls13_basic_ssl_server_openssl_client() -> [{doc,"Test TLS 1.3 basic connection between ssl server and openssl s_client"}]. @@ -5551,11 +5677,7 @@ tls13_client_auth_empty_cert_alert_ssl_server_openssl_client(Config) -> Client = ssl_test_lib:start_basic_client(openssl, 'tlsv1.3', Port, ClientOpts), - ssl_test_lib:check_result(Server, - {error, - {tls_alert, - {certificate_required, - "received SERVER ALERT: Fatal - Certificate required - certificate_required"}}}), + ssl_test_lib:check_server_alert(Server, certificate_required), ssl_test_lib:close(Server), ssl_test_lib:close_port(Client). @@ -5589,11 +5711,7 @@ tls13_client_auth_empty_cert_alert_ssl_server_ssl_client(Config) -> {mfa, {ssl_test_lib, send_recv_result_active, []}}, {options, ClientOpts}]), - ssl_test_lib:check_result(Server, - {error, - {tls_alert, - {certificate_required, - "received SERVER ALERT: Fatal - Certificate required - certificate_required"}}}), + ssl_test_lib:check_server_alert(Server, certificate_required), ssl_test_lib:close(Server), ssl_test_lib:close_port(Client). @@ -5745,11 +5863,7 @@ tls13_hrr_client_auth_empty_cert_alert_ssl_server_openssl_client(Config) -> Client = ssl_test_lib:start_basic_client(openssl, 'tlsv1.3', Port, ClientOpts), - ssl_test_lib:check_result(Server, - {error, - {tls_alert, - {certificate_required, - "received SERVER ALERT: Fatal - Certificate required - certificate_required"}}}), + ssl_test_lib:check_server_alert(Server, certificate_required), ssl_test_lib:close(Server), ssl_test_lib:close_port(Client). @@ -5785,11 +5899,7 @@ tls13_hrr_client_auth_empty_cert_alert_ssl_server_ssl_client(Config) -> {mfa, {ssl_test_lib, send_recv_result_active, []}}, {options, ClientOpts}]), - ssl_test_lib:check_result(Server, - {error, - {tls_alert, - {certificate_required, - "received SERVER ALERT: Fatal - Certificate required - certificate_required"}}}), + ssl_test_lib:check_server_alert(Server, certificate_required), ssl_test_lib:close(Server), ssl_test_lib:close_port(Client). @@ -5945,13 +6055,7 @@ tls13_unsupported_sign_algo_client_auth_ssl_server_openssl_client(Config) -> Client = ssl_test_lib:start_basic_client(openssl, 'tlsv1.3', Port, ClientOpts), - ssl_test_lib:check_result( - Server, - {error, - {tls_alert, - {insufficient_security, - "received SERVER ALERT: Fatal - Insufficient Security - " - "\"No suitable signature algorithm\""}}}), + ssl_test_lib:check_server_alert(Server, insufficient_security), ssl_test_lib:close(Server), ssl_test_lib:close_port(Client). @@ -5984,13 +6088,7 @@ tls13_unsupported_sign_algo_client_auth_ssl_server_ssl_client(Config) -> {mfa, {ssl_test_lib, send_recv_result_active, []}}, {options, ClientOpts}]), - ssl_test_lib:check_result( - Server, - {error, - {tls_alert, - {insufficient_security, - "received SERVER ALERT: Fatal - Insufficient Security - " - "\"No suitable signature algorithm\""}}}), + ssl_test_lib:check_server_alert(Server, insufficient_security), ssl_test_lib:close(Server), ssl_test_lib:close_port(Client). @@ -6024,12 +6122,7 @@ tls13_unsupported_sign_algo_cert_client_auth_ssl_server_openssl_client(Config) - Client = ssl_test_lib:start_basic_client(openssl, 'tlsv1.3', Port, ClientOpts), - ssl_test_lib:check_result( - Server, - {error, - {tls_alert, - {certificate_required, - "received SERVER ALERT: Fatal - Certificate required - certificate_required"}}}), + ssl_test_lib:check_server_alert(Server, certificate_required), ssl_test_lib:close(Server), ssl_test_lib:close_port(Client). @@ -6068,12 +6161,7 @@ tls13_unsupported_sign_algo_cert_client_auth_ssl_server_ssl_client(Config) -> {mfa, {ssl_test_lib, send_recv_result_active, []}}, {options, ClientOpts}]), - ssl_test_lib:check_result( - Server, - {error, - {tls_alert, - {certificate_required, - "received SERVER ALERT: Fatal - Certificate required - certificate_required"}}}), + ssl_test_lib:check_server_alert(Server, certificate_required), ssl_test_lib:close(Server), ssl_test_lib:close_port(Client). @@ -6101,6 +6189,132 @@ tls13_connection_information(Config) -> ssl_test_lib:close_port(Client). +tls13_ssl_server_with_alpn_ssl_client() -> + [{doc,"Test TLS 1.3 between ssl server with ALPN configured and ssl client"}]. + +tls13_ssl_server_with_alpn_ssl_client(Config) -> + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + %% Set versions + ServerOpts = [{versions, ['tlsv1.2','tlsv1.3']}, + {alpn_preferred_protocols, [<<5,6>>, <<1>>]}|ServerOpts0], + ClientOpts = [{versions, ['tlsv1.2','tlsv1.3']}|ClientOpts0], + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts}]), + + ssl_test_lib:check_result(Server, ok, Client, ok), + + ssl_test_lib:close(Server), + ssl_test_lib:close_port(Client). + + +tls13_ssl_server_with_alpn_ssl_client_empty_alpn() -> + [{doc,"Test TLS 1.3 between ssl server with ALPN configured and ssl client with empty ALPN"}]. + +tls13_ssl_server_with_alpn_ssl_client_empty_alpn(Config) -> + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + %% Set versions + ServerOpts = [{versions, ['tlsv1.2','tlsv1.3']}, + {alpn_preferred_protocols, [<<5,6>>, <<1>>]}|ServerOpts0], + ClientOpts = [{versions, ['tlsv1.2','tlsv1.3']}, + {alpn_advertised_protocols, []}|ClientOpts0], + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts}]), + + ssl_test_lib:check_server_alert(Server, no_application_protocol), + + ssl_test_lib:close(Server), + ssl_test_lib:close_port(Client). + + +tls13_ssl_server_with_alpn_ssl_client_bad_alpn() -> + [{doc,"Test TLS 1.3 between ssl server with ALPN configured and ssl client with bad ALPN"}]. + +tls13_ssl_server_with_alpn_ssl_client_bad_alpn(Config) -> + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + %% Set versions + ServerOpts = [{versions, ['tlsv1.2','tlsv1.3']}, + {alpn_preferred_protocols, [<<5,6>>, <<1>>]}|ServerOpts0], + ClientOpts = [{versions, ['tlsv1.2','tlsv1.3']}, + {alpn_advertised_protocols, [<<1,2,3,4>>]}|ClientOpts0], + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts}]), + + ssl_test_lib:check_server_alert(Server, no_application_protocol), + + ssl_test_lib:close(Server), + ssl_test_lib:close_port(Client). + +tls13_ssl_server_with_alpn_ssl_client_alpn() -> + [{doc,"Test TLS 1.3 between ssl server with ALPN configured and ssl client with correct ALPN"}]. + +tls13_ssl_server_with_alpn_ssl_client_alpn(Config) -> + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + %% Set versions + ServerOpts = [{versions, ['tlsv1.2','tlsv1.3']}, + {alpn_preferred_protocols, [<<5,6>>, <<1>>]}|ServerOpts0], + ClientOpts = [{versions, ['tlsv1.2','tlsv1.3']}, + {alpn_advertised_protocols, [<<1,2,3,4>>, <<5,6>>]}|ClientOpts0], + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts}]), + + ssl_test_lib:check_result(Server, ok, Client, ok), + + ssl_test_lib:close(Server), + ssl_test_lib:close_port(Client). + + %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- diff --git a/lib/ssl/test/ssl_certificate_verify_SUITE.erl b/lib/ssl/test/ssl_certificate_verify_SUITE.erl index 55dee9a48f..4de4a35e59 100644 --- a/lib/ssl/test/ssl_certificate_verify_SUITE.erl +++ b/lib/ssl/test/ssl_certificate_verify_SUITE.erl @@ -40,6 +40,7 @@ %%-------------------------------------------------------------------- all() -> [ + {group, 'tlsv1.3'}, {group, 'tlsv1.2'}, {group, 'tlsv1.1'}, {group, 'tlsv1'}, @@ -50,6 +51,7 @@ all() -> groups() -> [ + {'tlsv1.3', [], all_protocol_groups()}, {'tlsv1.2', [], all_protocol_groups()}, {'tlsv1.1', [], all_protocol_groups()}, {'tlsv1', [], all_protocol_groups()}, @@ -89,7 +91,8 @@ tests() -> critical_extension_verify_server, critical_extension_verify_none, customize_hostname_check, - incomplete_chain + incomplete_chain, + long_chain ]. error_handling_tests()-> @@ -300,7 +303,13 @@ server_require_peer_cert_fail(Config) when is_list(Config) -> {from, self()}, {options, [{active, Active} | BadClientOpts]}]), - ssl_test_lib:check_server_alert(Server, Client, handshake_failure). + Version = proplists:get_value(version,Config), + case Version of + 'tlsv1.3' -> + ssl_test_lib:check_server_alert(Server, Client, certificate_required); + _ -> + ssl_test_lib:check_server_alert(Server, Client, handshake_failure) + end. %%-------------------------------------------------------------------- server_require_peer_cert_empty_ok() -> @@ -853,6 +862,7 @@ invalid_signature_server(Config) when is_list(Config) -> {from, self()}, {options, [{verify, verify_peer} | ClientOpts]}]), ssl_test_lib:check_server_alert(Server, Client, unknown_ca). + %%-------------------------------------------------------------------- invalid_signature_client() -> @@ -1157,6 +1167,44 @@ incomplete_chain(Config) when is_list(Config) -> ssl_test_lib:close(Server), ssl_test_lib:close(Client). +long_chain() -> + [{doc,"Test option verify_peer"}]. +long_chain(Config) when is_list(Config) -> + #{server_config := ServerConf, + client_config := ClientConf} = public_key:pkix_test_data(#{server_chain => #{root => [{key, ssl_test_lib:hardcode_rsa_key(1)}], + intermediates => [[{key, ssl_test_lib:hardcode_rsa_key(2)}], + [{key, ssl_test_lib:hardcode_rsa_key(3)}], + [{key, ssl_test_lib:hardcode_rsa_key(4)}]], + peer => [{key, ssl_test_lib:hardcode_rsa_key(5)}]}, + client_chain => #{root => [{key, ssl_test_lib:hardcode_rsa_key(3)}], + intermediates => [[{key, ssl_test_lib:hardcode_rsa_key(2)}]], + peer => [{key, ssl_test_lib:hardcode_rsa_key(1)}]}}), + [ServerRoot| _] = ServerCas = proplists:get_value(cacerts, ServerConf), + ClientCas = proplists:get_value(cacerts, ClientConf), + + Active = proplists:get_value(active, Config), + ReceiveFunction = proplists:get_value(receive_function, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, ReceiveFunction, []}}, + {options, [{active, Active}, {verify, verify_peer}, + {cacerts, [ServerRoot]} | + proplists:delete(cacerts, ServerConf)]}]), + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, ReceiveFunction, []}}, + {options, [{active, Active}, + {verify, verify_peer}, + {depth, 5}, + {cacerts, ServerCas ++ ClientCas} | + proplists:delete(cacerts, ClientConf)]}]), + ssl_test_lib:check_result(Server, ok, Client, ok), + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). + %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 733ee993a8..32fd917937 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -159,6 +159,7 @@ connect(ListenSocket, Node, N, _, Timeout, SslOpts, [_|_] =ContOpts) -> case ssl:handshake(AcceptSocket, SslOpts, Timeout) of {ok, Socket0, Ext} -> + [_|_] = maps:get(sni, Ext), ct:log("Ext ~p:~n", [Ext]), ct:log("~p:~p~nssl:handshake_continue(~p,~p,~p)~n", [?MODULE,?LINE, Socket0, ContOpts,Timeout]), case ssl:handshake_continue(Socket0, ContOpts, Timeout) of @@ -427,41 +428,42 @@ check_result(Pid, Msg) -> {got, Unexpected}}, ct:fail(Reason) end. + check_server_alert(Pid, Alert) -> receive {Pid, {error, {tls_alert, {Alert, STxt}}}} -> check_server_txt(STxt), + ok; + {Pid, {error, closed}} -> ok end. check_server_alert(Server, Client, Alert) -> receive {Server, {error, {tls_alert, {Alert, STxt}}}} -> check_server_txt(STxt), - receive - {Client, {error, {tls_alert, {Alert, CTxt}}}} -> - check_client_txt(CTxt), - ok; - {Client, {error, closed}} -> - ok - end + check_client_alert(Client, Alert) end. check_client_alert(Pid, Alert) -> receive {Pid, {error, {tls_alert, {Alert, CTxt}}}} -> check_client_txt(CTxt), + ok; + {Pid, {ssl_error, _, {tls_alert, {Alert, CTxt}}}} -> + check_client_txt(CTxt), + ok; + {Pid, {error, closed}} -> ok end. check_client_alert(Server, Client, Alert) -> receive {Client, {error, {tls_alert, {Alert, CTxt}}}} -> check_client_txt(CTxt), - receive - {Server, {error, {tls_alert, {Alert, STxt}}}} -> - check_server_txt(STxt), - ok; - {Server, {error, closed}} -> - ok - end + check_server_alert(Server, Alert); + {Client, {ssl_error, _, {tls_alert, {Alert, CTxt}}}} -> + check_client_txt(CTxt), + ok; + {Client, {error, closed}} -> + ok end. check_server_txt("TLS server" ++ _) -> ok; @@ -1102,7 +1104,15 @@ run_client_error(Opts) -> Options = proplists:get_value(options, Opts), ct:log("~p:~p~nssl:connect(~p, ~p, ~p)~n", [?MODULE,?LINE, Host, Port, Options]), Error = Transport:connect(Host, Port, Options), - Pid ! {self(), Error}. + case Error of + {error, _} -> + Pid ! {self(), Error}; + {ok, _Socket} -> + receive + {ssl_error, _, {tls_alert, _}} = SslError -> + Pid ! {self(), SslError} + end + end. accepters(N) -> accepters([], N). @@ -1641,6 +1651,8 @@ is_tls_version('dtlsv1.2') -> true; is_tls_version('dtlsv1') -> true; +is_tls_version('tlsv1.3') -> + true; is_tls_version('tlsv1.2') -> true; is_tls_version('tlsv1.1') -> @@ -2167,7 +2179,8 @@ clean_env() -> application:unset_env(ssl, session_cache_server_max), application:unset_env(ssl, ssl_pem_cache_clean), application:unset_env(ssl, bypass_pem_cache), - application:unset_env(ssl, alert_timeout). + application:unset_env(ssl, alert_timeout), + application:unset_env(ssl, internal_active_n). clean_start() -> ssl:stop(), diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index cbc32cd5a8..01dee392f5 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 9.3.1 +SSL_VSN = 9.3.3 diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index 385f092069..66624c43be 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -31,6 +31,23 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 3.9.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug that could cause a loop when formatting + terms using the control sequences <c>p</c> or <c>P</c> + and limiting the output with the option + <c>chars_limit</c>. </p> + <p> + Own Id: OTP-15875 Aux Id: ERL-967 </p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.9.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -289,6 +306,40 @@ </section> +<section><title>STDLIB 3.8.2.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug that could cause a loop when formatting + terms using the control sequences <c>p</c> or <c>P</c> + and limiting the output with the option + <c>chars_limit</c>. </p> + <p> + Own Id: OTP-15875 Aux Id: ERL-967 </p> + </item> + </list> + </section> + +</section> + +<section><title>STDLIB 3.8.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug that could cause a failure when formatting + binaries using the control sequences <c>p</c> or <c>P</c> + and limiting the output with the option + <c>chars_limit</c>. </p> + <p> + Own Id: OTP-15847 Aux Id: ERL-957 </p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.8.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/src/io_lib_pretty.erl b/lib/stdlib/src/io_lib_pretty.erl index 0cb3b01aae..77f02eafe0 100644 --- a/lib/stdlib/src/io_lib_pretty.erl +++ b/lib/stdlib/src/io_lib_pretty.erl @@ -462,7 +462,9 @@ find_upper(Lower, Term, T, Dl, Dd, D, RF, Enc, Str) -> case If of {_, _, _Dots=0, _} -> % even if Len > T If; - {_, Len, _, _} when Len =< T, D1 < D orelse D < 0 -> + {_, _Len=T, _, _} -> % increasing the depth is meaningless + If; + {_, Len, _, _} when Len < T, D1 < D orelse D < 0 -> find_upper(If, Term, T, D1, Dd2, D, RF, Enc, Str); _ -> search_depth(Lower, If, Term, T, Dl, D1, RF, Enc, Str) diff --git a/lib/stdlib/src/stdlib.appup.src b/lib/stdlib/src/stdlib.appup.src index 4990c81dfe..0c270e9dd5 100644 --- a/lib/stdlib/src/stdlib.appup.src +++ b/lib/stdlib/src/stdlib.appup.src @@ -40,7 +40,8 @@ {<<"^3\\.8\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.8\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.9$">>,[restart_new_emulator]}, - {<<"^3\\.9\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}], + {<<"^3\\.9\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.9\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], [{<<"^3\\.5$">>,[restart_new_emulator]}, {<<"^3\\.5\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -54,4 +55,5 @@ {<<"^3\\.8\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.8\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.9$">>,[restart_new_emulator]}, - {<<"^3\\.9\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}]}. + {<<"^3\\.9\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.9\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index dd49288417..09238ae2b4 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -75,7 +75,8 @@ -export([throughput_benchmark/0, throughput_benchmark/1, test_throughput_benchmark/1, - long_throughput_benchmark/1]). + long_throughput_benchmark/1, + lookup_catree_par_vs_seq_init_benchmark/0]). -export([exit_large_table_owner/1, exit_many_large_table_owner/1, exit_many_tables_owner/1, @@ -6728,6 +6729,14 @@ do_work(WorksDoneSoFar, Table, ProbHelpTab, Range, Operations) -> end. prefill_table(T, KeyRange, Num, ObjFun) -> + Parent = self(), + spawn_link(fun() -> + prefill_table_helper(T, KeyRange, Num, ObjFun), + Parent ! done + end), + receive done -> ok end. + +prefill_table_helper(T, KeyRange, Num, ObjFun) -> Seed = rand:uniform(KeyRange), %%io:format("prefill_table: Seed = ~p\n", [Seed]), RState = unique_rand_start(KeyRange, Seed), @@ -6740,11 +6749,77 @@ prefill_table_loop(T, RS0, N, ObjFun) -> ets:insert(T, ObjFun(Key)), prefill_table_loop(T, RS1, N-1, ObjFun). +inserter_proc_starter(T, ToInsert, Parent) -> + receive + start -> ok + end, + inserter_proc(T, ToInsert, [], Parent, false). + +inserter_proc(T, [], Inserted, Parent, _) -> + inserter_proc(T, Inserted, [], Parent, true); +inserter_proc(T, [I | ToInsert], Inserted, Parent, CanStop) -> + Stop = + case CanStop of + true -> + receive + stop -> Parent ! stopped + after 0 -> no_stop + end; + false -> no_stop + end, + case Stop of + no_stop -> + ets:insert(T, I), + inserter_proc(T, ToInsert, [I | Inserted], Parent, CanStop); + _ -> ok + end. + +prefill_table_parallel(T, KeyRange, Num, ObjFun) -> + Parent = self(), + spawn_link(fun() -> + prefill_table_parallel_helper(T, KeyRange, Num, ObjFun), + Parent ! done + end), + receive done -> ok end. + +prefill_table_parallel_helper(T, KeyRange, Num, ObjFun) -> + NrOfSchedulers = erlang:system_info(schedulers), + Seed = rand:uniform(KeyRange), + %%io:format("prefill_table: Seed = ~p\n", [Seed]), + RState = unique_rand_start(KeyRange, Seed), + InsertMap = prefill_insert_map_loop(T, RState, Num, ObjFun, #{}, NrOfSchedulers), + Self = self(), + Pids = [ + begin + InserterFun = + fun() -> + inserter_proc_starter(T, ToInsert, Self) + end, + spawn_link(InserterFun) + end + || ToInsert <- maps:values(InsertMap)], + [Pid ! start || Pid <- Pids], + timer:sleep(1000), + [Pid ! stop || Pid <- Pids], + [receive stopped -> ok end || _Pid <- Pids]. + +prefill_insert_map_loop(_, _, 0, _, InsertMap, _NrOfSchedulers) -> + InsertMap; +prefill_insert_map_loop(T, RS0, N, ObjFun, InsertMap, NrOfSchedulers) -> + {Key, RS1} = unique_rand_next(RS0), + Sched = N rem NrOfSchedulers, + PrevInserts = maps:get(Sched, InsertMap, []), + NewPrevInserts = [ObjFun(Key) | PrevInserts], + NewInsertMap = maps:put(Sched, NewPrevInserts, InsertMap), + prefill_insert_map_loop(T, RS1, N-1, ObjFun, NewInsertMap, NrOfSchedulers). + -record(ets_throughput_bench_config, {benchmark_duration_ms = 3000, recover_time_ms = 1000, thread_counts = not_set, key_ranges = [1000000], + init_functions = [fun prefill_table/4], + nr_of_repeats = 1, scenarios = [ [ @@ -6838,7 +6913,7 @@ prefill_table_loop(T, RS0, N, ObjFun) -> notify_res_fun = fun(_Name, _Throughput) -> ok end, print_result_paths_fun = fun(ResultPath, _LatestResultPath) -> - Comment = + Comment = io_lib:format("<a href=\"file:///~s\">Result visualization</a>",[ResultPath]), {comment, Comment} end @@ -6848,7 +6923,7 @@ stdout_notify_res(ResultPath, LatestResultPath) -> io:format("Result Location: /~s~n", [ResultPath]), io:format("Latest Result Location: ~s~n", [LatestResultPath]). -throughput_benchmark() -> +throughput_benchmark() -> throughput_benchmark( #ets_throughput_bench_config{ print_result_paths_fun = fun stdout_notify_res/2}). @@ -6856,9 +6931,11 @@ throughput_benchmark() -> throughput_benchmark( #ets_throughput_bench_config{ benchmark_duration_ms = BenchmarkDurationMs, - recover_time_ms = RecoverTimeMs, - thread_counts = ThreadCountsOpt, - key_ranges = KeyRanges, + recover_time_ms = RecoverTimeMs, + thread_counts = ThreadCountsOpt, + key_ranges = KeyRanges, + init_functions = InitFuns, + nr_of_repeats = NrOfRepeats, scenarios = Scenarios, table_types = TableTypes, etsmem_fun = ETSMemFun, @@ -6872,21 +6949,21 @@ throughput_benchmark( Start = rand:uniform(KeyRange), Last = lists:foldl( - fun(_, Prev) -> + fun(_, Prev) -> case Prev of '$end_of_table'-> ok; _ -> try ets:next(T, Prev) of Normal -> Normal catch - error:badarg -> + error:badarg -> % sets (not ordered_sets) cannot handle when the argument % to next is not in the set rand:uniform(KeyRange) end end end, - Start, + Start, lists:seq(1, SeqSize)), case Last =:= -1 of true -> io:format("Will never be printed"); @@ -6898,26 +6975,26 @@ throughput_benchmark( Start = rand:uniform(KeyRange), Last = Start + SeqSize, case -1 =:= ets:select_count(T, - ets:fun2ms(fun({X}) when X > Start andalso X =< Last -> true end)) of + ets:fun2ms(fun({X}) when X > Start andalso X =< Last -> true end)) of true -> io:format("Will never be printed"); false -> ok end end, %% Mapping benchmark operation names to their corresponding functions that do them - Operations = + Operations = #{insert => - fun(T,KeyRange) -> + fun(T,KeyRange) -> Num = rand:uniform(KeyRange), ets:insert(T, {Num}) end, delete => - fun(T,KeyRange) -> + fun(T,KeyRange) -> Num = rand:uniform(KeyRange), ets:delete(T, Num) end, lookup => - fun(T,KeyRange) -> + fun(T,KeyRange) -> Num = rand:uniform(KeyRange), ets:lookup(T, Num) end, @@ -6928,8 +7005,8 @@ throughput_benchmark( nextseq1000 => fun(T,KeyRange) -> NextSeqOp(T,KeyRange,1000) end, selectAll => - fun(T,_KeyRange) -> - case -1 =:= ets:select_count(T, ets:fun2ms(fun(_X) -> true end)) of + fun(T,_KeyRange) -> + case -1 =:= ets:select_count(T, ets:fun2ms(fun(_X) -> true end)) of true -> io:format("Will never be printed"); false -> ok end @@ -6951,7 +7028,7 @@ throughput_benchmark( NewCurrent = Current + OpPropability, [{NewCurrent, OpName}| Calculate(Res, NewCurrent)] end, - RenderScenario = + RenderScenario = fun R([], StringSoFar) -> StringSoFar; R([{Fraction, Operation}], StringSoFar) -> @@ -6978,7 +7055,7 @@ throughput_benchmark( false -> ok end end, - DataHolder = + DataHolder = fun DataHolderFun(Data)-> receive {get_data, Pid} -> Pid ! {ets_bench_data, Data}; @@ -6992,18 +7069,21 @@ throughput_benchmark( DataHolderPid ! io_lib:format(Str, List) end, GetData = - fun () -> + fun () -> DataHolderPid ! {get_data, self()}, receive {ets_bench_data, Data} -> Data end end, %% Function that runs a benchmark instance and returns the number %% of operations that were performed RunBenchmark = - fun({NrOfProcs, TableConfig, Scenario, Range, Duration}) -> + fun({NrOfProcs, TableConfig, Scenario, Range, Duration, InitFun}) -> ProbHelpTab = CalculateOpsProbHelpTab(Scenario, 0), Table = ets:new(t, TableConfig), Nobj = Range div 2, - prefill_table(Table, Range, Nobj, fun(K) -> {K} end), + case InitFun of + not_set -> prefill_table(Table, Range, Nobj, fun(K) -> {K} end); + _ -> InitFun(Table, Range, Nobj, fun(K) -> {K} end) + end, Nobj = ets:info(Table, size), SafeFixTableIfRequired(Table, Scenario, true), ParentPid = self(), @@ -7016,12 +7096,14 @@ throughput_benchmark( end, ChildPids = lists:map(fun(_N) ->spawn_link(Worker)end, lists:seq(1, NrOfProcs)), + erlang:garbage_collect(), + timer:sleep(RecoverTimeMs), lists:foreach(fun(Pid) -> Pid ! start end, ChildPids), timer:sleep(Duration), lists:foreach(fun(Pid) -> Pid ! stop end, ChildPids), TotalWorksDone = lists:foldl( - fun(_, Sum) -> - receive + fun(_, Sum) -> + receive Count -> Sum + Count end end, 0, ChildPids), @@ -7032,27 +7114,32 @@ throughput_benchmark( RunBenchmarkInSepProcess = fun(ParameterTuple) -> P = self(), - spawn_link(fun()-> P ! {bench_result, RunBenchmark(ParameterTuple)} end), - Result = receive {bench_result, Res} -> Res end, - timer:sleep(RecoverTimeMs), - Result + Results = + [begin + spawn_link(fun()-> P ! {bench_result, RunBenchmark(ParameterTuple)} end), + receive {bench_result, Res} -> Res end + end || _ <- lists:seq(1, NrOfRepeats)], + lists:sum(Results) / NrOfRepeats end, RunBenchmarkAndReport = fun(ThreadCount, TableType, Scenario, KeyRange, - Duration) -> + Duration, + InitFunName, + InitFun) -> Result = RunBenchmarkInSepProcess({ThreadCount, TableType, Scenario, KeyRange, - Duration}), + Duration, + InitFun}), Throughput = Result/(Duration/1000.0), PrintData("; ~f",[Throughput]), - Name = io_lib:format("Scenario: ~w, Key Range Size: ~w, " + Name = io_lib:format("Scenario: ~s, ~w, Key Range Size: ~w, " "# of Processes: ~w, Table Type: ~w", - [Scenario, KeyRange, ThreadCount, TableType]), + [InitFunName, Scenario, KeyRange, ThreadCount, TableType]), NotifyResFun(Name, Throughput) end, ThreadCounts = @@ -7087,17 +7174,29 @@ throughput_benchmark( PrintData("$~n",[]), lists:foreach( fun(TableType) -> - PrintData("~w ",[TableType]), lists:foreach( - fun(ThreadCount) -> - RunBenchmarkAndReport(ThreadCount, - TableType, - Scenario, - KeyRange, - BenchmarkDurationMs) + fun(InitFunArg) -> + {InitFunName, InitFun} = + case InitFunArg of + {FunName, Fun} -> {FunName, Fun}; + Fun -> {"", Fun} + end, + PrintData("~s,~w ",[InitFunName,TableType]), + lists:foreach( + fun(ThreadCount) -> + RunBenchmarkAndReport(ThreadCount, + TableType, + Scenario, + KeyRange, + BenchmarkDurationMs, + InitFunName, + InitFun) + end, + ThreadCounts), + PrintData("$~n",[]) end, - ThreadCounts), - PrintData("$~n",[]) + InitFuns) + end, TableTypes) end, @@ -7121,7 +7220,7 @@ throughput_benchmark( test_throughput_benchmark(Config) when is_list(Config) -> throughput_benchmark( #ets_throughput_bench_config{ - benchmark_duration_ms = 100, + benchmark_duration_ms = 100, recover_time_ms = 0, thread_counts = [1, erlang:system_info(schedulers)], key_ranges = [50000], @@ -7136,7 +7235,7 @@ long_throughput_benchmark(Config) when is_list(Config) -> recover_time_ms = 1000, thread_counts = [1, N div 2, N], key_ranges = [1000000], - scenarios = + scenarios = [ [ {0.5, insert}, @@ -7171,15 +7270,15 @@ long_throughput_benchmark(Config) when is_list(Config) -> {0.01, partial_select1000} ] ], - table_types = + table_types = [ [ordered_set, public, {write_concurrency, true}, {read_concurrency, true}], [set, public, {write_concurrency, true}, {read_concurrency, true}] ], etsmem_fun = fun etsmem/0, verify_etsmem_fun = fun verify_etsmem/1, - notify_res_fun = - fun(Name, Throughput) -> + notify_res_fun = + fun(Name, Throughput) -> SummaryTable = proplists:get_value(ets_benchmark_result_summary_tab, Config), AddToSummaryCounter = @@ -7209,13 +7308,47 @@ long_throughput_benchmark(Config) when is_list(Config) -> total_throughput_ordered_set) end, ct_event:notify( - #event{name = benchmark_data, + #event{name = benchmark_data, data = [{suite,"ets_bench"}, {name, Name}, {value,Throughput}]}) end }). +%% This function compares the lookup operation's performance for +%% ordered_set ETS tables with and without write_concurrency enabled +%% when the data structures have been populated in parallel and +%% sequentially. +%% +%% The main purpose of this function is to check that the +%% implementation of ordered_set with write_concurrency (CA tree) +%% adapts its structure to contention even when only lookup operations +%% are used. +lookup_catree_par_vs_seq_init_benchmark() -> + N = erlang:system_info(schedulers), + throughput_benchmark( + #ets_throughput_bench_config{ + benchmark_duration_ms = 600000, + recover_time_ms = 1000, + thread_counts = [1, N div 2, N], + key_ranges = [1000000], + init_functions = [{"seq_init", fun prefill_table/4}, + {"par_init", fun prefill_table_parallel/4}], + nr_of_repeats = 1, + scenarios = + [ + [ + {1.0, lookup} + ] + ], + table_types = + [ + [ordered_set, public, {write_concurrency, true}], + [ordered_set, public] + ], + print_result_paths_fun = fun stdout_notify_res/2 + }). + add_lists(L1,L2) -> add_lists(L1,L2,[]). add_lists([],[],Acc) -> diff --git a/lib/stdlib/test/io_SUITE.erl b/lib/stdlib/test/io_SUITE.erl index 2478961e59..4eb5b1772c 100644 --- a/lib/stdlib/test/io_SUITE.erl +++ b/lib/stdlib/test/io_SUITE.erl @@ -32,7 +32,7 @@ io_with_huge_message_queue/1, format_string/1, maps/1, coverage/1, otp_14178_unicode_atoms/1, otp_14175/1, otp_14285/1, limit_term/1, otp_14983/1, otp_15103/1, otp_15076/1, - otp_15159/1, otp_15639/1, otp_15705/1, otp_15847/1]). + otp_15159/1, otp_15639/1, otp_15705/1, otp_15847/1, otp_15875/1]). -export([pretty/2, trf/3]). @@ -65,7 +65,7 @@ all() -> io_lib_width_too_small, io_with_huge_message_queue, format_string, maps, coverage, otp_14178_unicode_atoms, otp_14175, otp_14285, limit_term, otp_14983, otp_15103, otp_15076, otp_15159, - otp_15639, otp_15705, otp_15847]. + otp_15639, otp_15705, otp_15847, otp_15875]. %% Error cases for output. error_1(Config) when is_list(Config) -> @@ -2714,3 +2714,7 @@ otp_15847(_Config) -> "{someRecord,<<...>>,...}" = pretty(T, [{chars_limit,20}, {encoding,latin1}]), ok. + +otp_15875(_Config) -> + S = io_lib:format("~tp", [[{0, [<<"00">>]}]], [{chars_limit, 18}]), + "[{0,[<<48,...>>]}]" = lists:flatten(S). diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index cc0ab6684c..c2f586fef5 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -1 +1 @@ -STDLIB_VSN = 3.9.1 +STDLIB_VSN = 3.9.2 diff --git a/lib/tools/doc/src/notes.xml b/lib/tools/doc/src/notes.xml index 000c6e1ce7..09ae5ef04a 100644 --- a/lib/tools/doc/src/notes.xml +++ b/lib/tools/doc/src/notes.xml @@ -186,6 +186,21 @@ </section> +<section><title>Tools 2.11.2.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p><c>cover</c> would fail to start if two processes + tried to start it at the exact same time.</p> + <p> + Own Id: OTP-15813 Aux Id: ERL-943 </p> + </item> + </list> + </section> + +</section> + <section><title>Tools 2.11.2.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/wx/c_src/wxe_driver.c b/lib/wx/c_src/wxe_driver.c index c9d299e0df..b94ec2f32d 100644 --- a/lib/wx/c_src/wxe_driver.c +++ b/lib/wx/c_src/wxe_driver.c @@ -214,7 +214,7 @@ standard_outputv(ErlDrvData drv_data, ErlIOVec* ev) if(binref == NULL) { /* realloc */ max = sd->max_bins + DEF_BINS; - driver_realloc(sd->bin, sizeof(WXEBinRef)*max); + sd->bin = driver_realloc(sd->bin, sizeof(WXEBinRef)*max); for(i=sd->max_bins; i < max; i++) { sd->bin[i].from = 0; } diff --git a/make/otp_version_tickets_in_merge b/make/otp_version_tickets_in_merge index a9ca1cb131..fd9b36720a 100644 --- a/make/otp_version_tickets_in_merge +++ b/make/otp_version_tickets_in_merge @@ -1,11 +1,6 @@ -OTP-15828 -OTP-15829 -OTP-15832 -OTP-15834 -OTP-15838 -OTP-15839 -OTP-15841 -OTP-15845 -OTP-15846 -OTP-15847 -OTP-15848 +OTP-15805 +OTP-15819 +OTP-15867 +OTP-15879 +OTP-15887 +OTP-15888 diff --git a/otp_versions.table b/otp_versions.table index 04e612fc90..92e04a3035 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,6 +1,10 @@ +OTP-22.0.4 : erts-10.4.3 kernel-6.4.1 ssl-9.3.3 # asn1-5.0.9 common_test-1.17.3 compiler-7.4.2 crypto-4.5.1 debugger-4.2.7 dialyzer-4.0.1 diameter-2.2.1 edoc-0.11 eldap-1.2.8 erl_docgen-0.9.1 erl_interface-3.12 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.19 inets-7.0.8 jinterface-1.10 megaco-3.18.5 mnesia-4.16 observer-2.9.1 odbc-2.12.4 os_mon-2.5 parsetools-2.1.8 public_key-1.6.7 reltool-0.8 runtime_tools-1.13.3 sasl-3.4 snmp-5.3 ssh-4.7.7 stdlib-3.9.2 syntax_tools-2.2 tftp-1.0.1 tools-3.2 wx-1.8.8 xmerl-1.3.21 : +OTP-22.0.3 : compiler-7.4.2 dialyzer-4.0.1 erts-10.4.2 ssl-9.3.2 stdlib-3.9.2 # asn1-5.0.9 common_test-1.17.3 crypto-4.5.1 debugger-4.2.7 diameter-2.2.1 edoc-0.11 eldap-1.2.8 erl_docgen-0.9.1 erl_interface-3.12 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.19 inets-7.0.8 jinterface-1.10 kernel-6.4 megaco-3.18.5 mnesia-4.16 observer-2.9.1 odbc-2.12.4 os_mon-2.5 parsetools-2.1.8 public_key-1.6.7 reltool-0.8 runtime_tools-1.13.3 sasl-3.4 snmp-5.3 ssh-4.7.7 syntax_tools-2.2 tftp-1.0.1 tools-3.2 wx-1.8.8 xmerl-1.3.21 : OTP-22.0.2 : compiler-7.4.1 crypto-4.5.1 erts-10.4.1 stdlib-3.9.1 # asn1-5.0.9 common_test-1.17.3 debugger-4.2.7 dialyzer-4.0 diameter-2.2.1 edoc-0.11 eldap-1.2.8 erl_docgen-0.9.1 erl_interface-3.12 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.19 inets-7.0.8 jinterface-1.10 kernel-6.4 megaco-3.18.5 mnesia-4.16 observer-2.9.1 odbc-2.12.4 os_mon-2.5 parsetools-2.1.8 public_key-1.6.7 reltool-0.8 runtime_tools-1.13.3 sasl-3.4 snmp-5.3 ssh-4.7.7 ssl-9.3.1 syntax_tools-2.2 tftp-1.0.1 tools-3.2 wx-1.8.8 xmerl-1.3.21 : OTP-22.0.1 : ssl-9.3.1 # asn1-5.0.9 common_test-1.17.3 compiler-7.4 crypto-4.5 debugger-4.2.7 dialyzer-4.0 diameter-2.2.1 edoc-0.11 eldap-1.2.8 erl_docgen-0.9.1 erl_interface-3.12 erts-10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.19 inets-7.0.8 jinterface-1.10 kernel-6.4 megaco-3.18.5 mnesia-4.16 observer-2.9.1 odbc-2.12.4 os_mon-2.5 parsetools-2.1.8 public_key-1.6.7 reltool-0.8 runtime_tools-1.13.3 sasl-3.4 snmp-5.3 ssh-4.7.7 stdlib-3.9 syntax_tools-2.2 tftp-1.0.1 tools-3.2 wx-1.8.8 xmerl-1.3.21 : OTP-22.0 : asn1-5.0.9 common_test-1.17.3 compiler-7.4 crypto-4.5 debugger-4.2.7 dialyzer-4.0 edoc-0.11 eldap-1.2.8 erl_docgen-0.9.1 erl_interface-3.12 erts-10.4 hipe-3.19 inets-7.0.8 jinterface-1.10 kernel-6.4 megaco-3.18.5 mnesia-4.16 observer-2.9.1 odbc-2.12.4 os_mon-2.5 public_key-1.6.7 reltool-0.8 runtime_tools-1.13.3 sasl-3.4 snmp-5.3 ssh-4.7.7 ssl-9.3 stdlib-3.9 syntax_tools-2.2 tools-3.2 wx-1.8.8 xmerl-1.3.21 # diameter-2.2.1 et-1.6.4 eunit-2.3.7 ftp-1.0.2 parsetools-2.1.8 tftp-1.0.1 : +OTP-21.3.8.4 : common_test-1.17.2.1 erts-10.3.5.3 kernel-6.3.1.2 public_key-1.6.6.1 ssl-9.2.3.3 stdlib-3.8.2.2 # asn1-5.0.8 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 inets-7.0.7 jinterface-1.9.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6 syntax_tools-2.1.7 tftp-1.0.1 tools-3.1.0.1 wx-1.8.7 xmerl-1.3.20.1 : +OTP-21.3.8.3 : erts-10.3.5.2 kernel-6.3.1.1 ssl-9.2.3.2 stdlib-3.8.2.1 # asn1-5.0.8 common_test-1.17.2 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 inets-7.0.7 jinterface-1.9.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.6 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6 syntax_tools-2.1.7 tftp-1.0.1 tools-3.1.0.1 wx-1.8.7 xmerl-1.3.20.1 : OTP-21.3.8.2 : xmerl-1.3.20.1 # asn1-5.0.8 common_test-1.17.2 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 erts-10.3.5.1 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 inets-7.0.7 jinterface-1.9.1 kernel-6.3.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.6 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6 ssl-9.2.3.1 stdlib-3.8.2 syntax_tools-2.1.7 tftp-1.0.1 tools-3.1.0.1 wx-1.8.7 : OTP-21.3.8.1 : erts-10.3.5.1 ssl-9.2.3.1 tools-3.1.0.1 # asn1-5.0.8 common_test-1.17.2 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 inets-7.0.7 jinterface-1.9.1 kernel-6.3.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.6 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6 stdlib-3.8.2 syntax_tools-2.1.7 tftp-1.0.1 wx-1.8.7 xmerl-1.3.20 : OTP-21.3.8 : common_test-1.17.2 eldap-1.2.7 erl_interface-3.11.3 erts-10.3.5 public_key-1.6.6 ssl-9.2.3 stdlib-3.8.2 # asn1-5.0.8 compiler-7.3.2 crypto-4.4.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 erl_docgen-0.9 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.18.3 inets-7.0.7 jinterface-1.9.1 kernel-6.3.1 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6 syntax_tools-2.1.7 tftp-1.0.1 tools-3.1 wx-1.8.7 xmerl-1.3.20 : @@ -36,6 +40,7 @@ OTP-21.0.3 : erts-10.0.3 # asn1-5.0.6 common_test-1.16 compiler-7.2.2 crypto-4.3 OTP-21.0.2 : compiler-7.2.2 erts-10.0.2 public_key-1.6.1 stdlib-3.5.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0.1 : compiler-7.2.1 erts-10.0.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0 : asn1-5.0.6 common_test-1.16 compiler-7.2 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 erts-10.0 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 mnesia-4.15.4 observer-2.8 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 # megaco-3.18.3 odbc-2.12.1 snmp-5.2.11 : +OTP-20.3.8.22 : common_test-1.15.4.3 erts-9.3.3.11 tools-2.11.2.2 # asn1-5.0.5.2 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4.1 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.2 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11.1 ssh-4.6.9.4 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 wx-1.8.3 xmerl-1.3.16.1 : OTP-20.3.8.21 : common_test-1.15.4.2 erl_interface-3.10.2.2 erts-9.3.3.10 snmp-5.2.11.1 ssh-4.6.9.4 tools-2.11.2.1 # asn1-5.0.5.2 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4.1 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 wx-1.8.3 xmerl-1.3.16.1 : OTP-20.3.8.20 : common_test-1.15.4.1 # asn1-5.0.5.2 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4.1 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.9 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.3 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16.1 : OTP-20.3.8.19 : diameter-2.1.4.1 erts-9.3.3.9 # asn1-5.0.5.2 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.3 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16.1 : diff --git a/system/doc/efficiency_guide/commoncaveats.xml b/system/doc/efficiency_guide/commoncaveats.xml index 7b2128d888..7b00fa5d63 100644 --- a/system/doc/efficiency_guide/commoncaveats.xml +++ b/system/doc/efficiency_guide/commoncaveats.xml @@ -62,7 +62,7 @@ dangerous in a system that runs continuously. If only certain well-defined atoms are allowed as input, <seealso marker="erts:erlang#list_to_existing_atom/1">list_to_existing_atom/1</seealso> - can be used to + can be used to guard against a denial-of-service attack. (All atoms that are allowed must have been created earlier, for example, by simply using all of them in a module and loading that module.)</p> |