Introduce sender in distributed signals and dflag configuration

3 files changed, 143 insertions, 15 deletions

diff --git a/erts/doc/src/alt_dist.xml b/erts/doc/src/alt_dist.xml
index 300f75dc13..d3731a5391 100644
--- a/erts/doc/src/alt_dist.xml
+++ b/erts/doc/src/alt_dist.xml
@@ -760,11 +760,80 @@
 	  </p>
 	  <p>Only used during handshake phase.</p>
 	</item>
+	
+	<tag><marker id="hs_data_add_flags"/><c>add_flags</c></tag>
+	<item>
+	  <p>
+	    <seealso marker="erl_dist_protocol#dflags">Distribution flags</seealso>
+	    to add to the connection. Currently all (non obsolete) flags will
+	    automatically be enabled.
+	  </p>
+	  <p>
+	    This flag field is optional.
+	  </p>
+	</item>
+
+	<tag><marker id="hs_data_reject_flags"/><c>reject_flags</c></tag>
+	<item>
+	  <p>
+	    <seealso marker="erl_dist_protocol#dflags">Distribution flags</seealso>
+	    to reject. Currently the following distribution flags can be rejected:
+	  </p>
+	  <taglist>
+	    <tag><c>DFLAG_DIST_HDR_ATOM_CACHE</c></tag>
+	    <item>Do not use atom cache over this connection.</item>
+	    <tag><c>DFLAGS_STRICT_ORDER_DELIVERY</c></tag>
+	    <item>Do not use any features that require strict
+	    order delivery.</item>
+	  </taglist>
+	  <p>
+	    This flag field is optional.
+	  </p>
+	</item>
+
+	<tag><marker id="hs_data_require_flags"/><c>require_flags</c></tag>
+	<item>
+	  <p>
+	    Require these <seealso marker="erl_dist_protocol#dflags">distribution
+	    flags</seealso> to be used. The connection will be aborted during the
+	    handshake if the other end does not use them.
+	  </p>
+	  <p>
+	    This flag field is optional.
+	  </p>
+	</item>
 
       </taglist>
     </section>
 
     <section>
+      <marker id="distribution_data_delivery"/>
+      <title>Distribution Data Delivery</title>
+      <p>
+	When using the default configuration, the data to pass
+	over a connection needs to be delivered as is
+	to the node on the receiving end in the <em>exact same
+	order</em>, with no loss of data what so ever, as sent
+	from the sending node.
+      </p>
+      <p>
+	The data delivery order can be relaxed by disabling
+	features that require strict ordering. This is done by
+	passing the <c>?DFLAGS_STRICT_ORDER_DELIVERY</c>
+	<seealso marker="erl_dist_protocol#dflags">distribution
+	flags</seealso> in the
+	<seealso marker="alt_dist#hs_data_reject_flags"><c>reject_flags</c></seealso>
+	field of the <seealso marker="#hs_data_record"><c>#hs_data{}</c></seealso>
+	record used when setting up the connection. When relaxed
+	ordering is used, only the order of signals with the same
+	sender/receiver pair has to be preserved.
+	However, note that disabling the features that require
+	strict ordering may have a negative impact on performance,
+	throughput, and/or latency. 
+      </p>
+    </section>
+
+    <section>
       <marker id="enable_your_distribution_module"/>
       <title>Enable Your Distribution Module</title>
 
diff --git a/erts/doc/src/erl_dist_protocol.xml b/erts/doc/src/erl_dist_protocol.xml
index 610351db6c..a78b13aaa4 100644
--- a/erts/doc/src/erl_dist_protocol.xml
+++ b/erts/doc/src/erl_dist_protocol.xml
@@ -829,7 +829,31 @@ DiB == gen_digest(ChA, ICA)?
         <item>
           <p>The node understand UTF-8 encoded atoms.</p>
         </item>
+        <tag><c>-define(DFLAG_MAP_TAG, 16#20000).</c></tag>
+        <item>
+          <p>The node understand the map tag.</p>
+        </item>
+        <tag><c>-define(DFLAG_BIG_CREATION, 16#40000).</c></tag>
+        <item>
+          <p>The node understand big node creation.</p>
+        </item>
+        <tag><c>-define(DFLAG_SEND_SENDER, 16#80000).</c></tag>
+        <item>
+          <p>
+	    Use the <c>SEND_SENDER</c>
+	    <seealso marker="#control_message">control message</seealso>
+	    instead of the <c>SEND</c> control message and use the
+	    <c>SEND_SENDER_TT</c> control message instead
+	    of the <c>SEND_TT</c> control message.
+	  </p>
+        </item>
       </taglist>
+      <p>
+	There are also a collection of <c>DFLAG</c>s bitwise or:ed
+	together in the <c>DFLAGS_STRICT_ORDER_DELIVERY</c> macro.
+	These flags corresponds to features that require strict
+	ordering of data over distribution channels.
+      </p>
     </section>
   </section>
 
@@ -922,6 +946,7 @@ DiB == gen_digest(ChA, ICA)?
       </item>
     </taglist>
 
+    <marker id="control_message"/>
     <p>The <c>ControlMessage</c> is a tuple, where the first element indicates
       which distributed operation it encodes:</p>
 
@@ -1028,4 +1053,49 @@ DiB == gen_digest(ChA, ICA)?
       </item>
     </taglist>
   </section>
+
+  <section>
+    <title>New Ctrlmessages for Erlang/OTP 21</title>
+    <taglist>
+      <tag><c>SEND_SENDER</c></tag>
+      <item>
+        <p><c>{22, FromPid, ToPid}</c></p>
+        <p>Followed by <c>Message</c>.</p>
+	<p>
+	  This control messages replace the <c>SEND</c> control
+	  message and will be sent when the distribution flag
+	  <seealso marker="erl_dist_protocol#dflags"><c>DFLAG_SEND_SENDER</c></seealso>
+	  has been negotiated in the connection setup handshake.
+	</p>
+	<note><p>
+	  Messages encoded before the connection has
+	  been set up may still use the <c>SEND</c> control
+	  message. However, once a <c>SEND_SENDER</c> or <c>SEND_SENDER_TT</c>
+	  control message has been sent, no more <c>SEND</c>
+	  control messages will be sent in the same direction
+	  on the connection.
+	</p></note>
+      </item>
+      <tag><c>SEND_SENDER_TT</c></tag>
+      <item>
+        <p><c>{23, FromPid, ToPid, TraceToken}</c></p>
+        <p>Followed by <c>Message</c>.</p>
+	<p>
+	  This control messages replace the <c>SEND_TT</c> control
+	  message and will be sent when the distribution flag
+	  <seealso marker="erl_dist_protocol#dflags"><c>DFLAG_SEND_SENDER</c></seealso>
+	  has been negotiated in the connection setup handshake.
+	</p>
+	<note><p>
+	  Messages encoded before the connection has
+	  been set up may still use the <c>SEND_TT</c> control
+	  message. However, once a <c>SEND_SENDER</c> or <c>SEND_SENDER_TT</c>
+	  control message has been sent, no more <c>SEND_TT</c>
+	  control messages will be sent in the same direction
+	  on the connection.
+	</p></note>
+      </item>
+    </taglist>
+  </section>
+
 </chapter>
diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml
index 8b0f97f30f..f1fcbd7532 100644
--- a/erts/doc/src/erlang.xml
+++ b/erts/doc/src/erlang.xml
@@ -1236,12 +1236,6 @@ end</code>
 	  <seealso marker="erlang#dist_ctrl_get_data_notification/1"><c>erlang:dist_ctrl_get_data_notification(DHandle)</c></seealso>.
 	</p>
 	<note><p>
-	  The data retreived from this function needs to be delivered
-	  as is to the node on the other end in the exact same order,
-	  with no loss of data what so ever, as retrived from this
-	  function.
-	</p></note>
-	<note><p>
 	  Only the process registered as distribution
 	  controller for the distribution channel identified by
 	  <c><anno>DHandle</anno></c> is allowed to call this
@@ -1253,7 +1247,7 @@ end</code>
 	  controllers. <c><anno>DHandle</anno></c> is retrived
 	  via the callback
 	  <seealso marker="erts:alt_dist#hs_data_f_handshake_complete"><c>f_handshake_complete</c></seealso>.
-	  More information about this can be found in the documentation of
+	  More information can be found in the documentation of
 	  <seealso marker="erts:alt_dist#distribution_module">ERTS
 	  User's Guide ➜ How to implement an Alternative Carrier
 	  for the Erlang Distribution ➜ Distribution Module</seealso>.
@@ -1289,7 +1283,7 @@ end</code>
 	  controllers. <c><anno>DHandle</anno></c> is retrived
 	  via the callback
 	  <seealso marker="erts:alt_dist#hs_data_f_handshake_complete"><c>f_handshake_complete</c></seealso>.
-	  More information about this can be found in the documentation of
+	  More information can be found in the documentation of
 	  <seealso marker="erts:alt_dist#distribution_module">ERTS
 	  User's Guide ➜ How to implement an Alternative Carrier
 	  for the Erlang Distribution ➜ Distribution Module</seealso>.
@@ -1322,7 +1316,7 @@ end</code>
 	  controllers. <c><anno>DHandle</anno></c> is retrived
 	  via the callback
 	  <seealso marker="erts:alt_dist#hs_data_f_handshake_complete"><c>f_handshake_complete</c></seealso>.
-	  More information about this can be found in the documentation of
+	  More information can be found in the documentation of
 	  <seealso marker="erts:alt_dist#distribution_module">ERTS
 	  User's Guide ➜ How to implement an Alternative Carrier
 	  for the Erlang Distribution ➜ Distribution Module</seealso>.
@@ -1339,11 +1333,6 @@ end</code>
 	  local node.
 	</p>
 	<note><p>
-	  The data passed to the VM using this function needs to be
-	  passed in the exact same order, and with no loss of data
-	  what so ever, as sent from the node on the other end.
-	</p></note>
-	<note><p>
 	  Only the process registered as distribution
 	  controller for the distribution channel identified by
 	  <c><anno>DHandle</anno></c> is allowed to call this
@@ -1360,7 +1349,7 @@ end</code>
 	  controllers. <c><anno>DHandle</anno></c> is retrived
 	  via the callback
 	  <seealso marker="erts:alt_dist#hs_data_f_handshake_complete"><c>f_handshake_complete</c></seealso>.
-	  More information about this can be found in the documentation of
+	  More information can be found in the documentation of
 	  <seealso marker="erts:alt_dist#distribution_module">ERTS
 	  User's Guide ➜ How to implement an Alternative Carrier
 	  for the Erlang Distribution ➜ Distribution Module</seealso>.