aboutsummaryrefslogtreecommitdiffstats
path: root/erts/doc/src/erl_dist_protocol.xml
diff options
context:
space:
mode:
Diffstat (limited to 'erts/doc/src/erl_dist_protocol.xml')
-rw-r--r--erts/doc/src/erl_dist_protocol.xml288
1 files changed, 282 insertions, 6 deletions
diff --git a/erts/doc/src/erl_dist_protocol.xml b/erts/doc/src/erl_dist_protocol.xml
index 6c725fc82d..0252187be5 100644
--- a/erts/doc/src/erl_dist_protocol.xml
+++ b/erts/doc/src/erl_dist_protocol.xml
@@ -547,13 +547,289 @@ If Result > 0, the packet only consists of [119, Result].
-->
</section>
-
+ <marker id="distribution_handshake"/>
<section>
- <title>Handshake</title>
- <p>
- The handshake is discussed in detail in the internal documentation for
- the kernel (Erlang) application.
- </p>
+ <title>Distribution Handshake</title>
+ <p>
+ This section describes the distribution handshake protocol
+ introduced in the OTP-R6 release of Erlang/OTP. This
+ description was previously located in
+ <c>$ERL_TOP/lib/kernel/internal_doc/distribution_handshake.txt</c>,
+ and has more or less been copied and "formatted" here. It has been
+ more or less unchanged since the year 1999, but the handshake
+ should not have changed much since then either.
+ </p>
+ <section>
+ <title>General</title>
+ <p>
+ The TCP/IP distribution uses a handshake which expects a
+ connection based protocol, i.e. the protocol does not include
+ any authentication after the handshake procedure.
+ </p>
+ <p>
+ This is not entirely safe, as it is vulnerable against takeover
+ attacks, but it is a tradeoff between fair safety and performance.
+ </p>
+ <p>
+ The cookies are never sent in cleartext and the handshake procedure
+ expects the client (called A) to be the first one to prove that it can
+ generate a sufficient digest. The digest is generated with the
+ MD5 message digest algorithm and the challenges are expected to be very
+ random numbers.
+ </p>
+ </section>
+ <section>
+ <title>Definitions</title>
+ <p>
+ A challenge is a 32 bit integer number in big endian order. Below the function
+ <c>gen_challenge()</c> returns a random 32 bit integer used as a challenge.
+ </p>
+ <p>
+ A digest is a (16 bytes) MD5 hash of the Challenge (as text) concatenated
+ with the cookie (as text). Below, the function <c>gen_digest(Challenge, Cookie)</c>
+ generates a digest as described above.
+ </p>
+ <p>
+ An out_cookie is the cookie used in outgoing communication to a certain node,
+ so that A's out_cookie for B should correspond with B's in_cookie for A and
+ the other way around. A's out_cookie for B and A's in_cookie for B need <em>NOT</em>
+ be the same. Below the function <c>out_cookie(Node)</c> returns the current
+ node's out_cookie for <c>Node</c>.
+ </p>
+ <p>
+ An in_cookie is the cookie expected to be used by another node when
+ communicating with us, so that A's in_cookie for B corresponds with B's
+ out_cookie for A. Below the function <c>in_cookie(Node)</c> returns the current
+ node's <c>in_cookie</c> for <c>Node</c>.
+ </p>
+ <p>
+ The cookies are text strings that can be viewed as passwords.
+ </p>
+ <p>
+ Every message in the handshake starts with a 16 bit big endian integer
+ which contains the length of the message (not counting the two initial bytes).
+ In erlang this corresponds to the <c>gen_tcp</c> option <c>{packet, 2}</c>. Note that after
+ the handshake, the distribution switches to 4 byte packet headers.
+ </p>
+
+ </section>
+ <section>
+ <title>The Handshake in Detail</title>
+ <p>
+ Imagine two nodes, node A, which initiates the handshake and node B, which
+ accepts the connection.
+ </p>
+ <taglist>
+ <tag>1) connect/accept</tag>
+ <item><p>A connects to B via TCP/IP and B accepts the connection.</p></item>
+ <tag>2) send_name/receive_name</tag>
+ <item><p>A sends an initial identification to B. B receives the message.
+ The message looks like this (every "square" being one byte and the packet
+ header removed):
+ </p>
+<pre>
++---+--------+--------+-----+-----+-----+-----+-----+-----+-...-+-----+
+|'n'|Version0|Version1|Flag0|Flag1|Flag2|Flag3|Name0|Name1| ... |NameN|
++---+--------+--------+-----+-----+-----+-----+-----+-----+-... +-----+
+</pre>
+ <p>
+ The 'n' is just a message tag.
+ Version0 and Version1 is the distribution version selected by node A,
+ based on information from EPMD. (16 bit big endian)
+ Flag0 ... Flag3 are capability flags, the capabilities defined in
+ <c>$ERL_TOP/lib/kernel/include/dist.hrl</c>.
+ (32 bit big endian)
+ Name0 ... NameN is the full nodename of A, as a string of bytes (the
+ packet length denotes how long it is).
+ </p></item>
+ <tag>3) recv_status/send_status</tag>
+ <item><p>B sends a status message to A, which indicates
+ if the connection is allowed. The following status codes are defined:</p>
+ <taglist>
+ <tag><c>ok</c></tag>
+ <item>The handshake will continue.</item>
+ <tag><c>ok_simultaneous</c></tag>
+ <item>The handshake will continue, but A is informed that B
+ has another ongoing connection attempt that will be
+ shut down (simultaneous connect where A's name is
+ greater than B's name, compared literally).</item>
+ <tag><c>nok</c></tag>
+ <item>The handshake will not continue, as B already has an ongoing handshake
+ which it itself has initiated. (simultaneous connect where B's name is
+ greater than A's).</item>
+ <tag><c>not_allowed</c></tag>
+ <item>The connection is disallowed for some (unspecified) security
+ reason.</item>
+ <tag><c>alive</c></tag>
+ <item>A connection to the node is already active, which either means
+ that node A is confused or that the TCP connection breakdown
+ of a previous node with this name has not yet reached node B.
+ See 3B below.</item>
+ </taglist>
+ <p>This is the format of the status message:</p>
+<pre>
++---+-------+-------+-...-+-------+
+|'s'|Status0|Status1| ... |StatusN|
++---+-------+-------+-...-+-------+
+</pre>
+ <p>
+ 's' is the message tag Status0 ... StatusN is the status as a string (not terminated)
+ </p>
+ </item>
+ <tag>3B) send_status/recv_status</tag>
+ <item><p>If status was 'alive', node A will answer with
+ another status message containing either 'true' which means that the
+ connection should continue (The old connection from this node is broken), or
+ <c>'false'</c>, which simply means that the connection should be closed, the
+ connection attempt was a mistake.</p></item>
+ <tag>4) recv_challenge/send_challenge</tag>
+ <item><p>If the status was <c>ok</c> or <c>ok_simultaneous</c>,
+ The handshake continues with B sending A another message, the challenge.
+ The challenge contains the same type of information as the "name" message
+ initially sent from A to B, with the addition of a 32 bit challenge:</p>
+<pre>
++---+--------+--------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-...-+-----+
+|'n'|Version0|Version1|Flag0|Flag1|Flag2|Flag3|Chal0|Chal1|Chal2|Chal3|Name0|Name1| ... |NameN|
++---+--------+--------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-... +-----+
+</pre>
+ <p>
+ Where Chal0 ... Chal3 is the challenge as a 32 bit big endian integer
+ and the other fields are B's version, flags and full nodename.
+ </p></item>
+ <tag>5) send_challenge_reply/recv_challenge_reply</tag>
+ <item><p>Now A has generated a digest and its own challenge. Those are
+ sent together in a package to B:</p>
+<pre>
++---+-----+-----+-----+-----+-----+-----+-----+-----+-...-+------+
+|'r'|Chal0|Chal1|Chal2|Chal3|Dige0|Dige1|Dige2|Dige3| ... |Dige15|
++---+-----+-----+-----+-----+-----+-----+-----+-----+-...-+------+
+</pre>
+ <p>
+ Where 'r' is the tag, Chal0 ... Chal3 is A's challenge for B to handle and
+ Dige0 ... Dige15 is the digest that A constructed from the challenge B sent
+ in the previous step.
+ </p></item>
+ <tag>6) recv_challenge_ack/send_challenge_ack</tag>
+ <item><p>B checks that the digest received from A is correct and generates a
+ digest from the challenge received from A. The digest is then sent to A. The
+ message looks like this:</p>
+<pre>
++---+-----+-----+-----+-----+-...-+------+
+|'a'|Dige0|Dige1|Dige2|Dige3| ... |Dige15|
++---+-----+-----+-----+-----+-...-+------+
+</pre>
+ <p>
+ Where 'a' is the tag and Dige0 ... Dige15 is the digest calculated by B
+ for A's challenge.</p></item>
+ <tag>7)</tag>
+ <item><p>A checks the digest from B and the connection is up.</p></item>
+ </taglist>
+ </section>
+ <section>
+ <title>Semigraphic View</title>
+<pre>
+A (initiator) B (acceptor)
+
+TCP connect -----------------------------------------&gt;
+ TCP accept
+
+send_name -----------------------------------------&gt;
+ recv_name
+
+ &lt;---------------------------------------- send_status
+recv_status
+(if status was 'alive'
+ send_status - - - - - - - - - - - - - - - - - - - -&gt;
+ recv_status)
+ ChB = gen_challenge()
+ (ChB)
+ &lt;---------------------------------------- send_challenge
+recv_challenge
+
+ChA = gen_challenge(),
+OCA = out_cookie(B),
+DiA = gen_digest(ChB,OCA)
+ (ChA, DiA)
+send_challenge_reply --------------------------------&gt;
+ recv_challenge_reply
+ ICB = in_cookie(A),
+ check:
+ DiA == gen_digest
+ (ChB, ICB) ?
+ - if OK:
+ OCB = out_cookie(A),
+ DiB = gen_digest
+ (DiB) (ChA, OCB)
+ &lt;----------------------------------------- send_challenge_ack
+recv_challenge_ack DONE
+ICA = in_cookie(B), - else
+check: CLOSE
+DiB == gen_digest(ChA,ICA) ?
+- if OK
+ DONE
+- else
+ CLOSE
+</pre>
+ </section>
+ <marker id="dflags"/>
+ <section>
+ <title>The Currently Defined Distribution Flags</title>
+ <p>
+ Currently (OTP-R16) the following capability flags are defined:
+ </p>
+<pre>
+%% The node should be published and part of the global namespace
+-define(DFLAG_PUBLISHED,1).
+
+%% The node implements an atom cache (obsolete)
+-define(DFLAG_ATOM_CACHE,2).
+
+%% The node implements extended (3 * 32 bits) references. This is
+%% required today. If not present connection will be refused.
+-define(DFLAG_EXTENDED_REFERENCES,4).
+
+%% The node implements distributed process monitoring.
+-define(DFLAG_DIST_MONITOR,8).
+
+%% The node uses separate tag for fun's (lambdas) in the distribution protocol.
+-define(DFLAG_FUN_TAGS,16#10).
+
+%% The node implements distributed named process monitoring.
+-define(DFLAG_DIST_MONITOR_NAME,16#20).
+
+%% The (hidden) node implements atom cache (obsolete)
+-define(DFLAG_HIDDEN_ATOM_CACHE,16#40).
+
+%% The node understand new fun-tags
+-define(DFLAG_NEW_FUN_TAGS,16#80).
+
+%% The node is capable of handling extended pids and ports. This is
+%% required today. If not present connection will be refused.
+-define(DFLAG_EXTENDED_PIDS_PORTS,16#100).
+
+%%
+-define(DFLAG_EXPORT_PTR_TAG,16#200).
+
+%%
+-define(DFLAG_BIT_BINARIES,16#400).
+
+%% The node understands new float format
+-define(DFLAG_NEW_FLOATS,16#800).
+
+%%
+-define(DFLAG_UNICODE_IO,16#1000).
+
+%% The node implements atom cache in distribution header.
+-define(DFLAG_DIST_HDR_ATOM_CACHE,16#2000).
+
+%% The node understand the SMALL_ATOM_EXT tag
+-define(DFLAG_SMALL_ATOM_TAGS, 16#4000).
+
+%% The node understand UTF-8 encoded atoms
+-define(DFLAG_UTF8_ATOMS, 16#10000).
+
+</pre>
+ </section>
</section>
<section>