aboutsummaryrefslogtreecommitdiffstats
path: root/erts/doc
diff options
context:
space:
mode:
Diffstat (limited to 'erts/doc')
-rw-r--r--erts/doc/src/Makefile2
-rw-r--r--erts/doc/src/alt_dist.xml736
-rw-r--r--erts/doc/src/erl.xml42
-rw-r--r--erts/doc/src/erl_dist_protocol.xml70
-rw-r--r--erts/doc/src/erl_nif.xml284
-rw-r--r--erts/doc/src/erlang.xml230
-rw-r--r--erts/doc/src/notes.xml122
-rw-r--r--erts/doc/src/zlib.xml227
8 files changed, 1580 insertions, 133 deletions
diff --git a/erts/doc/src/Makefile b/erts/doc/src/Makefile
index b96cbbce40..444adf4a6e 100644
--- a/erts/doc/src/Makefile
+++ b/erts/doc/src/Makefile
@@ -173,6 +173,8 @@ release_docs_spec: docs
"$(RELSYSDIR)/doc/html"
$(INSTALL_DATA) $(ERL_TOP)/erts/example/time_compat.erl \
"$(RELSYSDIR)/doc/html"
+ $(INSTALL_DATA) $(ERL_TOP)/lib/kernel/examples/gen_tcp_dist/src/gen_tcp_dist.erl \
+ "$(RELSYSDIR)/doc/html"
$(INSTALL_DATA) $(INFO_FILE) "$(RELSYSDIR)"
$(INSTALL_DIR) "$(RELEASE_PATH)/man/man3"
$(INSTALL_DATA) $(MAN3DIR)/* "$(RELEASE_PATH)/man/man3"
diff --git a/erts/doc/src/alt_dist.xml b/erts/doc/src/alt_dist.xml
index be969a8267..d3731a5391 100644
--- a/erts/doc/src/alt_dist.xml
+++ b/erts/doc/src/alt_dist.xml
@@ -47,23 +47,30 @@
runs on. The reason the C code is not made portable, is simply
readability.</p>
- <note>
- <p>This section was written a long time ago. Most of it is still
- valid, but some things have changed since then.
- Most notably is the driver interface. Some updates have been made
- to the documentation of the driver presented here,
- but more can be done and is planned for the future.
- The reader is encouraged to read the
- <seealso marker="erl_driver"><c>erl_driver</c></seealso> and
- <seealso marker="driver_entry"><c>driver_entry</c></seealso>
- documentation also.</p>
- </note>
-
<section>
<title>Introduction</title>
<p>To implement a new carrier for the Erlang distribution, the main
steps are as follows.</p>
+ <note><p>
+ As of ERTS version 10.0 support for distribution controller
+ processes has been introduced. That is, the traffic over a
+ distribution channel can be managed by a process instead of
+ only by a port. This makes it possible to implement large
+ parts of the logic in Erlang code, and you perhaps do not
+ even need a new driver for the protocol. One example could
+ be Erlang distribution over UDP using <c>gen_udp</c> (your
+ Erlang code will of course have to take care of retranspissions,
+ etc in this example). That is, depending on what you want
+ to do you perhaps do not need to implement a driver at all
+ and can then skip the driver related sections below.
+ The <c>gen_tcp_dist</c> example described in the
+ <seealso marker="#distribution_module">Distribution
+ Module</seealso> section utilize distribution controller
+ processes and can be worth having a look at if you want to
+ use distribution controller processes.
+ </p></note>
+
<section>
<title>Writing an Erlang Driver</title>
<p>First, the protocol must be available to the Erlang machine, which
@@ -152,7 +159,712 @@
</section>
<section>
+ <marker id="distribution_module"/>
+ <title>Distribution Module</title>
+ <p>
+ The distribution module expose an API that <c>net_kernel</c> call
+ in order to manage connections to other nodes. The module name
+ should have the suffix <c>_dist</c>.
+ </p>
+ <p>
+ The module needs to create some kind of listening entity (process
+ or port) and an acceptor process that accepts incoming connections
+ using the listening entity. For each connection, the module at least
+ needs to create one connection supervisor process, which also is
+ responsible for the handshake when setting up the connection, and
+ a distribution controller (process or port) responsible for
+ transport of data over the connection. The distribution controller
+ and the connection supervisor process should be linked together
+ so both of them are cleaned up when the connection is taken down.
+ </p>
+ <p>
+ Note that there need to be exactly one distribution controller
+ per connection. A process or port can only be distribution
+ controller for one connection. The registration as distribution
+ controller cannot be undone. It will stick until the distribution
+ controller terminates. The distribution controller should not
+ ignore exit signals. It is allowed to trap exits, but it should
+ then voluntarily terminate when an exit signal is received.
+ </p>
+ <p>
+ An example implementation of a distribution module can be found
+ in
+ <url href="gen_tcp_dist.erl">$ERL_TOP/lib/kernel/examples/gen_tcp_dist/src/gen_tcp_dist.erl</url>.
+ It implements the distribution over TCP/IP using the <c>gen_tcp</c>
+ API with distribution controllers implemented by processes. This
+ instead of using port distribution controllers as the ordinary TCP/IP
+ distribution uses.
+ </p>
+
+ <section>
+ <marker id="distribution_module_exported_callback_functions"/>
+ <title>Exported Callback Functions</title>
+
+ <p>
+ The following functions are mandatory:
+ </p>
+ <taglist>
+ <tag><marker id="listen"/><c>listen(Name) -></c><br/>&nbsp;&nbsp;<c>{ok, {Listen, Address, Creation}} | {error, Error} </c></tag>
+ <item>
+ <p>
+ <c>listen/1</c> is called once in order to listen for incoming
+ connection requests. The call is made when the distribution is brought
+ up. The argument <c>Name</c> is the part of the node name before
+ the <c>@</c> sign in the full node name. It can be either an atom or a
+ string.
+ </p>
+ <p>
+ The return value consists of a <c>Listen</c> handle (which is
+ later passed to the <seealso marker="#accept"><c>accept/1</c></seealso>
+ callback), <c>Address</c> which is a <c>#net_address{}</c> record
+ with information about the address for the node (the
+ <c>#net_address{}</c> record is defined in
+ <c>kernel/include/net_address.hrl</c>), and <c>Creation</c> which
+ (currently) is an integer <c>1</c>, <c>2</c>, or <c>3</c>.
+ </p>
+ <p>
+ If <seealso marker="erts:epmd"><c>epmd</c></seealso> is to be used
+ for node discovery, you typically want to use the (unfortunately
+ undocumented) <c>erl_epmd</c> module (part of the <c>kernel</c>
+ application) in order to register the listen port with <c>epmd</c>
+ and retrieve <c>Creation</c> to use.
+ </p>
+ </item>
+
+ <tag><marker id="accept"/><c>accept(Listen) -></c><br/>&nbsp;&nbsp;<c>AcceptorPid</c></tag>
+ <item>
+ <p>
+ <c>accept/1</c> should spawn a process that accepts connections. This
+ process should preferably execute on <c>max</c> priority. The process
+ identifier of this process should be returned.
+ </p>
+ <p>
+ The <c>Listen</c> argument will be the same as the <c>Listen</c> handle
+ part of the return value of the
+ <seealso marker="#listen"><c>listen/1</c></seealso> callback above.
+ <c>accept/1</c> is called only once when the distribution protocol is
+ started.
+ </p>
+ <p>
+ The caller of this function is a representative for <c>net_kernel</c>
+ (this may or may not be the process registered as <c>net_kernel</c>)
+ and is in this document identified as <c>Kernel</c>.
+ When a connection has been accepted by the acceptor process, it needs
+ to inform <c>Kernel</c> about the accepted connection. This is done by
+ passing a message on the form:
+ </p>
+ <code type="none"><![CDATA[Kernel ! {accept, AcceptorPid, DistController, Family, Proto}]]></code>
+ <p>
+ <c>DistController</c> is either the process or port identifier
+ of the distribution controller for the connection. The
+ distribution controller should be created by the acceptor
+ processes when a new connection is accepted. Its job is to
+ dispatch traffic on the connection.
+ </p>
+ <c>Kernel</c> responds with one of the following messages:
+ <taglist>
+ <tag><c>{Kernel, controller, SupervisorPid}</c></tag>
+ <item>
+ <p>
+ The request was accepted and <c>SupervisorPid</c> is the
+ process identifier of the connection supervisor process
+ (which is created in the
+ <seealso marker="#accept_connection"><c>accept_connection/5</c></seealso>
+ callback).
+ </p>
+ </item>
+ <tag><c>{Kernel, unsupported_protocol}</c></tag>
+ <item>
+ <p>
+ The request was rejected. This is a fatal error. The acceptor
+ process should terminate.
+ </p>
+ </item>
+ </taglist>
+ <p>
+ When an accept sequence has been completed the acceptor process
+ is expected to continue accepting further requests.
+ </p>
+ </item>
+
+ <tag><marker id="accept_connection"/><c>accept_connection(AcceptorPid, DistCtrl, MyNode, Allowed, SetupTime) -></c><br/>&nbsp;&nbsp;<c>ConnectionSupervisorPid</c></tag>
+ <item>
+ <p>
+ <c>accept_connection/5</c> should spawn a process that will
+ perform the Erlang distribution handshake for the connection.
+ If the handshake successfully completes it should continue to
+ function as a connection supervisor. This process
+ should preferably execute on <c>max</c> priority.
+ </p>
+ <p>The arguments:</p>
+ <taglist>
+ <tag><c>AcceptorPid</c></tag>
+ <item>
+ <p>
+ Process identifier of the process created by the
+ <seealso marker="#accept"><c>accept/1</c></seealso>
+ callback.
+ </p>
+ </item>
+ <tag><c>DistCtrl</c></tag>
+ <item>
+ <p>The identifier of the distribution controller identifier
+ created by the acceptor process. To be passed along to
+ <c>dist_util:handshake_other_started(HsData)</c>.
+ </p>
+ </item>
+ <tag><c>MyNode</c></tag>
+ <item>
+ <p>
+ Node name of this node. To be passed along to
+ <c>dist_util:handshake_other_started(HsData)</c>.
+ </p>
+ </item>
+ <tag><c>Allowed</c></tag>
+ <item>
+ <p>
+ To be passed along to
+ <c>dist_util:handshake_other_started(HsData)</c>.
+ </p>
+ </item>
+ <tag><c>SetupTime</c></tag>
+ <item>
+ <p>
+ Time used for creating a setup timer by a
+ call to <c>dist_util:start_timer(SetupTime)</c>.
+ The timer should be passed along to
+ <c>dist_util:handshake_other_started(HsData)</c>.
+ </p>
+ </item>
+ </taglist>
+ <p>
+ The created process should provide callbacks and other
+ information needed for the handshake in a
+ <seealso marker="#hs_data_record"><c>#hs_data{}</c></seealso>
+ record and call <c>dist_util:handshake_other_started(HsData)</c>
+ with this record.
+ </p>
+ <p>
+ <c>dist_util:handshake_other_started(HsData)</c> will perform
+ the handshake and if the handshake successfully completes this
+ process will then continue in a connection supervisor loop
+ as long as the connection is up.
+ </p>
+ </item>
+
+ <tag><marker id="setup"/><c>setup(Node, Type, MyNode, LongOrShortNames, SetupTime) -></c><br/>&nbsp;&nbsp;<c>ConnectionSupervisorPid</c></tag>
+ <item>
+ <p>
+ <c>setup/5</c> should spawn a process that connects to
+ <c>Node</c>. When connection has been established it should
+ perform the Erlang distribution handshake for the connection.
+ If the handshake successfully completes it should continue to
+ function as a connection supervisor. This process
+ should preferably execute on <c>max</c> priority.
+ </p>
+ <p>The arguments:</p>
+ <taglist>
+ <tag><c>Node</c></tag>
+ <item>
+ <p>
+ Node name of remote node. To be passed along to
+ <c>dist_util:handshake_we_started(HsData)</c>.
+ </p>
+ </item>
+ <tag><c>Type</c></tag>
+ <item>
+ <p>
+ Connection type. To be passed along to
+ <c>dist_util:handshake_we_started(HsData)</c>.
+ </p>
+ </item>
+ <tag><c>MyNode</c></tag>
+ <item>
+ <p>
+ Node name of this node. To be passed along to
+ <c>dist_util:handshake_we_started(HsData)</c>.
+ </p>
+ </item>
+ <tag><c>LongOrShortNames</c></tag>
+ <item>
+ <p>
+ Either the atom <c>longnames</c> or
+ the atom <c>shortnames</c> indicating
+ whether long or short names is used.
+ </p>
+ </item>
+ <tag><c>SetupTime</c></tag>
+ <item>
+ <p>
+ Time used for creating a setup timer by a
+ call to <c>dist_util:start_timer(SetupTime)</c>.
+ The timer should be passed along to
+ <c>dist_util:handshake_we_started(HsData)</c>.
+ </p>
+ </item>
+ </taglist>
+ <p>
+ The caller of this function is a representative for <c>net_kernel</c>
+ (this may or may not be the process registered as <c>net_kernel</c>)
+ and is in this document identified as <c>Kernel</c>.
+ </p>
+ <p>
+ This function should, besides spawning the connection supervisor,
+ also create a distribution controller. The distribution
+ controller is either a process or a port which is responsible
+ for dispatching traffic.
+ </p>
+ <p>
+ The created process should provide callbacks and other
+ information needed for the handshake in a
+ <seealso marker="#hs_data_record"><c>#hs_data{}</c></seealso>
+ record and call <c>dist_util:handshake_we_started(HsData)</c>
+ with this record.
+ </p>
+ <p>
+ <c>dist_util:handshake_we_started(HsData)</c> will perform
+ the handshake and the handshake successfully completes this
+ process will then continue in a connection supervisor loop
+ as long as the connection is up.
+ </p>
+ </item>
+
+ <tag><marker id="close"/><c>close(Listen) -></c><br/>&nbsp;&nbsp;<c>void()</c></tag>
+
+ <item><p>
+ Called in order to close the <c>Listen</c> handle
+ that originally was passed from the
+ <seealso marker="#listen"><c>listen/1</c></seealso> callback.
+ </p></item>
+
+ <tag><marker id="select"/><c>select(NodeName) -></c><br/>&nbsp;&nbsp;<c>boolean()</c></tag>
+ <item>
+ <p>Return <c>true</c> if the host name part
+ of the <c>NodeName</c> is valid for use
+ with this protocol; otherwise, <c>false</c>.
+ </p>
+ </item>
+
+ </taglist>
+
+ <p>
+ There are also two optional functions that may be
+ exported:
+ </p>
+ <taglist>
+ <tag><marker id="select"/><c>setopts(Listen, Opts) -></c><br/>&nbsp;&nbsp;<c>ok | {error, Error}</c></tag>
+ <item>
+ <p>
+ The argument <c>Listen</c> is the handle originally passed
+ from the
+ <seealso marker="#listen"><c>listen/1</c></seealso> callback.
+ The argument <c>Opts</c> is a list of options to set on future
+ connections.
+ </p>
+ </item>
+
+ <tag><marker id="select"/><c>getopts(Listen, Opts) -></c><br/>&nbsp;&nbsp;<c>{ok, OptionValues} | {error, Error}</c></tag>
+ <item>
+ <p>
+ The argument <c>Listen</c> is the handle originally passed
+ from the
+ <seealso marker="#listen"><c>listen/1</c></seealso> callback.
+ The argument <c>Opts</c> is a list of options to read for future
+ connections.
+ </p>
+ </item>
+ </taglist>
+
+ </section>
+ <section>
+ <marker id="hs_data_record"/>
+ <title>The #hs_data{} Record</title>
+ <p>
+ The <c>dist_util:handshake_we_started/1</c> and
+ <c>dist_util:handshake_other_started/1</c> functions
+ takes a <c>#hs_data{}</c> record as argument. There
+ are quite a lot of fields in this record that you
+ need to set. The record is defined in
+ <c>kernel/include/dist_util.hrl</c>. Not documented
+ fields should not be set, i.e., should be left as
+ <c>undefined</c>.
+ </p>
+ <p>
+ The following <c>#hs_data{}</c> record fields need
+ to be set unless otherwise stated:</p>
+ <taglist>
+ <tag><marker id="hs_data_kernel_pid"/><c>kernel_pid</c></tag>
+ <item>
+ <p>
+ Process identifier of the <c>Kernel</c> process. That is,
+ the process that called either
+ <seealso marker="#setup"><c>setup/5</c></seealso> or
+ <seealso marker="#accept_connection"><c>accept_connection/5</c></seealso>.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_other_node"/><c>other_node</c></tag>
+ <item>
+ <p>Name of the other node. This field is only
+ mandatory when this node initiates the connection.
+ That is, when connection is set up via
+ <seealso marker="#setup"><c>setup/5</c></seealso>.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_this_node"/><c>this_node</c></tag>
+ <item>
+ <p>
+ The node name of this node.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_socket"/><c>socket</c></tag>
+ <item>
+ <p>
+ The identifier of the distribution controller.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_timer"/><c>timer</c></tag>
+ <item>
+ <p>
+ The timer created using <c>dist_util:start_timer/1</c>.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_allowed"/><c>allowed</c></tag>
+ <item>
+ <p>Information passed as <c>Allowed</c> to
+ <c>accept_connection/5</c>. This field is only
+ mandatory when the remote node initiated the
+ connection. That is, when the connection is set
+ up via
+ <seealso marker="#accept_connection"><c>accept_connection/5</c></seealso>.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_f_send"/><c>f_send</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr, Data) -> ok | {error, Error}]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier of
+ the distribution controller and <c>Data</c>
+ is io data to pass to the other side.
+ </p>
+ <p>Only used during handshake phase.</p>
+ </item>
+
+ <tag><marker id="hs_data_f_recv"/><c>f_recv</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr, Length) -> {ok, Packet} | {error, Reason}]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier of the distribution
+ controller.
+ If <c>Length</c> is <c>0</c>, all available bytes should be
+ returned. If <c>Length > 0</c>, exactly <c>Length</c> bytes
+ should be returned, or an error; possibly discarding less
+ than <c>Length</c> bytes of data when the connection is
+ closed from the other side.
+ It is used for passive receive of data from the
+ other end.
+ </p>
+ <p>Only used during handshake phase.</p>
+ </item>
+
+ <tag><marker id="hs_data_f_setopts_pre_nodeup"/><c>f_setopts_pre_nodeup</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr) -> ok | {error, Error}]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier of
+ the distribution controller. Called just
+ before the distribution channel is taken up
+ for normal traffic.
+ </p>
+ <p>Only used during handshake phase.</p>
+ </item>
+
+ <tag><marker id="hs_data_f_setopts_post_nodeup"/><c>f_setopts_post_nodeup</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr) -> ok | {error, Error}]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier of
+ the distribution controller. Called just
+ after distribution channel has been taken
+ up for normal traffic.
+ </p>
+ <p>Only used during handshake phase.</p>
+ </item>
+
+ <tag><marker id="hs_data_f_getll"/><c>f_getll</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr) -> ID]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier of
+ the distribution controller and <c>ID</c> is
+ the identifier of the low level entity that
+ handles the connection (often <c>DistCtrlr</c>
+ itself).
+ </p>
+ <p>Only used during handshake phase.</p>
+ </item>
+
+ <tag><marker id="hs_data_f_address"/><c>f_address</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr, Node) -> NetAddress]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier of
+ the distribution controller, <c>Node</c>
+ is the node name of the node on the other end,
+ and <c>NetAddress</c> is a <c>#net_address{}</c>
+ record with information about the address
+ for the <c>Node</c> on the other end of the
+ connection. The <c>#net_address{}</c> record
+ is defined in
+ <c>kernel/include/net_address.hrl</c>.
+ </p>
+ <p>Only used during handshake phase.</p>
+ </item>
+
+ <tag><marker id="hs_data_mf_tick"/><c>mf_tick</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr) -> void()]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier
+ of the distribution controller. This
+ function should send information over
+ the connection that is not interpreted
+ by the other end while increasing the
+ statistics of received packets on the
+ other end. This is usually implemented by
+ sending an empty packet.
+ </p>
+ <note><p>
+ It is of vital importance that this operation
+ does not block the caller for a long time.
+ This since it is called from the connection
+ supervisor.
+ </p></note>
+ <p>Used when connection is up.</p>
+ </item>
+
+ <tag><marker id="hs_data_mf_getstat"/><c>mf_getstat</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr) -> {ok, Received, Sent, PendSend}]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier
+ of the distribution controller, <c>Received</c>
+ is received packets, <c>Sent</c> is
+ sent packets, and <c>PendSend</c> is
+ amount of packets in queue to be sent
+ or a <c>boolean()</c> indicating whether
+ there are packets in queue to be sent.
+ </p>
+ <note><p>
+ It is of vital importance that this operation
+ does not block the caller for a long time.
+ This since it is called from the connection
+ supervisor.
+ </p></note>
+ <p>Used when connection is up.</p>
+ </item>
+
+ <tag><marker id="hs_data_request_type"/><c>request_type</c></tag>
+ <item>
+ <p>
+ The request <c>Type</c> as passed to
+ <seealso marker="#setup"><c>setup/5</c></seealso>.
+ This is only mandatory when the connection has
+ been initiated by this node. That is, the connection
+ is set up via <c>setup/5</c>.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_mf_setopts"/><c>mf_setopts</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrl, Opts) -> ok | {error, Error}]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier
+ of the distribution controller and <c>Opts</c>
+ is a list of options to set on the connection.
+ </p>
+ <p>This function is optional. Used when connection is up.</p>
+ </item>
+
+ <tag><marker id="hs_data_mf_getopts"/><c>mf_getopts</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrl, Opts) -> {ok, OptionValues} | {error, Error}]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier
+ of the distribution controller and <c>Opts</c>
+ is a list of options to read for the connection.
+ </p>
+ <p>This function is optional. Used when connection is up.</p>
+ </item>
+
+ <tag><marker id="hs_data_f_handshake_complete"/><c>f_handshake_complete</c></tag>
+ <item>
+ <p>
+ A fun with the following signature:
+ </p>
+ <code type="none"><![CDATA[fun (DistCtrlr, Node, DHandle) -> void()]]></code>
+ <p>
+ where <c>DistCtrlr</c> is the identifier
+ of the distribution controller, <c>Node</c> is
+ the node name of the node connected at the other
+ end, and <c>DHandle</c> is a distribution handle
+ needed by a distribution controller process when
+ calling the following BIFs:
+ </p>
+ <list>
+ <item><p><seealso marker="erts:erlang#dist_ctrl_get_data/1"><c>erlang:dist_ctrl_get_data/1</c></seealso></p></item>
+ <item><p><seealso marker="erts:erlang#dist_ctrl_get_data_notification/1"><c>erlang:dist_ctrl_get_data_notification/1</c></seealso></p></item>
+ <item><p><seealso marker="erts:erlang#dist_ctrl_input_handler/2"><c>erlang:dist_ctrl_input_handler/2</c></seealso></p></item>
+ <item><p><seealso marker="erts:erlang#dist_ctrl_put_data/2"><c>erlang:dist_ctrl_put_data/2</c></seealso></p></item>
+ </list>
+ <p>
+ This function is called when the handshake has
+ completed and the distribution channel is up.
+ The distribution controller can begin dispatching
+ traffic over the channel. This function is optional.
+ </p>
+ <p>Only used during handshake phase.</p>
+ </item>
+
+ <tag><marker id="hs_data_add_flags"/><c>add_flags</c></tag>
+ <item>
+ <p>
+ <seealso marker="erl_dist_protocol#dflags">Distribution flags</seealso>
+ to add to the connection. Currently all (non obsolete) flags will
+ automatically be enabled.
+ </p>
+ <p>
+ This flag field is optional.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_reject_flags"/><c>reject_flags</c></tag>
+ <item>
+ <p>
+ <seealso marker="erl_dist_protocol#dflags">Distribution flags</seealso>
+ to reject. Currently the following distribution flags can be rejected:
+ </p>
+ <taglist>
+ <tag><c>DFLAG_DIST_HDR_ATOM_CACHE</c></tag>
+ <item>Do not use atom cache over this connection.</item>
+ <tag><c>DFLAGS_STRICT_ORDER_DELIVERY</c></tag>
+ <item>Do not use any features that require strict
+ order delivery.</item>
+ </taglist>
+ <p>
+ This flag field is optional.
+ </p>
+ </item>
+
+ <tag><marker id="hs_data_require_flags"/><c>require_flags</c></tag>
+ <item>
+ <p>
+ Require these <seealso marker="erl_dist_protocol#dflags">distribution
+ flags</seealso> to be used. The connection will be aborted during the
+ handshake if the other end does not use them.
+ </p>
+ <p>
+ This flag field is optional.
+ </p>
+ </item>
+
+ </taglist>
+ </section>
+
+ <section>
+ <marker id="distribution_data_delivery"/>
+ <title>Distribution Data Delivery</title>
+ <p>
+ When using the default configuration, the data to pass
+ over a connection needs to be delivered as is
+ to the node on the receiving end in the <em>exact same
+ order</em>, with no loss of data what so ever, as sent
+ from the sending node.
+ </p>
+ <p>
+ The data delivery order can be relaxed by disabling
+ features that require strict ordering. This is done by
+ passing the <c>?DFLAGS_STRICT_ORDER_DELIVERY</c>
+ <seealso marker="erl_dist_protocol#dflags">distribution
+ flags</seealso> in the
+ <seealso marker="alt_dist#hs_data_reject_flags"><c>reject_flags</c></seealso>
+ field of the <seealso marker="#hs_data_record"><c>#hs_data{}</c></seealso>
+ record used when setting up the connection. When relaxed
+ ordering is used, only the order of signals with the same
+ sender/receiver pair has to be preserved.
+ However, note that disabling the features that require
+ strict ordering may have a negative impact on performance,
+ throughput, and/or latency.
+ </p>
+ </section>
+
+ <section>
+ <marker id="enable_your_distribution_module"/>
+ <title>Enable Your Distribution Module</title>
+
+ <p>For <c>net_kernel</c> to find out which distribution module to use,
+ the <c>erl</c> command-line argument <c>-proto_dist</c> is used. It
+ is followed by one or more distribution module names, with suffix
+ "_dist" removed. That is, <c>gen_tcp_dist</c> as a distribution module
+ is specified as <c>-proto_dist gen_tcp</c>.</p>
+
+ <p>If no <c>epmd</c> (TCP port mapper daemon) is used, also command-line
+ option <c>-no_epmd</c> is to be specified, which makes
+ Erlang skip the <c>epmd</c> startup, both as an OS process and as an
+ Erlang ditto.</p>
+ </section>
+
+ </section>
+
+ <section>
<title>The Driver</title>
+
+ <note>
+ <p>This section was written a long time ago. Most of it is still
+ valid, but some things have changed since then. Some updates have
+ been made to the documentation of the driver presented here,
+ but more can be done and is planned for the future.
+ The reader is encouraged to read the
+ <seealso marker="erl_driver"><c>erl_driver</c></seealso> and
+ <seealso marker="driver_entry"><c>driver_entry</c></seealso>
+ documentation also.</p>
+ </note>
+
<p>Although Erlang drivers in general can be beyond the scope of this
section, a brief introduction seems to be in place.</p>
diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml
index 638e88ca31..71fe08d4e6 100644
--- a/erts/doc/src/erl.xml
+++ b/erts/doc/src/erl.xml
@@ -538,20 +538,6 @@
<p>Note that a distributed node will fail to start if epmd is
not running.</p>
</item>
- <tag><marker id="smp"/><c><![CDATA[-smp [enable|auto|disable]]]></c></tag>
- <item>
- <p><c>-smp enable</c> and <c>-smp</c> start the Erlang runtime
- system with SMP support enabled. This can fail if no runtime
- system with SMP support is available. <c>-smp auto</c> starts
- the Erlang runtime system with SMP support enabled if it is
- available and more than one logical processor is detected.
- <c>-smp disable</c> starts a runtime system without SMP support.
- The runtime system without SMP support is deprecated and will
- be removed in a future major release.</p>
- <note>
- <p>See also flag<seealso marker="#+S"><c>+S</c></seealso>.</p>
- </note>
- </item>
<tag><c><![CDATA[-version]]></c> (emulator flag)</tag>
<item>
<p>Makes the emulator print its version number. The same
@@ -902,7 +888,7 @@
<c><![CDATA[+S Schedulers:SchedulerOnline]]></c></tag>
<item>
<p>Sets the number of scheduler threads to create and scheduler threads
- to set online when SMP support has been enabled. The maximum for both
+ to set online. The maximum for both
values is 1024. If the Erlang runtime system is able to determine the
number of logical processors configured and logical processors
available, <c>Schedulers</c> defaults to logical processors
@@ -920,8 +906,6 @@
<p>Specifying value <c>0</c> for <c>Schedulers</c> or
<c>SchedulersOnline</c> resets the number of scheduler threads or
scheduler threads online, respectively, to its default value.</p>
- <p>This option is ignored if the emulator does not have SMP support
- enabled (see flag <seealso marker="#smp"><c>-smp</c></seealso>).</p>
</item>
<tag><marker id="+SP"/><c><![CDATA[+SP
SchedulersPercentage:SchedulersOnlinePercentage]]></c></tag>
@@ -929,8 +913,8 @@
<p>Similar to <seealso marker="#+S"><c>+S</c></seealso> but uses
percentages to set the number of scheduler threads to create, based
on logical processors configured, and scheduler threads to set online,
- based on logical processors available, when SMP support has been
- enabled. Specified values must be &gt; 0. For example,
+ based on logical processors available.
+ Specified values must be &gt; 0. For example,
<c>+SP 50:25</c> sets the number of scheduler threads to 50% of the
logical processors configured, and the number of scheduler threads
online to 25% of the logical processors available.
@@ -945,15 +929,13 @@
and 8 logical cores available, the combination of the options
<c>+S 4:4 +SP 50:25</c> (in either order) results in 2 scheduler
threads (50% of 4) and 1 scheduler thread online (25% of 4).</p>
- <p>This option is ignored if the emulator does not have SMP support
- enabled (see flag <seealso marker="#smp"><c>-smp</c></seealso>).</p>
</item>
<tag><marker id="+SDcpu"/><c><![CDATA[+SDcpu
DirtyCPUSchedulers:DirtyCPUSchedulersOnline]]></c></tag>
<item>
<p>Sets the number of dirty CPU scheduler threads to create and dirty
- CPU scheduler threads to set online when threading support has been
- enabled. The maximum for both values is 1024, and each value is
+ CPU scheduler threads to set online.
+ The maximum for both values is 1024, and each value is
further limited by the settings for normal schedulers:</p>
<list type="bulleted">
<item>The number of dirty CPU scheduler threads created cannot exceed
@@ -977,16 +959,14 @@
executing on ordinary schedulers. If the amount of dirty CPU
schedulers was allowed to be unlimited, dirty CPU bound jobs would
potentially starve normal jobs.</p>
- <p>This option is ignored if the emulator does not have threading
- support enabled.</p>
</item>
<tag><marker id="+SDPcpu"/><c><![CDATA[+SDPcpu
DirtyCPUSchedulersPercentage:DirtyCPUSchedulersOnlinePercentage]]></c></tag>
<item>
<p>Similar to <seealso marker="#+SDcpu"><c>+SDcpu</c></seealso> but
uses percentages to set the number of dirty CPU scheduler threads to
- create and the number of dirty CPU scheduler threads to set online
- when threading support has been enabled. Specified values must be
+ create and the number of dirty CPU scheduler threads to set online.
+ Specified values must be
&gt; 0. For example, <c>+SDPcpu 50:25</c> sets the number of dirty
CPU scheduler threads to 50% of the logical processors configured
and the number of dirty CPU scheduler threads online to 25% of the
@@ -1003,13 +983,11 @@
the combination of the options <c>+SDcpu 4:4 +SDPcpu 50:25</c> (in
either order) results in 2 dirty CPU scheduler threads (50% of 4) and
1 dirty CPU scheduler thread online (25% of 4).</p>
- <p>This option is ignored if the emulator does not have threading
- support enabled.</p>
</item>
<tag><marker id="+SDio"/><c><![CDATA[+SDio DirtyIOSchedulers]]></c></tag>
<item>
- <p>Sets the number of dirty I/O scheduler threads to create when
- threading support has been enabled. Valid range is 0-1024. By
+ <p>Sets the number of dirty I/O scheduler threads to create.
+ Valid range is 0-1024. By
default, the number of dirty I/O scheduler threads created is 10,
same as the default number of threads in the <seealso
marker="#async_thread_pool_size">async thread pool</seealso>.</p>
@@ -1019,8 +997,6 @@
expected to execute on dirty I/O schedulers. If the user should schedule CPU
bound jobs on dirty I/O schedulers, these jobs might starve ordinary
jobs executing on ordinary schedulers.</p>
- <p>This option is ignored if the emulator does not have threading
- support enabled.</p>
</item>
<tag><c><![CDATA[+sFlag Value]]></c></tag>
<item>
diff --git a/erts/doc/src/erl_dist_protocol.xml b/erts/doc/src/erl_dist_protocol.xml
index 610351db6c..a78b13aaa4 100644
--- a/erts/doc/src/erl_dist_protocol.xml
+++ b/erts/doc/src/erl_dist_protocol.xml
@@ -829,7 +829,31 @@ DiB == gen_digest(ChA, ICA)?
<item>
<p>The node understand UTF-8 encoded atoms.</p>
</item>
+ <tag><c>-define(DFLAG_MAP_TAG, 16#20000).</c></tag>
+ <item>
+ <p>The node understand the map tag.</p>
+ </item>
+ <tag><c>-define(DFLAG_BIG_CREATION, 16#40000).</c></tag>
+ <item>
+ <p>The node understand big node creation.</p>
+ </item>
+ <tag><c>-define(DFLAG_SEND_SENDER, 16#80000).</c></tag>
+ <item>
+ <p>
+ Use the <c>SEND_SENDER</c>
+ <seealso marker="#control_message">control message</seealso>
+ instead of the <c>SEND</c> control message and use the
+ <c>SEND_SENDER_TT</c> control message instead
+ of the <c>SEND_TT</c> control message.
+ </p>
+ </item>
</taglist>
+ <p>
+ There are also a collection of <c>DFLAG</c>s bitwise or:ed
+ together in the <c>DFLAGS_STRICT_ORDER_DELIVERY</c> macro.
+ These flags corresponds to features that require strict
+ ordering of data over distribution channels.
+ </p>
</section>
</section>
@@ -922,6 +946,7 @@ DiB == gen_digest(ChA, ICA)?
</item>
</taglist>
+ <marker id="control_message"/>
<p>The <c>ControlMessage</c> is a tuple, where the first element indicates
which distributed operation it encodes:</p>
@@ -1028,4 +1053,49 @@ DiB == gen_digest(ChA, ICA)?
</item>
</taglist>
</section>
+
+ <section>
+ <title>New Ctrlmessages for Erlang/OTP 21</title>
+ <taglist>
+ <tag><c>SEND_SENDER</c></tag>
+ <item>
+ <p><c>{22, FromPid, ToPid}</c></p>
+ <p>Followed by <c>Message</c>.</p>
+ <p>
+ This control messages replace the <c>SEND</c> control
+ message and will be sent when the distribution flag
+ <seealso marker="erl_dist_protocol#dflags"><c>DFLAG_SEND_SENDER</c></seealso>
+ has been negotiated in the connection setup handshake.
+ </p>
+ <note><p>
+ Messages encoded before the connection has
+ been set up may still use the <c>SEND</c> control
+ message. However, once a <c>SEND_SENDER</c> or <c>SEND_SENDER_TT</c>
+ control message has been sent, no more <c>SEND</c>
+ control messages will be sent in the same direction
+ on the connection.
+ </p></note>
+ </item>
+ <tag><c>SEND_SENDER_TT</c></tag>
+ <item>
+ <p><c>{23, FromPid, ToPid, TraceToken}</c></p>
+ <p>Followed by <c>Message</c>.</p>
+ <p>
+ This control messages replace the <c>SEND_TT</c> control
+ message and will be sent when the distribution flag
+ <seealso marker="erl_dist_protocol#dflags"><c>DFLAG_SEND_SENDER</c></seealso>
+ has been negotiated in the connection setup handshake.
+ </p>
+ <note><p>
+ Messages encoded before the connection has
+ been set up may still use the <c>SEND_TT</c> control
+ message. However, once a <c>SEND_SENDER</c> or <c>SEND_SENDER_TT</c>
+ control message has been sent, no more <c>SEND_TT</c>
+ control messages will be sent in the same direction
+ on the connection.
+ </p></note>
+ </item>
+ </taglist>
+ </section>
+
</chapter>
diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml
index 5a69bed34c..419e41693e 100644
--- a/erts/doc/src/erl_nif.xml
+++ b/erts/doc/src/erl_nif.xml
@@ -206,7 +206,7 @@ ok
<seealso marker="#enif_make_resource">
<c>enif_make_resource</c></seealso>.
The term returned by <c>enif_make_resource</c> is opaque in nature.
- It can be stored and passed between processes on the same node, but
+ It can be stored and passed between processes, but
the only real end usage is to pass it back as an argument to a NIF.
The NIF can then call <seealso marker="#enif_get_resource">
<c>enif_get_resource</c></seealso> and get back a pointer to the
@@ -344,6 +344,81 @@ return term;</code>
<c>enif_convert_time_unit()</c></seealso></item>
</list>
</item>
+ <tag><marker id="enif_ioq"/>I/O Queues</tag>
+ <item>
+ <p>The Erlang nif library contains function for easily working
+ with I/O vectors as used by the unix system call <c>writev</c>.
+ The I/O Queue is not thread safe, so some other synchronization
+ mechanism has to be used.</p>
+ <list type="bulleted">
+ <item><seealso marker="#SysIOVec">
+ <c>SysIOVec</c></seealso></item>
+ <item><seealso marker="#ErlNifIOVec">
+ <c>ErlNifIOVec</c></seealso></item>
+ <item><seealso marker="#enif_ioq_create">
+ <c>enif_ioq_create()</c></seealso></item>
+ <item><seealso marker="#enif_ioq_destroy">
+ <c>enif_ioq_destroy()</c></seealso></item>
+ <item><seealso marker="#enif_ioq_enq_binary">
+ <c>enif_ioq_enq_binary()</c></seealso></item>
+ <item><seealso marker="#enif_ioq_enqv">
+ <c>enif_ioq_enqv()</c></seealso></item>
+ <item><seealso marker="#enif_ioq_deq">
+ <c>enif_ioq_deq()</c></seealso></item>
+ <item><seealso marker="#enif_ioq_peek">
+ <c>enif_ioq_peek()</c></seealso></item>
+ <item><seealso marker="#enif_inspect_iovec">
+ <c>enif_inspect_iovec()</c></seealso></item>
+ <item><seealso marker="#enif_free_iovec">
+ <c>enif_free_iovec()</c></seealso></item>
+ </list>
+ <p>Typical usage when writing to a file descriptor looks like this:</p>
+ <code type="none"><![CDATA[
+int writeiovec(ErlNifEnv *env, ERL_NIF_TERM term, ERL_NIF_TERM *tail,
+ ErlNifIOQueue *q, int fd) {
+
+ ErlNifIOVec vec, *iovec = &vec;
+ SysIOVec *sysiovec;
+ int saved_errno;
+ int iovcnt, n;
+
+ if (!enif_inspect_iovec(env, 64, term, tail, &iovec))
+ return -2;
+
+ if (enif_ioq_size(q) > 0) {
+ /* If the I/O queue contains data we enqueue the iovec and
+ then peek the data to write out of the queue. */
+ if (!enif_ioq_enqv(q, iovec, 0))
+ return -3;
+
+ sysiovec = enif_ioq_peek(q, &iovcnt);
+ } else {
+ /* If the I/O queue is empty we skip the trip through it. */
+ iovcnt = iovec->iovcnt;
+ sysiovec = iovec->iov;
+ }
+
+ /* Attempt to write the data */
+ n = writev(fd, sysiovec, iovcnt);
+ saved_errno = errno;
+
+ if (enif_ioq_size(q) == 0) {
+ /* If the I/O queue was initially empty we enqueue any
+ remaining data into the queue for writing later. */
+ if (n >= 0 && !enif_ioq_enqv(q, iovec, n))
+ return -3;
+ } else {
+ /* Dequeue any data that was written from the queue. */
+ if (n > 0 && !enif_ioq_deq(q, n, NULL))
+ return -4;
+ }
+
+ /* return n, which is either number of bytes written or -1 if
+ some error happened */
+ errno = saved_errno;
+ return n;
+}]]></code>
+ </item>
<tag><marker id="lengthy_work"/>Long-running NIFs</tag>
<item>
<p>As mentioned in the <seealso marker="#WARNING">warning</seealso> text
@@ -837,6 +912,36 @@ typedef enum {
</item>
</taglist>
</item>
+ <tag><marker id="SysIOVec"/><c>SysIOVec</c></tag>
+ <item>
+ <p>A system I/O vector, as used by <c>writev</c> on
+ Unix and <c>WSASend</c> on Win32. It is used in
+ <c>ErlNifIOVec</c> and by
+ <seealso marker="#enif_ioq_peek"><c>enif_ioq_peek</c></seealso>.</p>
+ </item>
+ <tag><marker id="ErlNifIOVec"/><c>ErlNifIOVec</c></tag>
+ <item>
+ <code type="none">
+typedef struct {
+ int iovcnt;
+ size_t size;
+ SysIOVec* iov;
+} ErlNifIOVec;</code>
+ <p>An I/O vector containing <c>iovcnt</c> <c>SysIOVec</c>s
+ pointing to the data. It is used by
+ <seealso marker="#enif_inspect_iovec">
+ <c>enif_inspect_iovec</c></seealso> and
+ <seealso marker="#enif_ioq_enqv">
+ <c>enif_ioq_enqv</c></seealso>.</p>
+ </item>
+ <tag><marker id="ErlNifIOQueueOpts"/><c>ErlNifIOQueueOpts</c></tag>
+ <item>
+ Options to configure a <c>ErlNifIOQueue</c>.
+ <taglist>
+ <tag>ERL_NIF_IOQ_NORMAL</tag>
+ <item><p>Create a normal I/O Queue</p></item>
+ </taglist>
+ </item>
</taglist>
</section>
@@ -1143,6 +1248,31 @@ typedef enum {
</func>
<func>
+ <name><ret>void</ret>
+ <nametext>enif_free_iovec(ErlNifIOvec* iov)</nametext></name>
+ <fsummary>Free an ErlIOVec</fsummary>
+ <desc>
+ <p>Frees an io vector returned from
+ <seealso marker="#enif_inspect_iovec">
+ <c>enif_inspect_iovec</c></seealso>.
+ This is needed only if a <c>NULL</c> environment is passed to
+ <seealso marker="#enif_inspect_iovec">
+ <c>enif_inspect_iovec</c></seealso>.</p>
+ <code type="none"><![CDATA[
+ErlNifIOVec *iovec = NULL;
+size_t max_elements = 128;
+ERL_NIF_TERM tail;
+if (!enif_inspect_iovec(NULL, max_elements, term, &tail, iovec))
+ return 0;
+
+// Do things with the iovec
+
+/* Free the iovector, possibly in another thread or nif function call */
+enif_free_iovec(iovec);]]></code>
+ </desc>
+ </func>
+
+ <func>
<name><ret>int</ret><nametext>enif_get_atom(ErlNifEnv* env, ERL_NIF_TERM
term, char* buf, unsigned size, ErlNifCharEncoding encode)</nametext>
</name>
@@ -1449,6 +1579,127 @@ typedef enum {
</func>
<func>
+ <name><ret>int</ret><nametext>enif_inspect_iovec(ErlNifEnv*
+ env, size_t max_elements, ERL_NIF_TERM iovec_term, ERL_NIF_TERM* tail,
+ ErlNifIOVec** iovec)</nametext></name>
+ <fsummary>Inspect a list of binaries as an ErlNifIOVec.</fsummary>
+ <desc>
+ <p>Fills <c>iovec</c> with the list of binaries provided in
+ <c>iovec_term</c>. The number of elements handled in the call is
+ limited to <c>max_elements</c>, and <c>tail</c> is set to the
+ remainder of the list. Note that the output may be longer than
+ <c>max_elements</c> on some platforms.
+ </p>
+ <p>To create a list of binaries from an arbitrary iolist, use
+ <seealso marker="erts:erlang#iolist_to_iovec/1">
+ <c>erlang:iolist_to_iovec/1</c></seealso>.</p>
+ <p>When calling this function, <c>iovec</c> should contain a pointer to
+ <c>NULL</c> or a ErlNifIOVec structure that should be used if
+ possible. e.g.
+ </p>
+ <code type="none">
+/* Don't use a pre-allocated structure */
+ErlNifIOVec *iovec = NULL;
+enif_inspect_iovec(env, max_elements, term, &amp;tail, &amp;iovec);
+
+/* Use a stack-allocated vector as an optimization for vectors with few elements */
+ErlNifIOVec vec, *iovec = &amp;vec;
+enif_inspect_iovec(env, max_elements, term, &amp;tail, &amp;iovec);
+</code>
+ <p>The contents of the <c>iovec</c> is valid until the called nif
+ function returns. If the <c>iovec</c> should be valid after the nif
+ call returns, it is possible to call this function with a
+ <c>NULL</c> environment. If no environment is given the <c>iovec</c>
+ owns the data in the vector and it has to be explicitly freed using
+ <seealso marker="#enif_free_iovec"><c>enif_free_iovec</c>
+ </seealso>.</p>
+ <p>Returns <c>true</c> on success, or <c>false</c> if <c>iovec_term</c>
+ not an iovec.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name><ret>ErlNifIOQueue *</ret>
+ <nametext>enif_ioq_create(ErlNifIOQueueOpts opts)</nametext></name>
+ <fsummary>Create a new IO Queue</fsummary>
+ <desc>
+ <p>Create a new I/O Queue that can be used to store data.
+ <c>opts</c> has to be set to <c>ERL_NIF_IOQ_NORMAL</c>.
+ </p>
+ </desc>
+ </func>
+
+ <func>
+ <name><ret>void</ret>
+ <nametext>enif_ioq_destroy(ErlNifIOQueue *q)</nametext></name>
+ <fsummary>Destroy an IO Queue and free it's content</fsummary>
+ <desc>
+ <p>Destroy the I/O queue and free all of it's contents</p>
+ </desc>
+ </func>
+
+ <func>
+ <name><ret>int</ret>
+ <nametext>enif_ioq_deq(ErlNifIOQueue *q, size_t count, size_t *size)</nametext></name>
+ <fsummary>Dequeue count bytes from the IO Queue</fsummary>
+ <desc>
+ <p>Dequeue <c>count</c> bytes from the I/O queue.
+ If <c>size</c> is not <c>NULL</c>, the new size of the queue
+ is placed there.</p>
+ <p>Returns <c>true</c> on success, or <c>false</c> if the I/O does
+ not contain <c>count</c> bytes. On failure the queue is left un-altered.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name><ret>int</ret>
+ <nametext>enif_ioq_enq_binary(ErlNifIOQueue *q, ErlNifBinary *bin, size_t skip)</nametext></name>
+ <fsummary>Enqueue the binary into the IO Queue</fsummary>
+ <desc>
+ <p>Enqueue the <c>bin</c> into <c>q</c> skipping the first <c>skip</c> bytes.</p>
+ <p>Returns <c>true</c> on success, or <c>false</c> if <c>skip</c> is greater
+ than the size of <c>bin</c>. Any ownership of the binary data is transferred
+ to the queue and <c>bin</c> is to be considered read-only for the rest of the NIF
+ call and then as released.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name><ret>int</ret>
+ <nametext>enif_ioq_enqv(ErlNifIOQueue *q, ErlNifIOVec *iovec, size_t skip)</nametext></name>
+ <fsummary>Enqueue the iovec into the IO Queue</fsummary>
+ <desc>
+ <p>Enqueue the <c>iovec</c> into <c>q</c> skipping the first <c>skip</c> bytes.</p>
+ <p>Returns <c>true</c> on success, or <c>false</c> if <c>skip</c> is greater
+ than the size of <c>iovec</c>.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name><ret>SysIOVec *</ret>
+ <nametext>enif_ioq_peek(ErlNifIOQueue *q, int *iovlen)</nametext></name>
+ <fsummary>Peek inside the IO Queue</fsummary>
+ <desc>
+ <p>Get the I/O queue as a pointer to an array of <c>SysIOVec</c>s.
+ It also returns the number of elements in <c>iovlen</c>.
+ This is the only way to get data out of the queue.</p>
+ <p>Nothing is removed from the queue by this function, that must be done
+ with <seealso marker="#enif_ioq_deq"><c>enif_ioq_deq</c></seealso>.</p>
+ <p>The returned array is suitable to use with the Unix system
+ call <c>writev</c>.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name><ret>size_t</ret>
+ <nametext>enif_ioq_size(ErlNifIOQueue *q)</nametext></name>
+ <fsummary>Get the current size of the IO Queue</fsummary>
+ <desc>
+ <p>Get the size of <c>q</c>.</p>
+ </desc>
+ </func>
+
+ <func>
<name><ret>int</ret>
<nametext>enif_is_atom(ErlNifEnv* env, ERL_NIF_TERM term)</nametext>
</name>
@@ -1952,10 +2203,33 @@ typedef enum {
details, see the <seealso marker="#enif_resource_example">example of
creating and returning a resource object</seealso> in the User's
Guide.</p>
- <p>Notice that the only defined behavior of using a resource term in
- an Erlang program is to store it and send it between processes on the
- same node. Other operations, such as matching or
- <c>term_to_binary</c>, have unpredictable (but harmless) results.</p>
+ <note>
+ <p>Since ERTS 9.0 (OTP-20.0), resource terms have a defined behavior
+ when compared and serialized through <c>term_to_binary</c> or passed
+ between nodes.</p>
+ <list type="bulleted">
+ <item>
+ <p>Two resource terms will compare equal iff they
+ would yield the same resource object pointer when passed to
+ <seealso marker="#enif_get_resource"><c>enif_get_resource</c></seealso>.</p>
+ </item>
+ <item>
+ <p>A resoure term can be serialized with <c>term_to_binary</c> and later
+ be fully recreated if the resource object is still alive when
+ <c>binary_to_term</c> is called. A <em>stale</em> resource term will be
+ returned from <c>binary_to_term</c> if the resource object has
+ been deallocated. <seealso marker="#enif_get_resource"><c>enif_get_resource</c></seealso>
+ will return false for stale resource terms.</p>
+ <p>The same principles of serialization apply when passing
+ resource terms in messages to remote nodes and back again. A
+ resource term will act stale on all nodes except the node where
+ its resource object is still alive in memory.</p>
+ </item>
+ </list>
+ <p>Before ERTS 9.0 (OTP-20.0), all resource terms did
+ compare equal to each other and to empty binaries (<c>&lt;&lt;&gt;&gt;</c>).
+ If serialized, they would be recreated as plain empty binaries.</p>
+ </note>
</desc>
</func>
diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml
index 105734d5b2..ca8ba044e7 100644
--- a/erts/doc/src/erlang.xml
+++ b/erts/doc/src/erlang.xml
@@ -60,6 +60,14 @@
</desc>
</datatype>
<datatype>
+ <name>iovec()</name>
+ <desc>
+ <p>A list of binaries. This datatype is useful to use
+ together with <seealso marker="erl_nif#enif_inspect_iovec">
+ <c>enif_inspect_iovec</c></seealso>.</p>
+ </desc>
+ </datatype>
+ <datatype>
<name name="message_queue_data"></name>
<desc>
<p>See <seealso marker="#process_flag_message_queue_data">
@@ -181,6 +189,14 @@
</taglist>
</desc>
</datatype>
+
+ <datatype>
+ <name name="dist_handle"></name>
+ <desc>
+ <p>An opaque handle identifing a distribution channel.</p>
+ </desc>
+ </datatype>
+
</datatypes>
<funcs>
@@ -424,6 +440,16 @@ Z = erlang:adler32_combine(X,Y,iolist_size(Data2)).</code>
<seealso marker="#binary_to_atom/2"><c>binary_to_atom/2</c></seealso>,
but the atom must exist.</p>
<p>Failure: <c>badarg</c> if the atom does not exist.</p>
+ <note>
+ <p>Note that the compiler may optimize away atoms. For
+ example, the compiler will rewrite
+ <c>atom_to_list(some_atom)</c> to <c>"some_atom"</c>. If
+ that expression is the only mention of the atom
+ <c>some_atom</c> in the containing module, the atom will not
+ be created when the module is loaded, and a subsequent call
+ to <c>binary_to_existing_atom(&lt;&lt;"some_atom"&gt;&gt;, utf8)</c>
+ will fail.</p>
+ </note>
</desc>
</func>
@@ -1215,6 +1241,141 @@ end</code>
</func>
<func>
+ <name name="dist_ctrl_get_data" arity="1"/>
+ <fsummary>Get distribution channel data to pass to another node.</fsummary>
+ <desc>
+ <p>
+ Get distribution channel data from the local node that is
+ to be passed to the remote node. The distribution channel
+ is identified by <c><anno>DHandle</anno></c>. If no data
+ is available, the atom <c>none</c> is returned. One
+ can request to be informed by a message when more
+ data is available by calling
+ <seealso marker="erlang#dist_ctrl_get_data_notification/1"><c>erlang:dist_ctrl_get_data_notification(DHandle)</c></seealso>.
+ </p>
+ <note><p>
+ Only the process registered as distribution
+ controller for the distribution channel identified by
+ <c><anno>DHandle</anno></c> is allowed to call this
+ function.
+ </p></note>
+ <p>
+ This function is used when implementing an alternative
+ distribution carrier using processes as distribution
+ controllers. <c><anno>DHandle</anno></c> is retrived
+ via the callback
+ <seealso marker="erts:alt_dist#hs_data_f_handshake_complete"><c>f_handshake_complete</c></seealso>.
+ More information can be found in the documentation of
+ <seealso marker="erts:alt_dist#distribution_module">ERTS
+ User's Guide ➜ How to implement an Alternative Carrier
+ for the Erlang Distribution ➜ Distribution Module</seealso>.
+ </p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="dist_ctrl_get_data_notification" arity="1"/>
+ <fsummary>Request notification about available outgoing distribution channel data.</fsummary>
+ <desc>
+ <p>
+ Request notification when more data is available to
+ fetch using
+ <seealso marker="erlang#dist_ctrl_get_data/1"><c>erlang:dist_ctrl_get_data(DHandle)</c></seealso>
+ for the distribution channel identified by
+ <c><anno>DHandle</anno></c>. When more data is present,
+ the caller will be sent the message <c>dist_data</c>.
+ Once a <c>dist_data</c> messages has been sent, no
+ more <c>dist_data</c> messages will be sent until
+ the <c>dist_ctrl_get_data_notification/1</c> function has been called
+ again.
+ </p>
+ <note><p>
+ Only the process registered as distribution
+ controller for the distribution channel identified by
+ <c><anno>DHandle</anno></c> is allowed to call this
+ function.
+ </p></note>
+ <p>
+ This function is used when implementing an alternative
+ distribution carrier using processes as distribution
+ controllers. <c><anno>DHandle</anno></c> is retrived
+ via the callback
+ <seealso marker="erts:alt_dist#hs_data_f_handshake_complete"><c>f_handshake_complete</c></seealso>.
+ More information can be found in the documentation of
+ <seealso marker="erts:alt_dist#distribution_module">ERTS
+ User's Guide ➜ How to implement an Alternative Carrier
+ for the Erlang Distribution ➜ Distribution Module</seealso>.
+ </p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="dist_ctrl_input_handler" arity="2"/>
+ <fsummary>Register distribution channel input handler process.</fsummary>
+ <desc>
+ <p>
+ Register an alternate input handler process for the
+ distribution channel identified by <c><anno>DHandle</anno></c>.
+ Once this function has been called, <c><anno>InputHandler</anno></c>
+ is the only process allowed to call
+ <seealso marker="erlang#dist_ctrl_put_data/2"><c>erlang:dist_ctrl_put_data(DHandle, Data)</c></seealso>
+ with the <c><anno>DHandle</anno></c> identifing this distribution
+ channel.
+ </p>
+ <note><p>
+ Only the process registered as distribution
+ controller for the distribution channel identified by
+ <c><anno>DHandle</anno></c> is allowed to call this
+ function.
+ </p></note>
+ <p>
+ This function is used when implementing an alternative
+ distribution carrier using processes as distribution
+ controllers. <c><anno>DHandle</anno></c> is retrived
+ via the callback
+ <seealso marker="erts:alt_dist#hs_data_f_handshake_complete"><c>f_handshake_complete</c></seealso>.
+ More information can be found in the documentation of
+ <seealso marker="erts:alt_dist#distribution_module">ERTS
+ User's Guide ➜ How to implement an Alternative Carrier
+ for the Erlang Distribution ➜ Distribution Module</seealso>.
+ </p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="dist_ctrl_put_data" arity="2"/>
+ <fsummary>Pass data into the VM from a distribution channel.</fsummary>
+ <desc>
+ <p>
+ Deliver distribution channel data from a remote node to the
+ local node.
+ </p>
+ <note><p>
+ Only the process registered as distribution
+ controller for the distribution channel identified by
+ <c><anno>DHandle</anno></c> is allowed to call this
+ function unless an alternate input handler process
+ has been registered using
+ <seealso marker="erlang#dist_ctrl_input_handler/2"><c>erlang:dist_ctrl_input_handler(DHandle, InputHandler)</c></seealso>.
+ If an alternate input handler has been registered, only
+ the registered input handler process is allowed to call
+ this function.
+ </p></note>
+ <p>
+ This function is used when implementing an alternative
+ distribution carrier using processes as distribution
+ controllers. <c><anno>DHandle</anno></c> is retrived
+ via the callback
+ <seealso marker="erts:alt_dist#hs_data_f_handshake_complete"><c>f_handshake_complete</c></seealso>.
+ More information can be found in the documentation of
+ <seealso marker="erts:alt_dist#distribution_module">ERTS
+ User's Guide ➜ How to implement an Alternative Carrier
+ for the Erlang Distribution ➜ Distribution Module</seealso>.
+ </p>
+ </desc>
+ </func>
+
+ <func>
<name name="element" arity="2"/>
<fsummary>Return the Nth element of a tuple.</fsummary>
<type_desc variable="N">1..tuple_size(<anno>Tuple</anno>)</type_desc>
@@ -1943,23 +2104,26 @@ os_prompt%</pre>
<item>The runtime system exits with integer value
<c><anno>Status</anno></c>
as status code to the calling environment (OS).
+ <note>
+ <p>On many platforms, the OS supports only status
+ codes 0-255. A too large status code is truncated by clearing
+ the high bits.</p>
+ </note>
</item>
<tag>string()</tag>
<item>An Erlang crash dump is produced with <c><anno>Status</anno></c>
as slogan. Then the runtime system exits with status code <c>1</c>.
- Note that only code points in the range 0-255 may be used
- and the string will be truncated if longer than 200 characters.
+ The string will be truncated if longer than 200 characters.
+ <note>
+ <p>Before ERTS 9.1 (OTP-20.1) only code points in the range 0-255
+ was accepted in the string. Now any unicode string is valid.</p>
+ </note>
</item>
<tag><c>abort</c></tag>
<item>The runtime system aborts producing a core dump, if that is
enabled in the OS.
</item>
</taglist>
- <note>
- <p>On many platforms, the OS supports only status
- codes 0-255. A too large status code is truncated by clearing
- the high bits.</p>
- </note>
<p>For integer <c><anno>Status</anno></c>, the Erlang runtime system
closes all ports and allows async threads to finish their
operations before exiting. To exit without such flushing, use
@@ -2125,6 +2289,15 @@ os_prompt%</pre>
</func>
<func>
+ <name name="iolist_to_iovec" arity="1"/>
+ <fsummary>Converts an iolist to a iovec.</fsummary>
+ <desc>
+ <p>Returns an iovec that is made from the integers and binaries in
+ <c><anno>IoListOrBinary</anno></c>.</p>
+ </desc>
+ </func>
+
+ <func>
<name name="is_alive" arity="0"/>
<fsummary>Check whether the local node is alive.</fsummary>
<desc>
@@ -2458,6 +2631,15 @@ os_prompt%</pre>
but only if there already exists such atom.</p>
<p>Failure: <c>badarg</c> if there does not already exist an atom
whose text representation is <c><anno>String</anno></c>.</p>
+ <note>
+ <p>Note that the compiler may optimize away atoms. For
+ example, the compiler will rewrite
+ <c>atom_to_list(some_atom)</c> to <c>"some_atom"</c>. If
+ that expression is the only mention of the atom
+ <c>some_atom</c> in the containing module, the atom will not
+ be created when the module is loaded, and a subsequent call
+ to <c>list_to_existing_atom("some_atom")</c> will fail.</p>
+ </note>
</desc>
</func>
@@ -4284,7 +4466,6 @@ RealSystem = system + MissedSystem</code>
<desc>
<p><c><anno>Locking</anno></c> is one of the following:</p>
<list type="bulleted">
- <item><c>false</c> (emulator without SMP support)</item>
<item><c>port_level</c> (port-specific locking)</item>
<item><c>driver_level</c> (driver-specific locking)</item>
</list>
@@ -4688,8 +4869,8 @@ RealSystem = system + MissedSystem</code>
selected for execution. Notice however that this does
<em>not</em> mean that no processes on priority <c>low</c>
or <c>normal</c> can run when processes
- are running on priority <c>high</c>. On the runtime
- system with SMP support, more processes can be running
+ are running on priority <c>high</c>. When using multiple
+ schedulers, more processes can be running
in parallel than processes on priority <c>high</c>. That is,
a <c>low</c> and a <c>high</c> priority process can
execute at the same time.</p>
@@ -4704,10 +4885,8 @@ RealSystem = system + MissedSystem</code>
execution.</p>
<note>
<p>Do not depend on the scheduling
- to remain exactly as it is today. Scheduling, at least on
- the runtime system with SMP support, is likely to be
- changed in a future release to use available
- processor cores better.</p>
+ to remain exactly as it is today. Scheduling is likely to be
+ changed in a future release to use available processor cores better.</p>
</note>
<p>There is <em>no</em> automatic mechanism for
avoiding priority inversion, such as priority inheritance
@@ -6219,8 +6398,7 @@ true</pre>
<p><c>statistics(exact_reductions)</c> is
a more expensive operation than
<seealso marker="#statistics_reductions">
- statistics(reductions)</seealso>,
- especially on an Erlang machine with SMP support.</p>
+ statistics(reductions)</seealso>.</p>
</note>
</desc>
</func>
@@ -6604,8 +6782,8 @@ ok
than available logical processors, this value may
be greater than <c>1.0</c>.</p>
<p>As of ERTS version 9.0, the Erlang runtime system
- with SMP support will as default have more schedulers
- than logical processors. This due to the dirty schedulers.</p>
+ will as default have more schedulers than logical processors.
+ This due to the dirty schedulers.</p>
<note>
<p><c>scheduler_wall_time</c> is by default disabled. To
enable it, use
@@ -7975,9 +8153,7 @@ ok
<taglist>
<tag><c>disabled</c></tag>
<item>
- <p>The emulator has only one scheduler thread. The
- emulator does not have SMP support, or have been
- started with only one scheduler thread.</p>
+ <p>The emulator has been started with only one scheduler thread.</p>
</item>
<tag><c>blocked</c></tag>
<item>
@@ -8340,8 +8516,7 @@ ok
</item>
<tag><c>smp_support</c></tag>
<item>
- <p>Returns <c>true</c> if the emulator has been compiled
- with SMP support, otherwise <c>false</c> is returned.</p>
+ <p>Returns <c>true</c>.</p>
</item>
<tag><marker id="system_info_start_time"/><c>start_time</c></tag>
<item>
@@ -8364,8 +8539,7 @@ ok
</item>
<tag><c>threads</c></tag>
<item>
- <p>Returns <c>true</c> if the emulator has been compiled
- with thread support, otherwise <c>false</c> is returned.</p>
+ <p>Returns <c>true</c>.</p>
</item>
<tag><c>thread_pool_size</c></tag>
<item>
@@ -10484,9 +10658,9 @@ true</pre>
<c>receive after 1 -> ok end</c>, except that <c>yield()</c>
is faster.</p>
<warning>
- <p>There is seldom or never any need to use this BIF,
- especially in the SMP emulator, as other processes have a
- chance to run in another scheduler thread anyway.
+ <p>There is seldom or never any need to use this BIF
+ as other processes have a chance to run in another scheduler
+ thread anyway.
Using this BIF without a thorough grasp of how the scheduler
works can cause performance degradation.</p>
</warning>
diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml
index 722f7aaebd..10b963a4e8 100644
--- a/erts/doc/src/notes.xml
+++ b/erts/doc/src/notes.xml
@@ -31,6 +31,109 @@
</header>
<p>This document describes the changes made to the ERTS application.</p>
+<section><title>Erts 9.0.5</title>
+
+ <section><title>Fixed Bugs and Malfunctions</title>
+ <list>
+ <item>
+ <p>
+ Fixed bug in <c>binary_to_term</c> and
+ <c>binary_to_atom</c> that could cause VM crash.
+ Typically happens when the last character of an UTF8
+ string is in the range 128 to 255, but truncated to only
+ one byte. Bug exists in <c>binary_to_term</c> since ERTS
+ version 5.10.2 (OTP_R16B01) and <c>binary_to_atom</c>
+ since ERTS version 9.0 (OTP-20.0).</p>
+ <p>
+ Own Id: OTP-14590 Aux Id: ERL-474 </p>
+ </item>
+ </list>
+ </section>
+
+</section>
+
+<section><title>Erts 9.0.4</title>
+
+ <section><title>Fixed Bugs and Malfunctions</title>
+ <list>
+ <item>
+ <p>
+ A timer internal bit-field used for storing scheduler id
+ was too small. As a result, VM internal timer data
+ structures could become inconsistent when using 1024
+ schedulers on the system. Note that systems with less
+ than 1024 schedulers are not effected by this bug.</p>
+ <p>
+ This bug was introduced in ERTS version 7.0 (OTP 18.0).</p>
+ <p>
+ Own Id: OTP-14548 Aux Id: OTP-11997, ERL-468 </p>
+ </item>
+ <item>
+ <p>
+ Automatic cleanup of a BIF timer, when the owner process
+ terminated, could race with the timeout of the timer.
+ This could cause the VM internal data structures to
+ become inconsistent which very likely caused a VM crash.</p>
+ <p>
+ This bug was introduced in ERTS version 9.0 (OTP 20.0).</p>
+ <p>
+ Own Id: OTP-14554 Aux Id: OTP-14356, ERL-468 </p>
+ </item>
+ </list>
+ </section>
+
+</section>
+
+<section><title>Erts 9.0.3</title>
+
+ <section><title>Fixed Bugs and Malfunctions</title>
+ <list>
+ <item>
+ <p>Binary append operations did not check for overflow,
+ resulting in nonsensical results when huge binaries were
+ appended.</p>
+ <p>
+ Own Id: OTP-14524</p>
+ </item>
+ </list>
+ </section>
+
+</section>
+
+<section><title>Erts 9.0.2</title>
+
+ <section><title>Fixed Bugs and Malfunctions</title>
+ <list>
+ <item>
+ <p>
+ Added missing release notes for OTP-14491 ("performance
+ bug in pre-allocators") which was included in erts-9.0.1
+ (OTP-20.0.1).</p>
+ <p>
+ Own Id: OTP-14494</p>
+ </item>
+ <item>
+ <p>Fixed a bug that prevented TCP sockets from being
+ closed properly on send timeouts.</p>
+ <p>
+ Own Id: OTP-14509</p>
+ </item>
+ <item>
+ <p>
+ Fixed bug in operator <c>bxor</c> causing erroneuos
+ result when one operand is a big <em>negative</em>
+ integer with the lowest <c>N*W</c> bits as zero and the
+ other operand not larger than <c>N*W</c> bits. <c>N</c>
+ is an integer of 1 or larger and <c>W</c> is 32 or 64
+ depending on word size.</p>
+ <p>
+ Own Id: OTP-14514</p>
+ </item>
+ </list>
+ </section>
+
+</section>
+
<section><title>Erts 9.0.1</title>
<section><title>Fixed Bugs and Malfunctions</title>
@@ -63,6 +166,17 @@
<p>
Own Id: OTP-14484</p>
</item>
+ <item>
+ <p>
+ Fix performance bug in pre-allocators that could cause
+ them to permanently fall back on normal more expensive memory
+ allocation. Pre-allocators are used for quick allocation
+ of short lived meta data used by messages and other
+ scheduled tasks. Bug exists since OTP_R15B02.
+ [this release note was missing in erts-9.0.1]</p>
+ <p>
+ Own Id: OTP-14491</p>
+ </item>
</list>
</section>
@@ -510,7 +624,7 @@
marker="erts:erl"><c>erl</c></seealso> command.</p>
<p>
See <url
- href="http://pcre.org/original/changelog.txt"><c>http://pcre.org/original/changelog.txt</c></url>
+ href="http://pcre.org/original/changelog.txt">http://pcre.org/original/changelog.txt</url>
for information about changes made to PCRE between the
versions 8.33 and 8.40.</p>
<p>
@@ -3448,8 +3562,7 @@
<p>The previously introduced "eager check I/O" feature is
now enabled by default.</p>
<p>Eager check I/O can be disabled using the <c>erl</c>
- command line argument: <seealso
- marker="erl#+secio"><c>+secio false</c></seealso></p>
+ command line argument: <c>+secio false</c></p>
<p>Characteristics impact compared to previous
default:</p> <list> <item>Lower latency and smoother
management of externally triggered I/O operations.</item>
@@ -4230,8 +4343,7 @@
prioritized to the same extent as when eager check I/O is
disabled.</p>
<p>Eager check I/O can be enabled using the <c>erl</c>
- command line argument: <seealso
- marker="erl#+secio"><c>+secio true</c></seealso></p>
+ command line argument: <c>+secio true</c></p>
<p>Characteristics impact when enabled:</p> <list>
<item>Lower latency and smoother management of externally
triggered I/O operations.</item> <item>A slightly reduced
diff --git a/erts/doc/src/zlib.xml b/erts/doc/src/zlib.xml
index 1d272c4c18..f5cc1b1e64 100644
--- a/erts/doc/src/zlib.xml
+++ b/erts/doc/src/zlib.xml
@@ -65,13 +65,17 @@ list_to_binary([Compressed|Last])</pre>
<tag><c>badarg</c></tag>
<item>Bad argument.
</item>
+ <tag><c>not_initialized</c></tag>
+ <item>The stream hasn't been initialized, eg. if
+ <seealso marker="#inflateInit/1"><c>inflateInit/1</c></seealso> wasn't
+ called prior to a call to
+ <seealso marker="#inflate/2"><c>inflate/2</c></seealso>.
+ </item>
<tag><c>data_error</c></tag>
<item>The data contains errors.
</item>
<tag><c>stream_error</c></tag>
<item>Inconsistent stream state.</item>
- <tag><c>einval</c></tag>
- <item>Bad value or wrong function called.</item>
<tag><c>{need_dictionary,Adler32}</c></tag>
<item>See <seealso marker="#inflate/2"><c>inflate/2</c></seealso>.
</item>
@@ -90,6 +94,9 @@ list_to_binary([Compressed|Last])</pre>
<name name="zlevel"/>
</datatype>
<datatype>
+ <name name="zflush"/>
+ </datatype>
+ <datatype>
<name name="zmemlevel"/>
</datatype>
<datatype>
@@ -112,6 +119,11 @@ list_to_binary([Compressed|Last])</pre>
<fsummary>Calculate the Adler checksum.</fsummary>
<desc>
<p>Calculates the Adler-32 checksum for <c><anno>Data</anno></c>.</p>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="erts:erlang#adler32/1">
+ <c>erlang:adler32/1</c></seealso> instead.</p>
+ </warning>
</desc>
</func>
@@ -127,6 +139,11 @@ list_to_binary([Compressed|Last])</pre>
Crc = lists:foldl(fun(Data,Crc0) ->
zlib:adler32(Z, Crc0, Data),
end, zlib:adler32(Z,&lt;&lt; &gt;&gt;), Datas)</pre>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="erts:erlang#adler32/2">
+ <c>erlang:adler32/2</c></seealso> instead.</p>
+ </warning>
</desc>
</func>
@@ -141,6 +158,11 @@ Crc = lists:foldl(fun(Data,Crc0) ->
<p>This function returns the <c><anno>Adler</anno></c> checksum of
<c>[Data1,Data2]</c>, requiring only <c><anno>Adler1</anno></c>,
<c><anno>Adler2</anno></c>, and <c><anno>Size2</anno></c>.</p>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="erts:erlang#adler32_combine/3">
+ <c>erlang:adler32_combine/3</c></seealso> instead.</p>
+ </warning>
</desc>
</func>
@@ -165,6 +187,12 @@ Crc = lists:foldl(fun(Data,Crc0) ->
<fsummary>Get current CRC.</fsummary>
<desc>
<p>Gets the current calculated CRC checksum.</p>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="erts:erlang#crc32/1">
+ <c>erlang:crc32/1</c></seealso> on the uncompressed data
+ instead.</p>
+ </warning>
</desc>
</func>
@@ -173,6 +201,11 @@ Crc = lists:foldl(fun(Data,Crc0) ->
<fsummary>Calculate CRC.</fsummary>
<desc>
<p>Calculates the CRC checksum for <c><anno>Data</anno></c>.</p>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="erts:erlang#crc32/1">
+ <c>erlang:crc32/1</c></seealso> instead.</p>
+ </warning>
</desc>
</func>
@@ -188,6 +221,11 @@ Crc = lists:foldl(fun(Data,Crc0) ->
Crc = lists:foldl(fun(Data,Crc0) ->
zlib:crc32(Z, Crc0, Data),
end, zlib:crc32(Z,&lt;&lt; &gt;&gt;), Datas)</pre>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="erts:erlang#crc32/2">
+ <c>erlang:crc32/2</c></seealso> instead.</p>
+ </warning>
</desc>
</func>
@@ -202,6 +240,11 @@ Crc = lists:foldl(fun(Data,Crc0) ->
<p>This function returns the <c><anno>CRC</anno></c> checksum of
<c>[Data1,Data2]</c>, requiring only <c><anno>CRC1</anno></c>,
<c><anno>CRC2</anno></c>, and <c><anno>Size2</anno></c>.</p>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="erts:erlang#crc32_combine/3">
+ <c>erlang:crc32_combine/3</c></seealso> instead.</p>
+ </warning>
</desc>
</func>
@@ -407,8 +450,8 @@ list_to_binary([B1,B2])</pre>
<seealso marker="#deflateInit/1"><c>deflateInit/1,2,6</c></seealso> or
<seealso marker="#deflateReset/1"><c>deflateReset/1</c></seealso>,
before any call of
- <seealso marker="#deflate/3"><c>deflate/3</c></seealso>.
- The compressor and decompressor must use the same dictionary (see
+ <seealso marker="#deflate/3"><c>deflate/3</c></seealso>.</p>
+ <p>The compressor and decompressor must use the same dictionary (see
<seealso marker="#inflateSetDictionary/2">
<c>inflateSetDictionary/2</c></seealso>).</p>
<p>The Adler checksum of the dictionary is returned.</p>
@@ -420,6 +463,10 @@ list_to_binary([B1,B2])</pre>
<fsummary>Get buffer size.</fsummary>
<desc>
<p>Gets the size of the intermediate buffer.</p>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release.</p>
+ </warning>
</desc>
</func>
@@ -443,14 +490,31 @@ list_to_binary([B1,B2])</pre>
<name name="inflate" arity="2"/>
<fsummary>Decompress data.</fsummary>
<desc>
- <p>Decompresses as much data as possible.
- It can introduce some output latency (reading
- input without producing any output).</p>
- <p>If a preset dictionary is needed at this point (see
- <seealso marker="#inflateSetDictionary/2">
- <c>inflateSetDictionary/2</c></seealso>), <c>inflate/2</c> throws a
- <c>{need_dictionary,Adler}</c> exception, where <c>Adler</c> is
- the Adler-32 checksum of the dictionary chosen by the compressor.</p>
+ <p>Equivalent to
+ <seealso marker="#inflate/3"><c>inflate(Z, Data, [])</c></seealso>
+ </p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="inflate" arity="3"/>
+ <fsummary>Decompress data.</fsummary>
+ <desc>
+ <p>Decompresses as much data as possible. It can introduce some output
+ latency (reading input without producing any output).</p>
+ <p>Currently the only available option is
+ <c>{exception_on_need_dict,boolean()}</c> which controls whether the
+ function should throw an exception when a preset dictionary is
+ required for decompression. When set to false, a
+ <c>need_dictionary</c> tuple will be returned instead. See
+ <seealso marker="#inflateSetDictionary/2">
+ <c>inflateSetDictionary/2</c></seealso> for details.</p>
+ <warning>
+ <p>This option defaults to <c>true</c> for backwards compatibility
+ but we intend to remove the exception behavior in a future
+ release. New code that needs to handle dictionaries manually
+ should always specify <c>{exception_on_need_dict,false}</c>.</p>
+ </warning>
</desc>
</func>
@@ -458,6 +522,11 @@ list_to_binary([B1,B2])</pre>
<name name="inflateChunk" arity="1"/>
<fsummary>Read next uncompressed chunk.</fsummary>
<desc>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="#safeInflate/2"><c>safeInflate/2</c>
+ </seealso> instead.</p>
+ </warning>
<p>Reads the next chunk of uncompressed data, initialized by
<seealso marker="#inflateChunk/2"><c>inflateChunk/2</c></seealso>.</p>
<p>This function is to be repeatedly called, while it returns
@@ -469,23 +538,27 @@ list_to_binary([B1,B2])</pre>
<name name="inflateChunk" arity="2"/>
<fsummary>Decompress data with limited output size.</fsummary>
<desc>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release. Use <seealso marker="#safeInflate/2"><c>safeInflate/2</c>
+ </seealso> instead.</p>
+ </warning>
<p>Like <seealso marker="#inflate/2"><c>inflate/2</c></seealso>,
- but decompresses no more data than will fit in the buffer configured
- through <seealso marker="#setBufSize/2"><c>setBufSize/2</c></seealso>.
- Is is useful when decompressing a stream with a high compression
- ratio, such that a small amount of compressed input can expand up to
- 1000 times.</p>
+ but decompresses no more data than will fit in the buffer configured
+ through <seealso marker="#setBufSize/2"><c>setBufSize/2</c>
+ </seealso>. Is is useful when decompressing a stream with a high
+ compression ratio, such that a small amount of compressed input can
+ expand up to 1000 times.</p>
<p>This function returns <c>{more, Decompressed}</c>, when there is
- more output available, and
- <seealso marker="#inflateChunk/1"><c>inflateChunk/1</c></seealso>
- is to be used to read it.</p>
- <p>This function can introduce some output latency (reading
- input without producing any output).</p>
- <p>If a preset dictionary is needed at this point (see
- <seealso marker="#inflateSetDictionary/2">
- <c>inflateSetDictionary/2</c></seealso>), this function throws a
- <c>{need_dictionary,Adler}</c> exception, where <c>Adler</c> is
- the Adler-32 checksum of the dictionary chosen by the compressor.</p>
+ more output available, and
+ <seealso marker="#inflateChunk/1"><c>inflateChunk/1</c></seealso>
+ is to be used to read it.</p>
+ <p>This function can introduce some output latency (reading input
+ without producing any output).</p>
+ <p>An exception will be thrown if a preset dictionary is required for
+ further decompression. See
+ <seealso marker="#inflateSetDictionary/2">
+ <c>inflateSetDictionary/2</c></seealso> for details.</p>
<p>Example:</p>
<pre>
walk(Compressed, Handler) ->
@@ -517,6 +590,18 @@ loop(Z, Handler, Uncompressed) ->
</func>
<func>
+ <name name="inflateGetDictionary" arity="1"/>
+ <fsummary>Return the decompression dictionary.</fsummary>
+ <desc>
+ <p>Returns the decompression dictionary currently in use
+ by the stream. This function must be called between
+ <seealso marker="#inflateInit/1"><c>inflateInit/1,2</c></seealso>
+ and <seealso marker="#inflateEnd/1"><c>inflateEnd</c></seealso>.</p>
+ <p>Only supported if ERTS was compiled with zlib >= 1.2.8.</p>
+ </desc>
+ </func>
+
+ <func>
<name name="inflateInit" arity="1"/>
<fsummary>Initialize a session for decompression.</fsummary>
<desc>
@@ -562,45 +647,83 @@ loop(Z, Handler, Uncompressed) ->
<fsummary>Initialize the decompression dictionary.</fsummary>
<desc>
<p>Initializes the decompression dictionary from the specified
- uncompressed byte sequence. This function must be called
- immediately after a call of
- <seealso marker="#inflate/2"><c>inflate/2</c></seealso>
- if this call threw a <c>{need_dictionary,Adler}</c> exception.
- The dictionary chosen by the compressor can be determined from the
- Adler value thrown by the call to <c>inflate/2</c>.
- The compressor and decompressor must use the same dictionary (see
- <seealso marker="#deflateSetDictionary/2">
- <c>deflateSetDictionary/2</c></seealso>).</p>
+ uncompressed byte sequence. This function must be called as a
+ response to an inflate operation (eg.
+ <seealso marker="#safeInflate/2"><c>safeInflate/2</c></seealso>)
+ returning <c>{need_dictionary,Adler,Output}</c> or in the case of
+ deprecated functions, throwing an
+ <c>{'EXIT',{{need_dictionary,Adler},_StackTrace}}</c> exception.</p>
+ <p>The dictionary chosen by the compressor can be determined from the
+ Adler value returned or thrown by the call to the inflate function.
+ The compressor and decompressor must use the same dictionary (See
+ <seealso marker="#deflateSetDictionary/2">
+ <c>deflateSetDictionary/2</c></seealso>).</p>
+ <p>After setting the dictionary the inflate operation should be
+ retried without new input.</p>
<p>Example:</p>
<pre>
-unpack(Z, Compressed, Dict) ->
+deprecated_unpack(Z, Compressed, Dict) ->
case catch zlib:inflate(Z, Compressed) of
- {'EXIT',{{need_dictionary,DictID},_}} ->
- zlib:inflateSetDictionary(Z, Dict),
+ {'EXIT',{{need_dictionary,_DictID},_}} ->
+ ok = zlib:inflateSetDictionary(Z, Dict),
Uncompressed = zlib:inflate(Z, []);
Uncompressed ->
Uncompressed
- end.</pre>
+ end.
+
+new_unpack(Z, Compressed, Dict) ->
+ case zlib:inflate(Z, Compressed, [{exception_on_need_dict, false}]) of
+ {need_dictionary, _DictId, Output} ->
+ ok = zlib:inflateSetDictionary(Z, Dict),
+ [Output | zlib:inflate(Z, [])];
+ Uncompressed ->
+ Uncompressed
+ end.</pre>
</desc>
</func>
<func>
- <name name="inflateGetDictionary" arity="1"/>
- <fsummary>Return the decompression dictionary.</fsummary>
+ <name name="open" arity="0"/>
+ <fsummary>Open a stream and return a stream reference.</fsummary>
<desc>
- <p>Returns the decompression dictionary currently in use
- by the stream. This function must be called between
- <seealso marker="#inflateInit/1"><c>inflateInit/1,2</c></seealso>
- and <seealso marker="#inflateEnd/1"><c>inflateEnd</c></seealso>.</p>
- <p>Only supported if ERTS was compiled with zlib >= 1.2.8.</p>
+ <p>Opens a zlib stream.</p>
</desc>
</func>
<func>
- <name name="open" arity="0"/>
- <fsummary>Open a stream and return a stream reference.</fsummary>
+ <name name="safeInflate" arity="2"/>
+ <fsummary>Decompress data with limited output size.</fsummary>
<desc>
- <p>Opens a zlib stream.</p>
+ <p>Like <seealso marker="#inflate/2"><c>inflate/2</c></seealso>,
+ but returns once it has expanded beyond a small
+ implementation-defined threshold. It's useful when decompressing
+ untrusted input which could have been maliciously crafted to expand
+ until the system runs out of memory.</p>
+ <p>This function returns <c>{continue | finished, Output}</c>, where
+ <anno>Output</anno> is the data that was decompressed in this call.
+ New input can be queued up on each call if desired, and the function
+ will return <c>{finished, Output}</c> once all queued data has been
+ decompressed.</p>
+ <p>This function can introduce some output latency (reading
+ input without producing any output).</p>
+ <p>If a preset dictionary is required for further decompression, this
+ function returns a <c>need_dictionary</c> tuple. See
+ <seealso marker="#inflateSetDictionary/2">
+ <c>inflateSetDictionary/2</c></seealso>) for details.</p>
+ <p>Example:</p>
+ <pre>
+walk(Compressed, Handler) ->
+ Z = zlib:open(),
+ zlib:inflateInit(Z),
+ loop(Z, Handler, zlib:safeInflate(Z, Compressed)),
+ zlib:inflateEnd(Z),
+ zlib:close(Z).
+
+loop(Z, Handler, {continue, Output}) ->
+ Handler(Output),
+ loop(Z, Handler, zlib:safeInflate(Z, []));
+loop(Z, Handler, {finished, Output}) ->
+ Handler(Output).</pre>
</desc>
</func>
@@ -609,6 +732,10 @@ unpack(Z, Compressed, Dict) ->
<fsummary>Set buffer size.</fsummary>
<desc>
<p>Sets the intermediate buffer size.</p>
+ <warning>
+ <p>This function is deprecated and will be removed in a future
+ release.</p>
+ </warning>
</desc>
</func>