Add initial Ranch guide

8 files changed, 550 insertions, 54 deletions

diff --git a/README.md b/README.md
index 2fbe9cd..08aea1e 100644
--- a/README.md
+++ b/README.md
@@ -35,57 +35,6 @@ Quick start
 Getting Started
 ---------------
 
-Ranch accepts connections received on a given port and using a given
-transport, like TCP or SSL, and forward them to a given protocol
-handler. Acceptors and protocol handler processes are of course
-supervised automatically.
-
-Ranch does nothing by default. You need to explicitly request Ranch
-to listen on a port with your chosen transport and protocol handlers.
-To do so, you must start a listener.
-
-A listener is a special kind of supervisor that manages both the
-acceptor pool and the protocol processes. It is named and can thus be
-started and stopped at will.
-
-An acceptor pool is a pool of processes whose only role is to accept
-new connections. It's good practice to have many of these processes
-as they are very cheap and allow much quicker response when you get
-many connections. Of course, as with everything else, you should
-**benchmark** before you decide what's best for you.
-
-Ranch includes both TCP and SSL transport handlers, abstracted through
-a single common interface.
-
-You can start and stop listeners by calling `ranch:start_listener/6` and
-`ranch:stop_listener/1` respectively.
-
-The following example demonstrates the startup of a very simple listener.
-
-``` erlang
-application:start(ranch),
-%% Name, NbAcceptors, Transport, TransOpts, Protocol, ProtoOpts
-ranch:start_listener(my_echo_listener, 100,
-    ranch_tcp, [{port, 1234}],
-    my_echo_protocol, [{log, "echo.log"}]
-).
-```
-
-Writing a protocol handler
---------------------------
-
-The only exported function a protocol handler needs is the start_link/4
-function, with arguments ListenerPid, Socket, Transport and Opts. ListenerPid
-is the pid to the listener's gen_server, managing the connections. Socket is of
-course the client socket; Transport is the module name of the chosen transport
-handler and Opts is protocol options defined when starting the listener.
-
-After initializing your protocol, it is recommended to call the
-function ranch:accept_ack/1 with the ListenerPid as argument,
-as it will ensure Ranch has been able to fully initialize the socket.
-Anything you do past this point is up to you!
-
-If you need to change some socket options, like enabling raw mode for example,
-you can call the <em>Transport:setopts/2</em> function. It is the protocol's
-responsability to manage the socket usage, there should be no need for an user
-to specify that kind of options while starting a listener.
+* [Read the guide](https://github.com/extend/ranch/blob/master/guide/toc.md)
+* Look at the examples in the ```examples/``` directory
+* Build API documentation with ```make docs```; open ```doc/index.html```
diff --git a/guide/embedded.md b/guide/embedded.md
new file mode 100644
index 0000000..82dab75
--- /dev/null
+++ b/guide/embedded.md
@@ -0,0 +1,50 @@
+Embedded mode
+=============
+
+Purpose
+-------
+
+Embedded mode allows you to insert Ranch listeners directly
+in your supervision tree. This allows for greater fault tolerance
+control by permitting the shutdown of a listener due to the
+failure of another part of the application and vice versa.
+
+Embedding
+---------
+
+To embed Ranch in your application you can simply add the child specs
+to your supervision tree. This can all be done in the ```init/1``` function
+of one of your application supervisors.
+
+Ranch requires at the minimum two kinds of child specs for embedding.
+First, you need to add ```ranch_sup``` to your supervision tree, only once,
+regardless of the number of listeners you will use. Then you need to
+add the child specs for each listener.
+
+Ranch has a convenience function for getting the listeners child specs
+called ```ranch:child_spec/6```, that works like ```ranch:start_listener/6```,
+except that it doesn't start anything, it only returns child specs.
+
+As for ```ranch_sup```, the child spec is simple enough to not require a
+convenience function.
+
+The following example adds both ```ranch_sup``` and one listener to another
+application's supervision tree.
+
+``` erlang
+init([]) ->
+    RanchSupSpec = {ranch_sup, {ranch_sup, start_link, []},
+        permanent, 5000, supervisor, [ranch_sup]},
+    ListenerSpec = ranch:child_spec(echo, 100,
+        ranch_tcp, [{port, 5555}],
+        echo_protocol, []
+    ),
+    {ok, {{one_for_one, 10, 10}, [RanchSupSpec, ListenerSpec]}}.
+```
+
+Remember, you can add as many listener child specs as needed, but only
+one ```ranch_sup``` spec!
+
+It is recommended that your architecture makes sure that all listeners
+are restarted if ```ranch_sup``` fails. See the Ranch internals chapter for
+more details on how Ranch does it.
diff --git a/guide/internals.md b/guide/internals.md
new file mode 100644
index 0000000..2dfd457
--- /dev/null
+++ b/guide/internals.md
@@ -0,0 +1,80 @@
+Internals
+=========
+
+This chapter may not apply to embedded Ranch as embedding allows you
+to use an architecture specific to your application, which may or may
+not be compatible with the description of the Ranch application.
+
+Architecture
+------------
+
+Ranch is an OTP application.
+
+Like all OTP applications, Ranch has a top supervisor. It is responsible
+for supervising the ```ranch_server``` process and all the listeners that
+will be started.
+
+The ```ranch_server``` gen_server is the central process keeping track of the
+listeners, the acceptors and the connection processes. It does so through
+the use of a public ets table called ```ranch_server``` too. This allows
+some operations to be sequential by going through the gen_server, while
+others just query the ets table directly, ensuring there is no bottleneck
+for the most common operations.
+
+Because the most common operation is keeping track of the number of
+connections currently being used for each listener, the ets table
+has ```write_concurrency``` enabled, allowing us to perform all these
+operations concurrently using ```ets:update_counter/3```. To read the number
+of connections we simply increment the counter by 0, which allows us
+to stay in a write context and still receive the counter's value.
+
+For increased fault tolerance, the owner of the ets table is
+```ranch_sup``` and not ```ranch_server``` as you could expect. This way,
+if the ```ranch_server``` gen_server fails, it doesn't lose any information
+and the restarted process can continue as if nothing happened. Note that
+this usage is not recommended by OTP.
+
+Listeners are grouped into the ```ranch_listener_sup``` supervisor and
+consist of three kinds of processes: the listener gen_server, the
+acceptor processes and the connection processes, both grouped under
+their own supervisor. All of these processes are registered to the
+```ranch_server``` gen_server with varying amount of information.
+
+All socket operations, including listening for connections, go through
+transport handlers. Accepted connections are given to the protocol handler.
+Transport handlers are simple callback modules for performing operations on
+sockets. Protocol handlers start a new process, which receives socket
+ownership, with no requirements on how the code should be written inside
+that new process.
+
+Efficiency considerations
+-------------------------
+
+Note that for everything related to efficiency and performance,
+you should perform the benchmarks yourself to get the numbers that
+matter to you. Generic benchmarks found on the web may or may not
+be of use to you, you can never know until you benchmark your own
+system.
+
+* * *
+
+The second argument to ```ranch:start_listener/6``` is the number of
+processes that will be accepting connections. Care should be taken
+when choosing this number.
+
+First of all, it should not be confused with the maximum number
+of connections. Acceptor processes are only used for accepting and
+have nothing else in common with connection processes. Therefore
+there is nothing to be gained from setting this number too high,
+in fact it can slow everything else down.
+
+Second, this number should be high enough to allow Ranch to accept
+connections concurrently. But the number of cores available doesn't
+seem to be the only factor for choosing this number, as we can
+observe faster accepts if we have more acceptors than cores. It
+might be entirely dependent on the protocol, however.
+
+Our observations suggest that using 100 acceptors on modern hardware
+is a good solution, as it's big enough to always have acceptors ready
+and it's low enough that it doesn't have a negative impact on the
+system's performances.
diff --git a/guide/introduction.md b/guide/introduction.md
new file mode 100644
index 0000000..c63eaef
--- /dev/null
+++ b/guide/introduction.md
@@ -0,0 +1,25 @@
+Introduction
+============
+
+Purpose
+-------
+
+Ranch is a socket acceptor pool for TCP protocols.
+
+Ranch aims to provide everything you need to accept TCP connections
+with a small code base and low latency while being easy to use directly
+as an application or to embed into your own.
+
+Prerequisites
+-------------
+
+It is assumed the developer already knows Erlang and has some experience
+with socket programming and TCP protocols.
+
+In order to run the examples available in this user guide, you will need
+Erlang and rebar installed and in your $PATH.
+
+Please see the [rebar repository](https://github.com/basho/rebar) for
+downloading and building instructions. Please look up the environment
+variables documentation of your system for details on how to update the
+$PATH information.
diff --git a/guide/listeners.md b/guide/listeners.md
new file mode 100644
index 0000000..63cdf81
--- /dev/null
+++ b/guide/listeners.md
@@ -0,0 +1,171 @@
+Listeners
+=========
+
+Purpose
+-------
+
+A listener is a set of processes whose role is to listen on a port
+for new connections. It manages a pool of acceptor processes, each
+of them indefinitely accepting connections. When it does, it starts
+a new process executing the protocol handler code. All the socket
+programming is abstracted through the user of transport handlers.
+
+The listener takes care of supervising all the acceptor and connection
+processes, allowing developers to focus on building their application.
+
+Starting and stopping
+---------------------
+
+Ranch does nothing by default. It is up to the application developer
+to request that Ranch listens for connections.
+
+A listener can be started and stopped at will.
+
+When starting a listener, a number of different settings are required:
+ *  A name to identify it locally and be able to interact with it.
+ *  The number of acceptors in the pool.
+ *  A transport handler and its associated options.
+ *  A protocol handler and its associated options.
+
+Ranch includes both TCP and SSL transport handlers, respectively
+```ranch_tcp``` and ```ranch_ssl```.
+
+A listener can be started by calling the ```ranch:start_listener/6```
+function. Before doing so however, you must ensure that the ```ranch```
+application is started.
+
+To start the ```ranch``` application:
+
+``` erlang
+ok = application:start(ranch).
+```
+
+You are then ready to start a listener. Let's call it ```tcp_echo```. It will
+have a pool of 100 acceptors, use a TCP transport and forward connections
+to the ```echo_protocol``` handler.
+
+``` erlang
+{ok, _} = ranch:start_listener(tcp_echo, 100,
+    ranch_tcp, [{port, 5555}],
+    echo_protocol, []
+).
+```
+
+You can try this out by compiling and running the ```tcp_echo``` example in the
+examples directory. To do so, open a shell in the ```examples/tcp_echo/```
+directory and run the following commands:
+
+```
+% rebar get-deps compile
+% ./start.sh
+Listening on port 5555
+```
+
+You can then connect to it using telnet and see the echo server reply
+everything you send to it. Then when you're done testing, you can use
+the ```Ctrl+]``` key to escape to the telnet command line and type
+```quit``` to exit.
+
+```
+% telnet localhost 5555
+Trying 127.0.0.1...
+Connected to localhost.
+Escape character is '^]'.
+Hello!
+Hello!
+It works!
+It works!
+^]
+
+telnet> quit
+Connection closed.
+```
+
+Listening on a random port
+--------------------------
+
+You do not have to specify a specific port to listen on. If you give
+the port number 0, or if you omit the port number entirely, Ranch will
+start listening on a random port.
+
+You can retrieve this port number by calling ```ranch:get_port/1```. The
+argument is the name of the listener you gave in ```ranch:start_listener/6```.
+
+``` erlang
+{ok, _} = ranch:start_listener(tcp_echo, 100,
+    ranch_tcp, [{port, 0}],
+    echo_protocol, []
+).
+Port = ranch:get_port(tcp_echo).
+```
+
+Listening on a port =< 1024
+---------------------------
+
+This is currently not possible. We recommend the use of load balancing
+or NAT firewall rules if the need arise. Proxies can sometimes also be
+used although that's a less efficient solution.
+
+Limiting the number of concurrent connections
+---------------------------------------------
+
+The ```max_connections``` transport option allows you to limit the number
+of concurrent connections. It defaults to 1024. Its purpose is to
+prevent your system from being overloaded and ensuring all the
+connections are handled optimally.
+
+``` erlang
+{ok, _} = ranch:start_listener(tcp_echo, 100,
+    ranch_tcp, [{port, 5555}, {max_connections, 100}],
+    echo_protocol, []
+).
+```
+
+You can disable this limit by setting its value to the atom ```infinity```.
+
+``` erlang
+{ok, _} = ranch:start_listener(tcp_echo, 100,
+    ranch_tcp, [{port, 5555}, {max_connections, infinity}],
+    echo_protocol, []
+).
+```
+
+You may not always want connections to be counted when checking for
+```max_connections```. For example you might have a protocol where both
+short-lived and long-lived connections are possible. If the long-lived
+connections are mostly waiting for messages, then they don't consume
+much resources and can safely be removed from the count.
+
+To remove the connection from the count, you must call the
+```ranch_listener:remove_connection/1``` from within the connection process,
+with the listener pid as the only argument.
+
+``` erlang
+ranch_listener:remove_connection(ListenerPid).
+```
+
+As seen in the chapter covering protocols, this pid is received as the
+first argument of the protocol's ```start_link/4``` callback.
+
+Upgrading
+---------
+
+Ranch allows you to upgrade the protocol options. This takes effect
+immediately and for all subsequent connections.
+
+To upgrade the protocol options, call ```ranch:set_protocol_options/2```
+with the name of the listener as first argument and the new options
+as the second.
+
+``` erlang
+ranch:set_protocol_options(tcp_echo, NewOpts).
+```
+
+All future connections will use the new options.
+
+You can also retrieve the current options similarly by
+calling ```ranch:get_protocol_options/1```.
+
+``` erlang
+Opts = ranch:get_protocol_options(tcp_echo).
+```
diff --git a/guide/protocols.md b/guide/protocols.md
new file mode 100644
index 0000000..7db8fec
--- /dev/null
+++ b/guide/protocols.md
@@ -0,0 +1,61 @@
+Protocols
+=========
+
+Purpose
+-------
+
+A protocol handler starts a connection process and defines the
+protocol logic executed in this process.
+
+Writing a protocol handler
+--------------------------
+
+All protocol handlers must implement the ```ranch_protocol``` behavior
+which defines a single callback, ```start_link/4```. This callback is
+responsible for spawning a new process for handling the connection.
+It receives four arguments: the listener's pid, the socket, the
+transport handler being used and the protocol options defined in
+the call to ```ranch:start_listener/6```. This callback must
+return ```{ok, Pid}```, with ```Pid``` the pid of the new process.
+
+The newly started process can then freely initialize itself. However,
+it must call ```ranch:accept_ack/1``` before doing any socket operation.
+This will ensure the connection process is the owner of the socket.
+It expects the listener's pid as argument.
+
+``` erlang
+ok = ranch:accept_ack(ListenerPid).
+```
+
+If your protocol code requires specific socket options, you should
+set them while initializing your connection process and before
+starting ```ranch:accept_ack/1```. You can use ```Transport:setopts/2```
+for that purpose.
+
+Following is the complete protocol code for the example found
+in ```examples/tcp_echo/```.
+
+``` erlang
+-module(echo_protocol).
+-behaviour(ranch_protocol).
+
+-export([start_link/4]).
+-export([init/4]).
+
+start_link(ListenerPid, Socket, Transport, Opts) ->
+    Pid = spawn_link(?MODULE, init, [ListenerPid, Socket, Transport, Opts]),
+    {ok, Pid}.
+
+init(ListenerPid, Socket, Transport, _Opts = []) ->
+    ok = ranch:accept_ack(ListenerPid),
+    loop(Socket, Transport).
+
+loop(Socket, Transport) ->
+    case Transport:recv(Socket, 0, 5000) of
+        {ok, Data} ->
+            Transport:send(Socket, Data),
+            loop(Socket, Transport);
+        _ ->
+            ok = Transport:close(Socket)
+    end.
+```
diff --git a/guide/toc.md b/guide/toc.md
new file mode 100644
index 0000000..87719df
--- /dev/null
+++ b/guide/toc.md
@@ -0,0 +1,28 @@
+Ranch User Guide
+================
+
+ *  [Introduction](introduction.md)
+   *  Purpose
+   *  Prerequisites
+ *  [Listeners](listeners.md)
+   *  Purpose
+   *  Starting and stopping
+   *  Listening on a random port
+   *  Listening on a port =< 1024
+   *  Limiting the number of concurrent connections
+   *  Upgrading
+ *  [Transports](transports.md)
+   *  Purpose
+   *  TCP transport
+   *  SSL transport
+   *  Sending and receiving data
+   *  Writing a transport handler
+ *  [Protocols](protocols.md)
+   *  Purpose
+   *  Writing a protocol handler
+ *  [Embedded mode](embedded.md)
+   *  Purpose
+   *  Embedding
+ *  [Internals](internals.md)
+   *  Architecture
+   *  Efficiency considerations
diff --git a/guide/transports.md b/guide/transports.md
new file mode 100644
index 0000000..4fee94a
--- /dev/null
+++ b/guide/transports.md
@@ -0,0 +1,132 @@
+Transports
+==========
+
+Purpose
+-------
+
+A transport defines the interface to interact with a socket.
+
+Transports can be used for connecting, listening and accepting
+connections, but also for receiving and sending data. Both
+passive and active mode are supported, although all sockets
+are initialized as passive.
+
+TCP transport
+-------------
+
+The TCP transport is a thin wrapper around ```gen_tcp```.
+
+SSL transport
+-------------
+
+The SSL transport is a thin wrapper around ```ssl```. It requires
+the ```crypto```, ```public_key``` and ```ssl``` applications to be started.
+You can start each of them individually, or you can call the
+```ssl:start/0``` convenience function.
+
+``` erlang
+ssl:start().
+```
+
+In a proper OTP setting, you will need to make your application
+depend on the ```crypto```, ```public_key``` and ```ssl``` applications.
+They will be started automatically when starting your release.
+
+The SSL transport ```accept/2``` function performs both transport
+and SSL accepts. Errors occurring during the SSL accept phase
+are returned as ```{error, {ssl_accept, atom()}}``` to differentiate
+on which socket the problem occurred.
+
+Sending and receiving data
+--------------------------
+
+This section assumes that ```Transport``` is a valid transport handler
+(like ```ranch_tcp``` or ```ranch_ssl```) and ```Socket``` is a connected
+socket obtained through the listener.
+
+You can send data to a socket by calling the ```Transport:send/2```
+function. The data can be given as ```iodata()```, which is defined as
+```binary() | iolist()```. All the following calls will work:
+
+``` erlang
+Transport:send(Socket, <<"Ranch is cool!">>).
+Transport:send(Socket, "Ranch is cool!").
+Transport:send(Socket, ["Ranch", ["is", "cool!"]]).
+Transport:send(Socket, ["Ranch", [<<"is">>, "cool!"]]).
+```
+
+You can receive data either in passive or in active mode. Passive mode
+means that you will perform a blocking ```Transport:recv/2``` call, while
+active mode means that you will receive the data as a message.
+
+By default, all data will be received as binary. It is possible to
+receive data as strings, although this is not recommended as binaries
+are a more efficient construct, especially for binary protocols.
+
+Receiving data using passive mode requires a single function call. The
+first argument is the socket, and the third argument is a timeout duration
+before the call returns with ```{error, timeout}```.
+
+The second argument is the amount of data in bytes that we want to receive.
+The function will wait for data until it has received exactly this amount.
+If you are not expecting a precise size, you can specify 0 which will make
+this call return as soon as data was read, regardless of its size.
+
+``` erlang
+{ok, Data} = Transport:recv(Socket, 0, 5000).
+```
+
+Active mode requires you to inform the socket that you want to receive
+data as a message and to write the code to actually receive it.
+
+There are two kinds of active modes: ```{active, once}``` and
+```{active, true}```. The first will send a single message before going
+back to passive mode; the second will send messages indefinitely.
+We recommend not using the ```{active, true}``` mode as it could quickly
+flood your process mailbox. It's better to keep the data in the socket
+and read it only when required.
+
+Three different messages can be received:
+ *  ```{OK, Socket, Data}```
+ *  ```{Closed, Socket}```
+ *  ```{Error, Socket, Reason}```
+
+The value of ```OK```, ```Closed``` and ```Error``` can be different
+depending on the transport being used. To be able to properly match
+on them you must first call the ```Transport:messages/0``` function.
+
+``` erlang
+{OK, Closed, Error} = Transport:messages().
+```
+
+To start receiving messages you will need to call the ```Transport:setopts/2```
+function, and do so every time you want to receive data.
+
+``` erlang
+{OK, Closed, Error} = Transport:messages(),
+Transport:setopts(Socket, [{active, once}]),
+receive
+    {OK, Socket, Data} ->
+        io:format("data received: ~p~n", [Data]);
+    {Closed, Socket} ->
+        io:format("socket got closed!~n");
+    {Error, Socket, Reason} ->
+        io:format("error happened: ~p~n", [Reason])
+end.
+```
+
+You can easily integrate active sockets with existing Erlang code as all
+you really need is just a few more clauses when receiving messages.
+
+Writing a transport handler
+---------------------------
+
+A transport handler is a module implementing the ```ranch_transport``` behavior.
+It defines a certain number of callbacks that must be written in order to
+allow transparent usage of the transport handler.
+
+The behavior doesn't define the socket options available when opening a
+socket. These do not need to be common to all transports as it's easy enough
+to write different initialization functions for the different transports that
+will be used. With one exception though. The ```setopts/2``` function *must*
+implement the ```{active, once}``` and the ```{active, true}``` options.