From 473dbb5ce698f5128858df1c85c33a696a0a0bbc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Lo=C3=AFc=20Hoguin?= Date: Fri, 23 Aug 2013 23:20:34 +0200 Subject: First draft of the guide --- guide/connect.md | 94 +++++++++++++++++++++++ guide/http.md | 208 ++++++++++++++++++++++++++++++++++++++++++++++++++ guide/introduction.md | 35 +++++++++ guide/protocols.md | 79 +++++++++++++++++++ guide/toc.md | 11 +++ guide/websocket.md | 85 +++++++++++++++++++++ 6 files changed, 512 insertions(+) create mode 100644 guide/connect.md create mode 100644 guide/http.md create mode 100644 guide/introduction.md create mode 100644 guide/protocols.md create mode 100644 guide/toc.md create mode 100644 guide/websocket.md diff --git a/guide/connect.md b/guide/connect.md new file mode 100644 index 0000000..4410e60 --- /dev/null +++ b/guide/connect.md @@ -0,0 +1,94 @@ +Connection +========== + +This chapter describes how to open, monitor and close +a connection using the Gun client. + +Opening a new connection +------------------------ + +Gun is designed with the SPDY and Websocket protocols in mind, +and as such establishes a permanent connection to the remote +server. Because of this, the connection must be initiated +before being able to send any request. + +The process that creates the connection is also known as the +owner of the connection. Only this process can perform operations +on the connection, and only this process will receive messages +from the connection. + +To open a new connection, the `gun:open/3` function can be used. + +``` erlang +{ok, Pid} = gun:open("twitter.com", 443, []). +``` + +The connection is managed by a separate process and is supervised +by the Gun supervisor directly. + +The connection can later be stopped either gracefully or abruptly +by the client. If an unexpected disconnection occurs, the client +will retry connecting every few seconds until it succeeds and +can resume normal operations. + +Monitoring the connection process +--------------------------------- + +The connection is managed by a separate process. Because +software errors are a reality, it is important to monitor +this process for failure. Thankfully, do to the asynchronous +nature of Gun, we only need to create a monitor once when +the connection is established. + +``` erlang +{ok, Pid} = gun:open("twitter.com", 443, []). +MRef = monitor(process, Pid). +``` + +There is no need to monitor again after that regardless of +the number of requests sent or messages received. + +You can detect the process failure when receiving messages. + +``` erlang +receive + {'DOWN', Tag, _, _, Reason} -> + error_logger:error_msg("Oops!"), + exit(Reason); + %% Receive Gun messages here... +end. +``` + +You will probably want to reopen the connection when that +happens. + +Closing the connection abruptly +------------------------------- + +The connection can be stopped abruptly at any time by calling +the `gun:close/1` function. + +``` erlang +gun:close(Pid). +``` + +The process is stopped immediately. + +Closing the connection gracefully +--------------------------------- + +The connection can also be stopped gracefully by calling the +`gun:shutdown/1` function. + +``` erlang +gun:shutdown(Pid). +``` + +Gun will refuse any new requests from both the Erlang side and +the server and will attempt to finish the currently opened +streams. For example if you performed a GET request just before +calling `gun:shutdown/1`, you will still receive the response +before Gun closes the connection. + +If you set a monitor beforehand, it will inform you when the +connection has finally been shutdown. diff --git a/guide/http.md b/guide/http.md new file mode 100644 index 0000000..1034a5d --- /dev/null +++ b/guide/http.md @@ -0,0 +1,208 @@ +Using HTTP +========== + +This chapter describes how to use the Gun client for +communicating with an HTTP or SPDY server. + +Streams +------- + +Every time a request is initiated, either by the client or the +server, Gun creates a "stream". The stream controls whether +the endpoints are still sending any data, and allows you to +identify incoming messages. + +Streams are references in Gun, and are therefore always unique. + +Streams can be canceled at any time. This will stop any further +messages to be sent to the controlling process. Depending on +its capabilities, the server will also be instructed to drop +the request. + +Canceling a stream may result in Gun dropping the connection +temporarily, to avoid uploading or downloading data that will +not be used. This situation can only occur with HTTP, as SPDY +features stream canceling as part of its protocol. + +To cancel a stream, the `gun:cancel/2` function can be used. + +``` erlang +gun:cancel(Pid, StreamRef}. +``` + +Sending requests +---------------- + +Gun provides many convenient functions for performing common +operations, like GET, POST or DELETE. It also provides a +general purpose function in case you need other methods. + +The availability of these methods on the server can vary +depending on the software used but also on a per-resource +basis. + +To retrieve a resource, `gun:get/{2,3}` can be used. If you +don't need the response body, `gun:head/{2,3}` is available. +As this type of requests can't have a body, only the path +and optionally the headers can be specified. + +``` erlang +%% Without headers. +StreamRef = gun:get(Pid, "/organizations/extend"). +%% With headers. +StreamRef = gun:get(Pid, "/organizations/extend", [ + {"accept", "application/json"}, + {"user-agent", "revolver/1.0"}]). +``` + +To create or update a resource, the functions `gun:patch/{3,4}`, +`gun:post/{3,4}` and `gun:put/{3,4}` can be used. As this type +of request is meant to come with a body, headers are not optional, +because you must specify at least the content-type of the body, +and if possible also the content-length. The body is however +optional, because there might not be any at all, or because it +will be subsequently streamed. If a body is set here it is assumed +to be the full body. + +``` erlang +%% Without body. +StreamRef = gun:put(Pid, "/organizations/extend", [ + {"content-length", 23}, + {"content-type", "application/json"}]). +%% With body. +StreamRef = gun:put(Pid, "/organizations/extend", [ + {"content-length", 23}, + {"content-type", "application/json"}], + "{\"msg\": \"Hello world!\"}"). +``` + +To delete a resource, the `gun:delete/{2,3}` function can be +used. It works similarly to the GET and HEAD functions. + +``` erlang +%% Without headers. +StreamRef = gun:delete(Pid, "/organizations/extend"). +%% With headers. +StreamRef = gun:delete(Pid, "/organizations/extend", [ + {"accept", "application/json"}, + {"user-agent", "revolver/1.0"}]). +``` + +To obtain the functionality available for a given resource, +the `gun:options/{2,3}` can be used. It also works like the +GET and HEAD functions. + +``` erlang +%% Without headers. +StreamRef = gun:options(Pid, "/organizations/extend"). +%% With headers. +StreamRef = gun:options(Pid, "/organizations/extend", [ + {"accept", "application/json"}, + {"user-agent", "revolver/1.0"}]). +``` + +You can obtain information about the server as a whole by +using the special path `"*"`. + +``` erlang +StreamRef = gun:options(Pid, "*"). +``` + +Streaming data +-------------- + +When a PATCH, POST or PUT operation is performed, and a +content-type is specified but no body is given, Gun will +expect data to be streamed to the connection using the +`gun:data/4` function. + +This function can be called as many times as needed until +all data is sent. The third argument needs to be `nofin` +when there is remaining data to be sent, and `fin` for the +last chunk. The last chunk may be empty if needed. + +For example, with an `IoDevice` opened like follow: + +``` erlang +{ok, IoDevice} = file:open(Filepath, [read, binary, raw]). +``` + +The following function will stream all data until the end +of the file: + +``` erlang +sendfile(Pid, StreamRef, IoDevice) -> + case file:read(IoDevice, 8000) of + eof -> + gun:data(Pid, StreamRef, fin, <<>>), + file:close(IoDevice); + {ok, Bin} -> + gun:data(Pid, StreamRef, nofin, Bin), + sendfile(Pid, StreamRef, IoDevice) + end. +``` + +Receiving responses +------------------- + +All data received from the server is sent to the controlling +process as a message. First a response message is sent, then +zero or more data messages. If something goes wrong, error +messages are sent instead. + +You can find out whether data will be sent by checking the +presence of the content-type or content-length header and +making sure the length isn't 0. If you already know a body is +going to be sent you can skip this check, however do make +sure you have a timeout just in case something goes wrong. + +``` erlang +StreamRef = gun:get(Pid, "/"), +receive + {'DOWN', Tag, _, _, Reason} -> + error_logger:error_msg("Oops!"), + exit(Reason); + {gun_response, Pid, StreamRef, Status, Headers} -> + receive_data(Pid, StreamRef) +after 1000 -> + exit(timeout) +end. +``` + +The `receive_data/2` function could look like this: + +``` erlang +receive_data(Pid, Tag, StreamRef) -> + receive + {'DOWN', Tag, _, _, Reason} -> + {error, incomplete}; + {gun_data, Pid, StreamRef, nofin, Data} -> + io:format("~s~n", [Data]), + receive_data(Pid, Tag, StreamRef); + {gun_data, Pid, StreamRef, fin, Data} -> + io:format("~s~n", [Data]) + after 1000 -> + {error, timeout} + end. +``` + +While it may seem verbose, using messages like this has the +advantage of never locking your process, allowing you to +easily debug your code. It also allows you to start more than +one connection and concurrently perform queries on all of them +at the same time. + +You may also use Gun in a synchronous manner by writing your +own functions that perform a receive like demonstrated above. + +Dealing with server-pushed streams +---------------------------------- + +When using SPDY the server may decide to push extra resources +after a request is performed. It will send a `gun_push` message +which contains two references, one for the pushed stream, and +one for the request this stream is associated with. + +Pushed streams typically feature a body. Replying to a pushed +stream is forbidden and Gun will send an error message if +attempted. diff --git a/guide/introduction.md b/guide/introduction.md new file mode 100644 index 0000000..ca417ec --- /dev/null +++ b/guide/introduction.md @@ -0,0 +1,35 @@ +Introduction +============ + +Purpose +------- + +Gun is an asynchronous SPDY, HTTP and Websocket client. + +Prerequisites +------------- + +Knowledge of Erlang, but also of the HTTP, SPDY and Websocket +protocols is required in order to read this guide. + +Supported platforms +------------------- + +Gun is tested and supported on Linux. + +Gun is developed for Erlang R16B+. + +Gun may be compiled on earlier Erlang versions with small source code +modifications but there is no guarantee that it will work as expected. + +Conventions +----------- + +In the HTTP protocol, the method name is case sensitive. All standard +method names are uppercase. + +Header names are case insensitive. Gun converts all the header names +to lowercase, and expects your application to provide lowercase header +names also. + +The same applies to any other case insensitive value. diff --git a/guide/protocols.md b/guide/protocols.md new file mode 100644 index 0000000..87ab0cf --- /dev/null +++ b/guide/protocols.md @@ -0,0 +1,79 @@ +Supported protocols +=================== + +This chapter describes the supported protocols and lists +the calls that are valid for each of them. + +HTTP +---- + +HTTP is a text request-response protocol. The client +initiates requests and then waits for the server responses. +The server has no means of creating requests or pushing +data to the client. + +SPDY +---- + +SPDY is a binary protocol based on HTTP, compatible with +the HTTP semantics, that reduces the complexity of parsing +requests and responses, compresses the HTTP headers and +allows the server to push data directly to the client. + +Websocket +--------- + +Websocket is a binary protocol established over HTTP that +allows asynchronous concurrent communication between the +client and the server. A Websocket server can push data to +the client at any time. + +Websocket over SPDY is not supported by the Gun client at +this time. + +Operations by protocol +---------------------- + +This table lists all Gun operations and whether they are +compatible with the supported protocols. + +| Operation | SPDY | HTTP | Websocket | +| ---------- | ---- | ---- | --------- | +| delete | yes | yes | no | +| get | yes | yes | no | +| head | yes | yes | no | +| options | yes | yes | no | +| patch | yes | yes | no | +| post | yes | yes | no | +| put | yes | yes | no | +| request | yes | yes | no | +| response | yes | no | no | +| data | yes | yes | no | +| cancel | yes | yes | no | +| ws_upgrade | no | yes | no | +| ws_send | no | no | yes | + +While the `cancel` operation is available to HTTP, its effects +will only be local, as there is no way to tell the server to +stop sending data. Gun instead just doesn't forward the messages +for this stream anymore. + +Messages by protocol +-------------------- + +This table lists all messages that can be received depending +on the current protocol. + +| Message | SPDY | HTTP | Websocket | +| -------------------------------- | ---- | ---- | --------- | +| {gun_request, ...} | yes | no | no | +| {gun_response, ...} | yes | yes | no | +| {gun_data, ...} | yes | yes | no | +| {gun_error, _, StreamRef, _} | yes | yes | no | +| {gun_error, _, _} | yes | yes | yes | +| {gun_ws_upgrade, _, ok} | no | yes | no | +| {gun_ws_upgrade, _, error, ...} | no | yes | no | +| {gun_ws, ...} | no | no | yes | + +Do not forget that other messages may still be in the mailbox +after you upgrade to Websocket. diff --git a/guide/toc.md b/guide/toc.md new file mode 100644 index 0000000..00bd6bd --- /dev/null +++ b/guide/toc.md @@ -0,0 +1,11 @@ +Gun User Guide +============== + +The Gun User Guide explains in details how the Gun client +should be used for communicating with Web servers. + + * [Introduction](introduction.md) + * [Connection](connect.md) + * [Supported protocols](protocols.md) + * [Using HTTP](http.md) + * [Using Websocket](websocket.md) diff --git a/guide/websocket.md b/guide/websocket.md new file mode 100644 index 0000000..bfe5f04 --- /dev/null +++ b/guide/websocket.md @@ -0,0 +1,85 @@ +Using Websocket +=============== + +This chapter describes how to use the Gun client for +communicating with a Websocket server. + +HTTP upgrade +------------ + +Websocket is a protocol built on top of HTTP. To use Websocket, +you must first request for the connection to be upgraded. + +Gun allows you to perform Websocket upgrade requests by using +the `gun:ws_upgrade/{2,3}` function. Gun will fill out all +necessary headers for performing the Websocket upgrade, but +you can optionally specify additional headers, for example if +you would like to setup a custom sub-protocol. + +``` erlang +%% Without headers. +gun:ws_upgrade(Pid, "/websocket"). +%% With headers. +gun:ws_upgrade(Pid, "/websocket", [ + {"sec-websocket-protocol", "mychat"} +]). +``` + +The success or failure of this operation will be sent as a +message. + +``` erlang +receive + {gun_ws_upgrade, Pid, ok} -> + upgrade_success(Pid); + {gun_ws_upgrade, Pid, error, Status, Headers} -> + exit({ws_upgrade_failed, Status, Headers}); + %% More clauses here as needed. +after 1000 -> + exit(timeout); +end. +``` + +Sending data +------------ + +You can then use the `gun:ws_send/2` function to send one or +more frames to the server. + +``` erlang +%% Send one text frame. +gun:ws_send(Pid, {text, "Hello!"}). +%% Send one text frame, one binary frame and close the connection. +gun:ws_send(Pid, [ + {text, "Hello!"}, + {binary, SomeBin}, + close +]). +``` + +Note that if you send a close frame, Gun will close the connection +cleanly and will not attempt to reconnect afterwards, similar to +calling `gun:shutdown/1`. + +Receiving data +-------------- + +Every time Gun receives a frame from the server a message will be +sent to the controlling process. This message will always contain +a single frame. + +``` erlang +receive + {gun_ws, Pid, Frame} -> + handle_frame(Pid, Frame); + {gun_error, Pid, Reason} -> + error_logger:error_msg("Oops! ~p~n", [Reason]), + upgrade_again(Pid) +end. +``` + +Gun will automatically send ping messages to the server to keep +the connection alive, however if the connection dies and Gun has +to reconnect it will not upgrade to Websocket automatically, you +need to perform the operation when you receive the `gun_error` +message. -- cgit v1.2.3