From 18b29168d6a28fe16dd062bc91cc47b965766d75 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Lo=C3=AFc=20Hoguin?= Date: Fri, 2 Sep 2016 13:06:36 +0200 Subject: Update documentation --- docs/en/cowboy/2.0/guide/cookies.asciidoc | 153 ++++++++----------- docs/en/cowboy/2.0/guide/cookies/index.html | 135 +++++++---------- docs/en/cowboy/2.0/guide/multipart.asciidoc | 190 ++++++++++++------------ docs/en/cowboy/2.0/guide/multipart/index.html | 188 +++++++++++------------ docs/en/cowboy/2.0/guide/ws_protocol.asciidoc | 45 ++++-- docs/en/cowboy/2.0/guide/ws_protocol/index.html | 45 ++++-- 6 files changed, 382 insertions(+), 374 deletions(-) (limited to 'docs') diff --git a/docs/en/cowboy/2.0/guide/cookies.asciidoc b/docs/en/cowboy/2.0/guide/cookies.asciidoc index 6068db37..58bd1d10 100644 --- a/docs/en/cowboy/2.0/guide/cookies.asciidoc +++ b/docs/en/cowboy/2.0/guide/cookies.asciidoc @@ -4,112 +4,94 @@ Cookies are a mechanism allowing applications to maintain state on top of the stateless HTTP protocol. -Cowboy provides facilities for handling cookies. It is highly -recommended to use them instead of writing your own, as the -implementation of cookies can vary greatly between clients. +Cookies are a name/value store where the names and values are +stored in plain text. They expire either after a delay +or when the browser closes. They can be configured on a +specific domain name or path, and restricted to secure +resources (sent or downloaded over HTTPS), or restricted +to the server (disallowing access from client-side scripts). + +Cookie names are de facto case sensitive. Cookies are stored client-side and sent with every subsequent request that matches the domain and path for which they were -stored, including requests for static files. For this reason -they can incur a cost which must be taken in consideration. - -Also consider that, regardless of the options used, cookies -are not to be trusted. They may be read and modified by any -program on the user's computer, but also by proxies. You -should always validate cookie values before using them. Do -not store any sensitive information in cookies either. - -When explicitly setting the domain, the cookie will be sent -for the domain and all subdomains from that domain. Otherwise -the current domain will be used. The same is true for the -path. - -When the server sets cookies, they will only be available -for requests that are sent after the client receives the -response. - -Cookies are sent in HTTP headers, therefore they must have -text values. It is your responsibility to encode any other -data type. Also note that cookie names are de facto case -sensitive. - -Cookies can be set for the client session (which generally -means until the browser is closed), or it can be set for -a number of seconds. Once it expires, or when the server -says the cookie must exist for up to 0 seconds, the cookie -is deleted by the client. To avoid this while the user -is browsing your site, you should set the cookie for -every request, essentially resetting the expiration time. - -Cookies can be restricted to secure channels. This typically -means that such a cookie will only be sent over HTTPS, -and that it will only be available by client-side scripts -that run from HTTPS webpages. - -Finally, cookies can be restricted to HTTP and HTTPS requests, -essentially disabling their access from client-side scripts. +stored, until they expire. This can create a non-negligible +cost. + +Cookies should not be considered secure. They are stored on +the user's computer in plain text, and can be read by any +program. They can also be read by proxies when using clear +connections. Always validate the value before using it, +and never store any sensitive information inside it. + +Cookies set by the server are only available in requests +following the client reception of the response containing +them. + +Cookies may be sent repeatedly. This is often useful to +update the expiration time and avoid losing a cookie. === Setting cookies -By default, cookies you set are defined for the session. +// @todo So I am not particularly happy about set_resp_cookie/4 +// having Opts as a *third* argument, instead of the *last* like +// all other functions that come with an Opts argument. We will +// probably need to change this before 2.0. + +By default cookies are defined for the duration of the session: [source,erlang] SessionID = generate_session_id(), -Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, [], Req). +Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, Req0). -You can also make them expire at a specific point in the -future. +They can also be set for a duration in seconds: [source,erlang] ---- SessionID = generate_session_id(), -Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, [ - {max_age, 3600} -], Req). +Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, + #{max_age => 3600}, Req0). ---- -You can delete cookies that have already been set. The value -is ignored. +To delete cookies, set `max_age` to 0: [source,erlang] ---- -Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, <<>>, [ - {max_age, 0} -], Req). +SessionID = generate_session_id(), +Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, + #{max_age => 0}, Req0). ---- -You can restrict them to a specific domain and path. -For example, the following cookie will be set for the domain -`my.example.org` and all its subdomains, but only on the path -`/account` and all its subdirectories. +To restrict cookies to a specific domain and path, the options +of the same name can be used: [source,erlang] ---- -Req2 = cowboy_req:set_resp_cookie(<<"inaccount">>, <<"1">>, [ - {domain, "my.example.org"}, - {path, "/account"} -], Req). +Req = cowboy_req:set_resp_cookie(<<"inaccount">>, <<"1">>, + #{domain => "my.example.org", path => "/account"}, Req0). ---- -You can restrict the cookie to secure channels, typically HTTPS. +Cookies will be sent with requests to this domain and all +its subdomains, and to resources on this path or deeper +in the path hierarchy. + +To restrict cookies to secure channels (typically resources +available over HTTPS): [source,erlang] ---- SessionID = generate_session_id(), -Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, [ - {secure, true} -], Req). +Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, + #{secure => true}, Req0). ---- -You can restrict the cookie to client-server communication -only. Such a cookie will not be available to client-side scripts. +To prevent client-side scripts from accessing a cookie: [source,erlang] ---- SessionID = generate_session_id(), -Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, [ - {http_only, true} -], Req). +Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, + #{http_only => true}, Req0). ---- Cookies may also be set client-side, for example using @@ -117,23 +99,23 @@ Javascript. === Reading cookies -As we said, the client sends cookies with every request. -But unlike the server, the client only sends the cookie -name and value. +The client only ever sends back the cookie name and value. +All other options that can be set are never sent back. -Cowboy provides two different ways to read cookies. You -can either parse them as a list of key/value pairs, or -match them into a map, optionally applying constraints -to the values or providing a default if they are missing. +Cowboy provides two functions for reading cookies. Both +involve parsing the cookie header(s) and so should not +be called repeatedly. -You can parse the cookies and then use standard library -functions to access individual values. +You can get all cookies as a key/value list: [source,erlang] Cookies = cowboy_req:parse_cookies(Req), {_, Lang} = lists:keyfind(<<"lang">>, 1, Cookies). -You can match the cookies into a map. +Or you can perform a match against cookies and retrieve +only the ones you need, while at the same time doing +any required post processing using xref:constraints[constraints]. +This function returns a map: [source,erlang] #{id := ID, lang := Lang} = cowboy_req:match_cookies([id, lang], Req). @@ -141,8 +123,7 @@ You can match the cookies into a map. You can use constraints to validate the values while matching them. The following snippet will crash if the `id` cookie is not an integer number or if the `lang` cookie is empty. Additionally -the `id` cookie value will be converted to an integer term, saving -you a conversion step. +the `id` cookie value will be converted to an integer term: [source,erlang] CookiesMap = cowboy_req:match_cookies([{id, int}, {lang, nonempty}], Req). @@ -150,14 +131,12 @@ CookiesMap = cowboy_req:match_cookies([{id, int}, {lang, nonempty}], Req). Note that if two cookies share the same name, then the map value will be a list of the two cookie values. -Read more about xref:constraints[constraints]. - A default value can be provided. The default will be used if the `lang` cookie is not found. It will not be used if -the cookie is found but has an empty value. +the cookie is found but has an empty value: [source,erlang] #{lang := Lang} = cowboy_req:match_cookies([{lang, [], <<"en-US">>}], Req). -If no default is provided and the value is missing, the -query string is deemed invalid and the process will crash. +If no default is provided and the value is missing, an +exception is thrown. diff --git a/docs/en/cowboy/2.0/guide/cookies/index.html b/docs/en/cowboy/2.0/guide/cookies/index.html index e4c982cc..0c957e61 100644 --- a/docs/en/cowboy/2.0/guide/cookies/index.html +++ b/docs/en/cowboy/2.0/guide/cookies/index.html @@ -71,108 +71,87 @@

Cookies are a mechanism allowing applications to maintain state on top of the stateless HTTP protocol.

-

Cowboy provides facilities for handling cookies. It is highly -recommended to use them instead of writing your own, as the -implementation of cookies can vary greatly between clients.

+

Cookies are a name/value store where the names and values are +stored in plain text. They expire either after a delay +or when the browser closes. They can be configured on a +specific domain name or path, and restricted to secure +resources (sent or downloaded over HTTPS), or restricted +to the server (disallowing access from client-side scripts).

+

Cookie names are de facto case sensitive.

Cookies are stored client-side and sent with every subsequent request that matches the domain and path for which they were -stored, including requests for static files. For this reason -they can incur a cost which must be taken in consideration.

-

Also consider that, regardless of the options used, cookies -are not to be trusted. They may be read and modified by any -program on the user’s computer, but also by proxies. You -should always validate cookie values before using them. Do -not store any sensitive information in cookies either.

-

When explicitly setting the domain, the cookie will be sent -for the domain and all subdomains from that domain. Otherwise -the current domain will be used. The same is true for the -path.

-

When the server sets cookies, they will only be available -for requests that are sent after the client receives the -response.

-

Cookies are sent in HTTP headers, therefore they must have -text values. It is your responsibility to encode any other -data type. Also note that cookie names are de facto case -sensitive.

-

Cookies can be set for the client session (which generally -means until the browser is closed), or it can be set for -a number of seconds. Once it expires, or when the server -says the cookie must exist for up to 0 seconds, the cookie -is deleted by the client. To avoid this while the user -is browsing your site, you should set the cookie for -every request, essentially resetting the expiration time.

-

Cookies can be restricted to secure channels. This typically -means that such a cookie will only be sent over HTTPS, -and that it will only be available by client-side scripts -that run from HTTPS webpages.

-

Finally, cookies can be restricted to HTTP and HTTPS requests, -essentially disabling their access from client-side scripts.

+stored, until they expire. This can create a non-negligible +cost.

+

Cookies should not be considered secure. They are stored on +the user’s computer in plain text, and can be read by any +program. They can also be read by proxies when using clear +connections. Always validate the value before using it, +and never store any sensitive information inside it.

+

Cookies set by the server are only available in requests +following the client reception of the response containing +them.

+

Cookies may be sent repeatedly. This is often useful to +update the expiration time and avoid losing a cookie.

Setting cookies

-

By default, cookies you set are defined for the session.

+

By default cookies are defined for the duration of the session:

SessionID = generate_session_id(),
-Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, [], Req).
-

You can also make them expire at a specific point in the -future.

+Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, Req0).
+

They can also be set for a duration in seconds:

SessionID = generate_session_id(),
-Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, [
-    {max_age, 3600}
-], Req).
-

You can delete cookies that have already been set. The value -is ignored.

+Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, + #{max_age => 3600}, Req0). +

To delete cookies, set max_age to 0:

-
Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, <<>>, [
-    {max_age, 0}
-], Req).
-

You can restrict them to a specific domain and path. -For example, the following cookie will be set for the domain -my.example.org and all its subdomains, but only on the path -/account and all its subdirectories.

+
SessionID = generate_session_id(),
+Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID,
+        #{max_age => 0}, Req0).
+

To restrict cookies to a specific domain and path, the options +of the same name can be used:

-
Req2 = cowboy_req:set_resp_cookie(<<"inaccount">>, <<"1">>, [
-    {domain, "my.example.org"},
-    {path, "/account"}
-], Req).
-

You can restrict the cookie to secure channels, typically HTTPS.

+
Req = cowboy_req:set_resp_cookie(<<"inaccount">>, <<"1">>,
+        #{domain => "my.example.org", path => "/account"}, Req0).
+

Cookies will be sent with requests to this domain and all +its subdomains, and to resources on this path or deeper +in the path hierarchy.

+

To restrict cookies to secure channels (typically resources +available over HTTPS):

SessionID = generate_session_id(),
-Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, [
-    {secure, true}
-], Req).
-

You can restrict the cookie to client-server communication -only. Such a cookie will not be available to client-side scripts.

+Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, + #{secure => true}, Req0). +

To prevent client-side scripts from accessing a cookie:

SessionID = generate_session_id(),
-Req2 = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, [
-    {http_only, true}
-], Req).
+Req = cowboy_req:set_resp_cookie(<<"sessionid">>, SessionID, + #{http_only => true}, Req0).

Cookies may also be set client-side, for example using Javascript.

@@ -180,15 +159,12 @@ Javascript.

Reading cookies

-

As we said, the client sends cookies with every request. -But unlike the server, the client only sends the cookie -name and value.

-

Cowboy provides two different ways to read cookies. You -can either parse them as a list of key/value pairs, or -match them into a map, optionally applying constraints -to the values or providing a default if they are missing.

-

You can parse the cookies and then use standard library -functions to access individual values.

+

The client only ever sends back the cookie name and value. +All other options that can be set are never sent back.

+

Cowboy provides two functions for reading cookies. Both +involve parsing the cookie header(s) and so should not +be called repeatedly.

+

You can get all cookies as a key/value list:

Cookies = cowboy_req:parse_cookies(Req),
 {_, Lang} = lists:keyfind(<<"lang">>, 1, Cookies).
-

You can match the cookies into a map.

+

Or you can perform a match against cookies and retrieve +only the ones you need, while at the same time doing +any required post processing using constraints. +This function returns a map:

You can use constraints to validate the values while matching them. The following snippet will crash if the id cookie is not an integer number or if the lang cookie is empty. Additionally -the id cookie value will be converted to an integer term, saving -you a conversion step.

+the id cookie value will be converted to an integer term:

CookiesMap = cowboy_req:match_cookies([{id, int}, {lang, nonempty}], Req).

Note that if two cookies share the same name, then the map value will be a list of the two cookie values.

-

Read more about constraints.

A default value can be provided. The default will be used if the lang cookie is not found. It will not be used if -the cookie is found but has an empty value.

+the cookie is found but has an empty value:

#{lang := Lang} = cowboy_req:match_cookies([{lang, [], <<"en-US">>}], Req).
-

If no default is provided and the value is missing, the -query string is deemed invalid and the process will crash.

+

If no default is provided and the value is missing, an +exception is thrown.

diff --git a/docs/en/cowboy/2.0/guide/multipart.asciidoc b/docs/en/cowboy/2.0/guide/multipart.asciidoc index 20d53d51..630b2107 100644 --- a/docs/en/cowboy/2.0/guide/multipart.asciidoc +++ b/docs/en/cowboy/2.0/guide/multipart.asciidoc @@ -2,36 +2,21 @@ == Multipart requests Multipart originates from MIME, an Internet standard that -extends the format of emails. Multipart messages are a -container for parts of any content-type. +extends the format of emails. -For example, a multipart message may have a part -containing text and a second part containing an -image. This is what allows you to attach files -to emails. +A multipart message is a list of parts. A part contains +headers and a body. The body of the parts may be +of any media type, and contain text or binary data. +It is possible for parts to contain a multipart media +type. In the context of HTTP, multipart is most often used -with the `multipart/form-data` content-type. This is -the content-type you have to use when you want browsers -to be allowed to upload files through HTML forms. +with the `multipart/form-data` media type. It is what +browsers use to upload files through HTML forms. -Multipart is of course not required for uploading -files, it is only required when you want to do so -through HTML forms. - -You can read and parse multipart messages using the -Req object directly. - -Cowboy defines two functions that allows you to get -information about each part and read their contents. - -=== Structure - -A multipart message is a list of parts. Parts may -contain either a multipart message or a non-multipart -content-type. This allows parts to be arranged in a -tree structure, although this is a rare case as far -as the Web is concerned. +The `multipart/byteranges` is also common. It is the +media type used to send arbitrary bytes from a resource, +enabling clients to resume downloads. === Form-data @@ -42,29 +27,24 @@ values and is therefore not fit for uploading files. That's where the `multipart/form-data` content-type comes in. When the form is configured to use this -content-type, the browser will use one part of the -message for each form field. This means that a file -input field will be sent in its own part, but the -same applies to all other kinds of fields. +content-type, the browser will create a multipart +message where each part corresponds to a field on +the form. For files, it also adds some metadata in +the part headers, like the file name. A form with a text input, a file input and a select choice box will result in a multipart message with three parts, one for each field. -The browser does its best to determine the content-type +The browser does its best to determine the media type of the files it sends this way, but you should not rely on it for determining the contents of the file. Proper investigation of the contents is recommended. -=== Checking the content-type +=== Checking for multipart messages -While there is a variety of multipart messages, the -most common on the Web is `multipart/form-data`. It's -the type of message being sent when an HTML form -allows uploading files. - -You can quickly figure out if a multipart message -has been sent by parsing the `content-type` header. +The content-type header indicates the presence of +a multipart message: [source,erlang] ---- @@ -74,96 +54,116 @@ has been sent by parsing the `content-type` header. === Reading a multipart message -To read a message you have to iterate over all its -parts. Then, for each part, you can inspect its headers -and read its body. +Cowboy provides two sets of functions for reading +request bodies as multipart messages. + +The `cowboy_req:read_part/1,2` functions return the +next part's headers, if any. + +The `cowboy_req:read_part_body/1,2` functions return +the current part's body. For large bodies you may +need to call the function multiple times. + +To read a multipart message you need to iterate over +all its parts: [source,erlang] ---- -multipart(Req) -> - case cowboy_req:part(Req) of - {ok, _Headers, Req2} -> - {ok, _Body, Req3} = cowboy_req:part_body(Req2), - multipart(Req3); - {done, Req2} -> - Req2 +multipart(Req0) -> + case cowboy_req:read_part(Req0) of + {ok, _Headers, Req1} -> + {ok, _Body, Req} = cowboy_req:read_part_body(Req1), + multipart(Req); + {done, Req} -> + Req end. ---- -Parts do not have a size limit. When a part body is -too big, Cowboy will return what it read so far and -allow you to continue if you wish to do so. +When part bodies are too large, Cowboy will return +a `more` tuple, and allow you to loop until the part +body has been fully read. The function `cow_multipart:form_data/1` can be used to quickly obtain information about a part from a -`multipart/form-data` message. This function will -tell you if the part is for a normal field or if it -is a file being uploaded. +`multipart/form-data` message. The function returns +a `data` or a `file` tuple depending on whether this +is a normal field or a file being uploaded. -This can be used for example to allow large part bodies -for files but crash when a normal field is too large. +The following snippet will use this function and +use different strategies depending on whether the +part is a file: [source,erlang] ---- -multipart(Req) -> - case cowboy_req:part(Req) of - {ok, Headers, Req2} -> - Req4 = case cow_multipart:form_data(Headers) of +multipart(Req0) -> + case cowboy_req:read_part(Req0) of + {ok, Headers, Req1} -> + Req = case cow_multipart:form_data(Headers) of {data, _FieldName} -> - {ok, _Body, Req3} = cowboy_req:part_body(Req2), - Req3; + {ok, _Body, Req2} = cowboy_req:read_part_body(Req1), + Req2; {file, _FieldName, _Filename, _CType, _CTransferEncoding} -> - stream_file(Req2) + stream_file(Req1) end, - multipart(Req4); - {done, Req2} -> - Req2 + multipart(Req); + {done, Req} -> + Req end. -stream_file(Req) -> - case cowboy_req:part_body(Req) of - {ok, _Body, Req2} -> - Req2; - {more, _Body, Req2} -> - stream_file(Req2) +stream_file(Req0) -> + case cowboy_req:read_part_body(Req0) of + {ok, _Body, Req} -> + Req; + {more, _Body, Req} -> + stream_file(Req) end. ---- -By default the body chunk Cowboy will return is limited -to 8MB. This can of course be overriden. Both functions -can take a second argument, the same list of options that -will be passed to `cowboy_req:body/2` function. +Both the part header and body reading functions can take +options that will be given to the request body reading +functions. By default, `cowboy_req:read_part/1` reads +up to 64KB for up to 5 seconds. `cowboy_req:read_part_body/1` +has the same defaults as `cowboy_req:read_body/1`. + +To change the defaults for part headers: + +[source,erlang] +cowboy_req:read_part(Req, #{length => 128000}). + +And for part bodies: + +[source,erlang] +cowboy_req:read_part_body(Req, #{length => 1000000, period => 7000}). === Skipping unwanted parts -If you do not want to read a part's body, you can skip it. -Skipping is easy. If you do not call the function to read -the part's body, Cowboy will automatically skip it when -you request the next part. +Part bodies do not have to be read. Cowboy will automatically +skip it when you request the next part's body. The following snippet reads all part headers and skips all bodies: [source,erlang] ---- -multipart(Req) -> - case cowboy_req:part(Req) of - {ok, _Headers, Req2} -> - multipart(Req2); - {done, Req2} -> - Req2 +multipart(Req0) -> + case cowboy_req:part(Req0) of + {ok, _Headers, Req} -> + multipart(Req); + {done, Req} -> + Req end. ---- Similarly, if you start reading the body and it ends up -being too big, you can simply continue with the next part, +being too big, you can simply continue with the next part. Cowboy will automatically skip what remains. -Note that the skipping rate may not be adequate for your -application. If you observe poor performance when skipping, -you might want to consider manually skipping by calling -the `cowboy_req:part_body/1` function directly. +While Cowboy can skip part bodies automatically, the read +rate is not configurable. Depending on your application +you may want to skip manually, in particular if you observe +poor performance while skipping. + +You do not have to read all parts either. You can stop +reading as soon as you find the data you need. -And if you started reading the message but decide that you -do not need the remaining parts, you can simply stop reading -entirely and Cowboy will automatically figure out what to do. +// @todo Cover the building of multipart messages. diff --git a/docs/en/cowboy/2.0/guide/multipart/index.html b/docs/en/cowboy/2.0/guide/multipart/index.html index dba83e90..5f661d44 100644 --- a/docs/en/cowboy/2.0/guide/multipart/index.html +++ b/docs/en/cowboy/2.0/guide/multipart/index.html @@ -70,33 +70,18 @@

Multipart requests

Multipart originates from MIME, an Internet standard that -extends the format of emails. Multipart messages are a -container for parts of any content-type.

-

For example, a multipart message may have a part -containing text and a second part containing an -image. This is what allows you to attach files -to emails.

+extends the format of emails.

+

A multipart message is a list of parts. A part contains +headers and a body. The body of the parts may be +of any media type, and contain text or binary data. +It is possible for parts to contain a multipart media +type.

In the context of HTTP, multipart is most often used -with the multipart/form-data content-type. This is -the content-type you have to use when you want browsers -to be allowed to upload files through HTML forms.

-

Multipart is of course not required for uploading -files, it is only required when you want to do so -through HTML forms.

-

You can read and parse multipart messages using the -Req object directly.

-

Cowboy defines two functions that allows you to get -information about each part and read their contents.

-
-

Structure

-
-

A multipart message is a list of parts. Parts may -contain either a multipart message or a non-multipart -content-type. This allows parts to be arranged in a -tree structure, although this is a rare case as far -as the Web is concerned.

-
-
+with the multipart/form-data media type. It is what +browsers use to upload files through HTML forms.

+

The multipart/byteranges is also common. It is the +media type used to send arbitrary bytes from a resource, +enabling clients to resume downloads.

Form-data

@@ -106,28 +91,24 @@ content-type. This type is just a list of keys and values and is therefore not fit for uploading files.

That’s where the multipart/form-data content-type comes in. When the form is configured to use this -content-type, the browser will use one part of the -message for each form field. This means that a file -input field will be sent in its own part, but the -same applies to all other kinds of fields.

+content-type, the browser will create a multipart +message where each part corresponds to a field on +the form. For files, it also adds some metadata in +the part headers, like the file name.

A form with a text input, a file input and a select choice box will result in a multipart message with three parts, one for each field.

-

The browser does its best to determine the content-type +

The browser does its best to determine the media type of the files it sends this way, but you should not rely on it for determining the contents of the file. Proper investigation of the contents is recommended.

-

Checking the content-type

+

Checking for multipart messages

-

While there is a variety of multipart messages, the -most common on the Web is multipart/form-data. It’s -the type of message being sent when an HTML form -allows uploading files.

-

You can quickly figure out if a multipart message -has been sent by parsing the content-type header.

+

The content-type header indicates the presence of +a multipart message:

Reading a multipart message

-

To read a message you have to iterate over all its -parts. Then, for each part, you can inspect its headers -and read its body.

+

Cowboy provides two sets of functions for reading +request bodies as multipart messages.

+

The cowboy_req:read_part/1,2 functions return the +next part’s headers, if any.

+

The cowboy_req:read_part_body/1,2 functions return +the current part’s body. For large bodies you may +need to call the function multiple times.

+

To read a multipart message you need to iterate over +all its parts:

-
multipart(Req) ->
-    case cowboy_req:part(Req) of
-        {ok, _Headers, Req2} ->
-            {ok, _Body, Req3} = cowboy_req:part_body(Req2),
-            multipart(Req3);
-        {done, Req2} ->
-            Req2
+
multipart(Req0) ->
+    case cowboy_req:read_part(Req0) of
+        {ok, _Headers, Req1} ->
+            {ok, _Body, Req} = cowboy_req:read_part_body(Req1),
+            multipart(Req);
+        {done, Req} ->
+            Req
     end.
-

Parts do not have a size limit. When a part body is -too big, Cowboy will return what it read so far and -allow you to continue if you wish to do so.

+

When part bodies are too large, Cowboy will return +a more tuple, and allow you to loop until the part +body has been fully read.

The function cow_multipart:form_data/1 can be used to quickly obtain information about a part from a -multipart/form-data message. This function will -tell you if the part is for a normal field or if it -is a file being uploaded.

-

This can be used for example to allow large part bodies -for files but crash when a normal field is too large.

+multipart/form-data message. The function returns +a data or a file tuple depending on whether this +is a normal field or a file being uploaded.

+

The following snippet will use this function and +use different strategies depending on whether the +part is a file:

-
multipart(Req) ->
-    case cowboy_req:part(Req) of
-        {ok, Headers, Req2} ->
-            Req4 = case cow_multipart:form_data(Headers) of
+
multipart(Req0) ->
+    case cowboy_req:read_part(Req0) of
+        {ok, Headers, Req1} ->
+            Req = case cow_multipart:form_data(Headers) of
                 {data, _FieldName} ->
-                    {ok, _Body, Req3} = cowboy_req:part_body(Req2),
-                    Req3;
+                    {ok, _Body, Req2} = cowboy_req:read_part_body(Req1),
+                    Req2;
                 {file, _FieldName, _Filename, _CType, _CTransferEncoding} ->
-                    stream_file(Req2)
+                    stream_file(Req1)
             end,
-            multipart(Req4);
-        {done, Req2} ->
-            Req2
+            multipart(Req);
+        {done, Req} ->
+            Req
     end.
 
-stream_file(Req) ->
-    case cowboy_req:part_body(Req) of
-        {ok, _Body, Req2} ->
-            Req2;
-        {more, _Body, Req2} ->
-            stream_file(Req2)
+stream_file(Req0) ->
+    case cowboy_req:read_part_body(Req0) of
+        {ok, _Body, Req} ->
+            Req;
+        {more, _Body, Req} ->
+            stream_file(Req)
     end.
-

By default the body chunk Cowboy will return is limited -to 8MB. This can of course be overriden. Both functions -can take a second argument, the same list of options that -will be passed to cowboy_req:body/2 function.

+

Both the part header and body reading functions can take +options that will be given to the request body reading +functions. By default, cowboy_req:read_part/1 reads +up to 64KB for up to 5 seconds. cowboy_req:read_part_body/1 +has the same defaults as cowboy_req:read_body/1.

+

To change the defaults for part headers:

+
+
+
cowboy_req:read_part(Req, #{length => 128000}).
+

And for part bodies:

+
+
+
cowboy_req:read_part_body(Req, #{length => 1000000, period => 7000}).

Skipping unwanted parts

-

If you do not want to read a part’s body, you can skip it. -Skipping is easy. If you do not call the function to read -the part’s body, Cowboy will automatically skip it when -you request the next part.

+

Part bodies do not have to be read. Cowboy will automatically +skip it when you request the next part’s body.

The following snippet reads all part headers and skips all bodies:

@@ -213,23 +214,22 @@ all bodies:

by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> -
multipart(Req) ->
-    case cowboy_req:part(Req) of
-        {ok, _Headers, Req2} ->
-            multipart(Req2);
-        {done, Req2} ->
-            Req2
+
multipart(Req0) ->
+    case cowboy_req:part(Req0) of
+        {ok, _Headers, Req} ->
+            multipart(Req);
+        {done, Req} ->
+            Req
     end.

Similarly, if you start reading the body and it ends up -being too big, you can simply continue with the next part, +being too big, you can simply continue with the next part. Cowboy will automatically skip what remains.

-

Note that the skipping rate may not be adequate for your -application. If you observe poor performance when skipping, -you might want to consider manually skipping by calling -the cowboy_req:part_body/1 function directly.

-

And if you started reading the message but decide that you -do not need the remaining parts, you can simply stop reading -entirely and Cowboy will automatically figure out what to do.

+

While Cowboy can skip part bodies automatically, the read +rate is not configurable. Depending on your application +you may want to skip manually, in particular if you observe +poor performance while skipping.

+

You do not have to read all parts either. You can stop +reading as soon as you find the data you need.

diff --git a/docs/en/cowboy/2.0/guide/ws_protocol.asciidoc b/docs/en/cowboy/2.0/guide/ws_protocol.asciidoc index 67b2cdf2..127c829c 100644 --- a/docs/en/cowboy/2.0/guide/ws_protocol.asciidoc +++ b/docs/en/cowboy/2.0/guide/ws_protocol.asciidoc @@ -11,18 +11,43 @@ connections between the client, typically a Web browser, and the server. It uses the HTTP Upgrade mechanism to establish the connection. -Websocket connections are asynchronous, unlike HTTP. This -means that not only can the client send frames to the server -at any time, but the server can also send frames to the client -without the client initiating anything other than the -Websocket connection itself. This allows the server to push -data to the client directly. +Websocket connections are fully asynchronous, unlike +HTTP/1.1 (synchronous) and HTTP/2 (asynchronous, but the +server can only initiate streams in response to requests). +With Websocket, the client and the server can both send +frames at any time without any restriction. It is closer +to TCP than any of the HTTP protocols. Websocket is an IETF standard. Cowboy supports the standard and all drafts that were previously implemented by browsers, excluding the initial flawed draft sometimes known as "version 0". +=== Websocket vs HTTP/2 + +For a few years Websocket was the only way to have a +bidirectional asynchronous connection with the server. +This changed when HTTP/2 was introduced. While HTTP/2 +requires the client to first perform a request before +the server can push data, this is only a minor restriction +as the client can do so just as it connects. + +Websocket was designed as a kind-of-TCP channel to a +server. It only defines the framing and connection +management and lets the developer implement a protocol +on top of it. For example you could implement IRC over +Websocket and use a Javascript IRC client to speak to +the server. + +HTTP/2 on the other hand is just an improvement over +the HTTP/1.1 connection and request/response mechanism. +It has the same semantics as HTTP/1.1. + +If all you need is to access an HTTP API, then HTTP/2 +should be your first choice. On the other hand, if what +you need is a different protocol, then you can use +Websocket to implement it. + === Implementation Cowboy implements Websocket as a protocol upgrade. Once the @@ -37,7 +62,9 @@ covering all aspects of the protocol. Cowboy passes the suite with 100% success, including all optional tests. Cowboy's Websocket implementation also includes the -x-webkit-deflate-frame compression draft which is being used -by some browsers to reduce the size of data being transmitted. +permessage-deflate and x-webkit-deflate-frame compression +extensions. + Cowboy will automatically use compression as long as the -`compress` protocol option is set when starting the listener. +`websocket_compress` protocol option is set when starting +the listener. diff --git a/docs/en/cowboy/2.0/guide/ws_protocol/index.html b/docs/en/cowboy/2.0/guide/ws_protocol/index.html index 70a3adfe..5a640c49 100644 --- a/docs/en/cowboy/2.0/guide/ws_protocol/index.html +++ b/docs/en/cowboy/2.0/guide/ws_protocol/index.html @@ -78,12 +78,12 @@ a vital component of soft realtime Web applications.

connections between the client, typically a Web browser, and the server. It uses the HTTP Upgrade mechanism to establish the connection.

-

Websocket connections are asynchronous, unlike HTTP. This -means that not only can the client send frames to the server -at any time, but the server can also send frames to the client -without the client initiating anything other than the -Websocket connection itself. This allows the server to push -data to the client directly.

+

Websocket connections are fully asynchronous, unlike +HTTP/1.1 (synchronous) and HTTP/2 (asynchronous, but the +server can only initiate streams in response to requests). +With Websocket, the client and the server can both send +frames at any time without any restriction. It is closer +to TCP than any of the HTTP protocols.

Websocket is an IETF standard. Cowboy supports the standard and all drafts that were previously implemented by browsers, excluding the initial flawed draft sometimes known as @@ -91,6 +91,30 @@ excluding the initial flawed draft sometimes known as

+

Websocket vs HTTP/2

+
+

For a few years Websocket was the only way to have a +bidirectional asynchronous connection with the server. +This changed when HTTP/2 was introduced. While HTTP/2 +requires the client to first perform a request before +the server can push data, this is only a minor restriction +as the client can do so just as it connects.

+

Websocket was designed as a kind-of-TCP channel to a +server. It only defines the framing and connection +management and lets the developer implement a protocol +on top of it. For example you could implement IRC over +Websocket and use a Javascript IRC client to speak to +the server.

+

HTTP/2 on the other hand is just an improvement over +the HTTP/1.1 connection and request/response mechanism. +It has the same semantics as HTTP/1.1.

+

If all you need is to access an HTTP API, then HTTP/2 +should be your first choice. On the other hand, if what +you need is a different protocol, then you can use +Websocket to implement it.

+
+
+

Implementation

Cowboy implements Websocket as a protocol upgrade. Once the @@ -103,10 +127,11 @@ the Autobahn test suite, which is an extensive suite of tests covering all aspects of the protocol. Cowboy passes the suite with 100% success, including all optional tests.

Cowboy’s Websocket implementation also includes the -x-webkit-deflate-frame compression draft which is being used -by some browsers to reduce the size of data being transmitted. -Cowboy will automatically use compression as long as the -compress protocol option is set when starting the listener.

+permessage-deflate and x-webkit-deflate-frame compression +extensions.

+

Cowboy will automatically use compression as long as the +websocket_compress protocol option is set when starting +the listener.

-- cgit v1.2.3