From 5d01c70ca399edf28e99dc760506329689fab6ba Mon Sep 17 00:00:00 2001 From: Ingela Anderton Andin Date: Thu, 1 Jun 2017 16:09:11 +0200 Subject: inets: httpd - Add chunk handling of client data If the client uses PUT or POST to send a HTTP body the server currently will gather the whole body in memory before passing it on to the mod-callback. For legacy reasons it also converts the binary body to a list, this is really bad for memory usage! Add new option max_client_body_chunk to enable chunked handling of content-length requests. Also make it possible for chunked-encoded data to be propagated in portions to the mod-callback. Conflicts: lib/inets/test/httpd_SUITE.erl --- lib/inets/doc/src/httpd.xml | 13 +++++++- lib/inets/doc/src/mod_esi.xml | 69 +++++++++++++++++++++++++++++-------------- 2 files changed, 59 insertions(+), 23 deletions(-) (limited to 'lib/inets/doc') diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml index d74635fc01..edf8731a82 100644 --- a/lib/inets/doc/src/httpd.xml +++ b/lib/inets/doc/src/httpd.xml @@ -279,7 +279,18 @@ requests defined by max_keep_alive_requests, the server closes the connection. The server closes it even if there are queued request. Default is no limit.

- + + + + {max_client_body_chunk, integer()} + +

Enforces chunking of a HTTP PUT or POST body data to be deliverd + to the mod_esi callback. Note this is not supported for mod_cgi. + Default is no limit e.i the whole body is deliverd as one entity, which could + be very memory consuming. mod_esi(3). +

+ + diff --git a/lib/inets/doc/src/mod_esi.xml b/lib/inets/doc/src/mod_esi.xml index 46cc796c8a..e2f0758cdf 100644 --- a/lib/inets/doc/src/mod_esi.xml +++ b/lib/inets/doc/src/mod_esi.xml @@ -121,35 +121,61 @@ - Module:Function(SessionID, Env, Input)-> _ + Module:Function(SessionID, Env, Input)-> {continue, State} | _ Creates a dynamic web page and returns it chunk by chunk to the server process by calling mod_esi:deliver/2. SessionID = term() Env = env() - Input = string() + Input = string() | chunked_data() + chunked_data() = {first, Data::binary()} | + {continue, Data::binary(), State::term()} | + {last, Data::binary(), State::term()} + State = term()

Module must be found in the code path and export Function with an arity of three. An erlScriptAlias must also be set up in the configuration file for the web server.

-

If the HTTP request is a 'post' request and a body is sent, - content_length is the length of the posted - data. If 'get' is used, query_string is the data after - ? in the URL.

-

ParsedHeader is the HTTP request as a key-value tuple - list. The keys in ParsedHeader are in lower case.

-

SessionID is an identifier - the server uses when deliver/2 is called. Do not - assume anything about the datatype.

-

Use this callback function to generate dynamic web - content dynamically. When a part of the page is generated, send the - data back to the client through deliver/2. Notice - that the first chunk of data sent to the client must at - least contain all HTTP header fields that the response - will generate. If the first chunk does not contain the - end of HTTP header, that is, "\r\n\r\n", - the server assumes that no HTTP header fields will be generated.

+ +

mod_esi:deliver/2 shall be used to generate the response + to the client and SessionID is an identifier that shall by used when + calling this function, do not assume anything about + the datatype. This function may be called + several times to chunk the the respons data. Notice that the + first chunk of data sent to the client must at least contain + all HTTP header fields that the response will generate. If the + first chunk does not contain the end of HTTP header, + that is, "\r\n\r\n", the server assumes that no HTTP + header fields will be generated.

+ +

Env environment data of the request see description above.

+ +

Input is query data of a GET request or the body of + a PUT or POST request. The default behavior (legacy reasons) + for delivering the body, is that the whole body is gathered and + converted to a string. But if the httpd config parameter + max_client_body_chunk + is set, the body will be delivered as binary chunks + instead. The maximum size of the chunks is either max_client_body_chunk + or decide by the client if it uses HTTP chunked encoding + to send the body. When using the chunking + mechanism this callback must return {continue, State::term()} + for all calls where Input is {first, + Data::binary()} or {continue, Data::binary(), + State::term()}. When Input is {last, + Data::binary(), State::term()} the return value will be ignored. +

Note that if the body is + small all data may be delivered in only one chunk and then the + callback will be called with {last, Data::binary(), undefined} + without getting called with {first, + Data::binary()}.

The input State is + the last returned State, in it the callback can include + any data that it needs to keep track of when handling the chunks. +

+

@@ -159,14 +185,13 @@ This function is deprecated and is only kept for backwards compatibility. Env = env() - Input = string() + Input = string() Response = string()

This callback format consumes much memory, as the whole response must be generated before it is sent to the - user. This function is deprecated and is only kept for backwards - compatibility. + user. This callback format is deprecated. For new development, use Module:Function/3.

 -- cgit v1.2.3