[httpc|httpd] Added support for IPv6 with ssl.

Own Id: OTP-5566

[httpc] Clients started stand-alone not properly handled. Also it was not documented how to use them, that is that once started, they are represented by a pid() and not by their profile().

Own Id: OTP-9365

[httpc] Add support for upload body streaming (PUT and POST).

For more info, see the definition of the Body argument of the request/4,5 function.

Filipe David Manana

Own Id: OTP-9094

[ftp] Added (type) spec for all exported functions.

Own Id: OTP-9114 Aux Id: seq11799

[httpd] mod_esi:deliver/2 made to accept binary data.

Bernard Duggan

Own Id: OTP-9123

[httpd] Prevent XSS in error pages. Prevent user controlled input from being interpreted as HTML in error pages by encoding the reserved HTML characters.

Michael Santos

Own Id: OTP-9124

[httpd] Improved error messages.

Ricardo Catalinas Jiménez

Own Id: OTP-9157

[httpd] Extended support for file descriptors. In order to be able to bind to a privileged port without running the erlang VM as root, the support for using file descriptors has been improved. It is now possible to add the file descriptor to the config (option fd) when calling the inets:start(httpd, ...) function.

Attila Rajmund Nohl

Own Id: OTP-9202

Aux Id: seq11819

The default ssl kind has now been changed to essl.

ossl will work for as long as the ssl application supports it.

See the httpd socket_type communication property or the httpc request/4,5 function for more info.

Own Id: OTP-9230

*** POTENTIAL INCOMPATIBILITY ***

[httpd] Wrong security property names used in documentation.

security_data_file used instead of data_file.

security_max_retries used instead of max_retries.

security_block_time used instead of block_time.

security_fail_expire_time used instead of fail_expire_time.

security_auth_timeout used instead of auth_timeout.

Garrett Smith

Own Id: OTP-9131

[httpd] Fix timeout message generated by mod_esi. When a mod_esi request times out, the code to send a timeout response was incorrect and generated an internal server error as well as an invalid response line.

Bernard Duggan

Own Id: OTP-9158

[httpc] httpc manager crashes. When a request results in a retry, the request id will be "reused" in the previous implementation a race condition could occur causing the manager to crash.

This is now avoided by using proc_lib:init_ack and gen_server:enter_loop to allow mor advanced initialization of httpc_handlers without blocking the httpc_manger and eliminating extra processes that can cause race conditions.

Own Id: OTP-9246

[httpc] Issuing a request (httpc:request) to an host with the ssl option {ip, {127,0,0,1}} results in an handler crash. The reason was that the connect call resulted in an exit with reason badarg (this was the same for both ssl and gen_tcp).

Exits was not catched. This has now been improved.

Own Id: OTP-9289

Aux Id: seq11845

[httpd] httpd_response:send_chunk handles empty list and empty binary - i.e. no chunk is sent, but it does not handle a list with an empty binary [<<>>]. This will be sent as an empty chunk - which in turn will be encoded by http_chunk to the same as a final chunk, which will make the http client believe that the end of the page is reached.

Own Id: OTP-8906

Miscellaneous inet6 related problems.

Own Id: OTP-8927

Updated http-server to make sure URLs in error-messages are URL-encoded. Added support in http-client to use URL-encoding. Also added the missing include directory for the inets application.

Own Id: OTP-8940

Aux Id: seq11735

Fix format_man_pages so it handles all man sections and remove warnings/errors in various man pages.

Own Id: OTP-8600

[httpc] Pipelined and queued requests not processed when connection closed remotelly.

Own Id: OTP-8906

[httpc] If a request times out (not connect timeout), the handler process exited (normal) but neglected to inform the manager process. For this reason, the manager did not clean up the request table., resulting in a memory leak. Also the manager did not create a monitor for the handler, so in an unforseen handler crash, this could also create a memory leak.

Own Id: OTP-8739

The service tftp was spelled wrong in documentation and in some parts of the code. It should be tftp.

Own Id: OTP-8741 Aux Id: seq11635

[httpc] Replaced the old http client api module (http) with the new, httpc in the users guide.

Own Id: OTP-8742

Eliminated warnings for auto-imported BIF clashes.

Own Id: OTP-8840

[httpc|httpd] - Now allow the use of the "new" ssl, by using the essl tag instead.

See the http_option option in the request/4,5 or the socket-type section of the Communication properties chapter for more info,

Own Id: OTP-7907

Deprecated functions designated to be removed in R14 has been removed. Also, some new functions has been marked as deprecated (the old http client api module).

Own Id: OTP-8564

*** POTENTIAL INCOMPATIBILITY ***

[httpd] - Improved mod_alias. Now able to do better URL rewrites.

See URL aliasing properties and the CGI properties section(s) for more info,

Own Id: OTP-8573

[httpc] - Made cookie handling more case insensitive.

Own Id: OTP-8609

Nicolas Thauvin

[httpc|httpd] - Netscape cookie dates can also be given with a 2-digit year (e.g. 06 = 2006).

Own Id: OTP-8610

Nicolas Thauvin

[httpd] - Added support (again) for the documented debugging features. See the User's Guide Configuration chapter for more info.

Own Id: OTP-8624

[httpc] - Memory leak plugged. The profile manager never cleaned up in its handler database. This meant that with each new request handler, another entry was created that was never deleted. Eventually the request id counter (used as a key) would wrap, but the machine would most likely run out of memory before that happened.

Own Id: OTP-8542

Lev Walkin

[httpc] - https requests with default port (443) not handled properly.

Own Id: OTP-8607

jebu ittiachen

[httpc] - Badly formated error reason for errors occuring during initial connect to a server. Also, the possible error reasons was not properly documented.

Own Id: OTP-8508

Aux Id: seq11407

[httpd] - Issues with ESI erl_script_timeout.

The erl_script_timeout config option is ducumented as a number of seconds. But when parsing the config, in the new format (not a config file), it was handled as if in number of milliseconds.

When the erl-script-timeout time was exceeded, the server incorrectly marked the answer as sent, thereby leaving client hanging (with an incomplete answer). This has been changed, so that now the socket will be closed.

[httpc] - Allow users to pass socket options to the transport module when making requests.

See the socket_opts option in the request/4 or set_options/1,2 for more info,

Own Id: OTP-8352

[httpc] Fix bug crafting Host header when port is not 80.

The host header should include the port number as well as the host name when making a request to a server listening on a port other than the HTTP default of 80. Currently, only the host name is included. This is important to make the http client more compliant with the HTTP specification.

Own Id: OTP-8371

Kelly McLaughlin

[httpc|httpd] http_chunk data handling/passing improvement.

This is a modification to the http_chunk module to forward any full chunk received, regardless of whether the size field for the following chunk has been received yet. This allows http_chunk to be used in situations where a long term HTTP connection is used to send periodic status updates as individual chunks. Previously a given chunk would not be forwarded to the client process until the size for the next chunk had been read which rendered the module difficult to use for the scenario described.

Bernard Duggan

Own Id: OTP-8351

Include the inets test suite in the release of the application.

Own Id: OTP-8349

[httpc] - It is now possible to configure the client to deliver an async reply to more receivers then the calling process.

See the receiver option for more info,

Own Id: OTP-8106

[httpd] - Methods "PUT" and "DELETE" now allowed.

huntermorris@gmail.com

Own Id: OTP-8103

[httpc] Several more or less critical fixes:

Initial call between the httpc manager and request handler was synchronous.

When the manager starts a new request handler, this is no longer a synchronous operation. Previously, the new request handler made the connection to the server and issuing of the first request (the reason for starting it) in the gen_server init function. If the connection for some reason "took some time", the manager hanged, leaving all other activities by that manager also hanging.

[httpd] The server did not fully support the documented module callback api. Specifically, the load function should be able to return the atom ok, but this was not accepted.

Own Id: OTP-8359

Fixing various documentation-related bugs (bad quotes).

Own Id: OTP-8327

Fixing minor Dialyzer and copyright problem(s).

Own Id: OTP-8315

[httpc] - Added basic sanity check of option value combinations.

adam.kocoloski@gmail.com

Own Id: OTP-8056

[ftpc] - Start of the FTP client has been changed in the following way:

Own Id: OTP-8258

The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.

Own Id: OTP-8249

[httpc] - Streaming to file did not work.

dizzyd@gmail.com

Own Id: OTP-8204

[ftpc] - The ls/2 function (LIST command) and the nlist/2 function (NLST command) with wildcards did not work properly.

These functions is documented as working on directories, but this is actually not according the standard. The LIST and NLST commands are specified to operate on a directory or other group of files, or a file.

Previously, an attempt was made to check if the listing returned by the server was actually an error message. This was done by changing remote directory (cd) into the (assumed) "directory". This may work if Pathname was actually a directory, but as this is not always the case, this test does not work. Instead, we now return the actual server result and leave the interpretation to the caller.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8247

Aux Id: seq11407

[httpc] - Fixes various bugs in timeout and keep-alive queue handling.

Jean-Sébastien Pédron

Own Id: OTP-8248

[httpd] - Unnecessarily strict matching when handling closing sockets.

Own Id: OTP-8280

[httpc] - Raise condition. When http:request is called and httpc_manager selects a session where there's already a pending request, then the connection handler for that session effectively resets its parser, readying it for the response to the second request. But if there are still some inbound packets for the response to the first request, things get confused.

tomas.abrahamsson@gmail.com

Own Id: OTP-8154

[httpc] - Added http option connect_timeout for http client request. The connect_timeout option is used for the initial request, when the client connects to the server. Default value is that of the timeout option.

See the request/4,5 function for more info.

Own Id: OTP-7298

[httpd] - Failed to create listen socket with invalid option combo. The http-server failed to create its listen socket when the bind-address was an IPv4-address (a tuple of size 4) and the ipfamily option was inet6fb4.

Own Id: OTP-8118

Aux Id: seq11321

[httpd] - Removed documentation for non-existing function (httpd_util:header/2,3,4).

Own Id: OTP-8101

[httpd] - When starting inets (the web-server) and supplying a descriptor on the command line (example: erl -httpd_8888 <descriptor>) it is now possible to specify which ip-family to use: inet | inet6 | inet6fb4.

Example: erl -httpd_8888 10|inet6

When starting the web-server either using a file with property list (the proplist_file) or a an property list, using the ipfamily option: {ipfamily, inet | inet6 | inet6fb4}.

Finally, when starting the web-server using the classical apache-style config file, the BindAddress directive has been augmented to allow the specification of the IpFamily: BindAddress blirk.ericsson.se|inet

Default is inet6fb4 which emulates the behaviour of the previous version.

See the Communication properties section for more info.

Own Id: OTP-8069

Aux Id: seq11086

[httpc] - Reception of unexpected data causes handler crash.

Own Id: OTP-8052

[httpc] Added support for web services using only basic auth, with a token as the user part and no password part.

twoggle@gmail.com

Own Id: OTP-7998

[httpc] - Bind HTTP client to IP-addr. It is now possible to specify an alternate ip-address and port to be used when the client connects to the server.

As a side-effect of this, the option ipv6 has been removed and replaced by the ipfamily option.

See http:set_options/1,2 for more info.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8004

Updated guard tests (i.e. is_list(L) instead of list(L) and possibly andalso/orelse instead of ","/";").

Own Id: OTP-7994

[httpc] - Remove use of the deprecated regexp module.

Own Id: OTP-8001

[httpc] - The option max_keep_alive_length was not handled properly.

Own Id: OTP-8005

[tftp] The callback watchdog has been removed, as it turned out to be counter productive when the disk was overloaded. Earlier a connection was aborted when a callback (which performs the file access in the TFTP server) took too long time.

[tftp] The error message "Too many connections" has been reclassified to be a warning.

Own Id: OTP-7888

[httpc] - Incorrect http version option check.

Mats Cronqvist

Own Id: OTP-7882

[httpc] - Unnecessary error report when client terminating as a result of the server closed the socket unexpectedly.

Own Id: OTP-7883

[httpc] - Failed transforming a relative URI to an absolute URI.

mtalyans@cisco.com

Own Id: OTP-7950

[httpd] - The HTTP server did not handle the config option ssl_ca_certificate_file.

karol.skocik@gmail.com

Own Id: OTP-7976

Ssl did not work correctly with the use of new style configuration due to sn old internal format that was not changed correctly in all places.

Own Id: OTP-7723 Aux Id: seq11143

[httpc] - Now streams 200 and 206 results and not only 200 results.

Own Id: OTP-7857

[httpc] - The inets http client will now use persistent connections without pipelining as default and if a pipeline timeout is set it will pipeline the requests on the persistent connections.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7463

[httpd] - added option ssl_password_callback_arguments.

Own Id: OTP-7724 Aux Id: seq11151

Changed the socket use so that it will become more robust to non-functional ipv6 and fallback on ipv4. This changes may for very special os-configurations cause a problem when used with erts-versions pre R13.

Own Id: OTP-7726

Removed deprecated function httpd_util:key1search/[2,3]

Own Id: OTP-7815

[httpd] - Updated inets so that it not uses the deprecated function ssl:accept/[2,3].

Own Id: OTP-7636 Aux Id: seq11086

Transient bug related to hot code swap of the TFTP server is now fixed. It could happen that the first TFTP server that was started after a code upgrade to Inets-5.0.6 crashed with a function clause error in tftp_engine:service_init/2.

Own Id: OTP-7574 Aux Id: seq11069

[httpd] - Validation of ssl_password_callback_module was incorrect.

Own Id: OTP-7597 Aux Id: seq11074

[httpd] - Misspelling in old apachelike configuration directive TransferDiskLogSize has been corrected.

Own Id: OTP-7598 Aux Id: seq11059

Minor problems found by dialyzer has been fixed.

Own Id: OTP-7605

Enhanched an info report.

Own Id: OTP-7450

Changed errro message from {wrong_type,{document_root,"/tmp/htdocs"}} to {invalid_option,{non_existing, document_root,"/tmp/htdocs"}}.

Own Id: OTP-7454

Relative paths in directory authentication did not work as intended, this has now been fixed.

Own Id: OTP-7490

The query-string passed to the callback function was not compliant with the documentation, it is now.

Own Id: OTP-7512

Parameters to error_logger:error_report/1 has been corrected.

Own Id: OTP-7257 Aux Id: OTP-7294, OTP-7258

[httpd] - If a Module/Function request matching an erl_script_alias registration does not exist as a function in the module registered a 404 error will now be issued instead of a 500 error.

Own Id: OTP-7323

[httpd] -The option auth_type for mod_auth is no longer mandatory, for backward-compatibility reasons.

Own Id: OTP-7341

[httpd] - Spelling error caused client connection header to be ignored.

Own Id: OTP-7315 Aux Id: seq10951

[httpd] - Call to the function mod_get:get_modification_date/1 was made too early resulting in that httpd did not send the 404 file missing response.

Own Id: OTP-7321

[httpc, httpd] - Now follows the recommendation regarding line terminators in section 19.3 in RFC 2616 e.i: "The line terminator for message-header fields is the sequence CRLF. However, we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore the leading CR".

Own Id: OTP-7304 Aux Id: seq10944

[tftp] If a callback (which performs the file access in the TFTP server) takes too long time (more than the double TFTP timeout), the server will abort the connection and send an error reply to the client. This implies that the server will release resources attached to the connection faster than before. The server simply assumes that the client has given up.

[tftp] If the TFTP server receives yet another request from the same client (same host and port) while it already has an active connection to the client, it will simply ignore the new request if the request is equal with the first one (same filename and options). This implies that the (new) client will be served by the already ongoing connection on the server side. By not setting up yet another connection, in parallel with the ongoing one, the server will consumer lesser resources.

[tftp] netascii mode is now supported when the client/server has native ascii support (Windows). The new optional parameter native_ascii in the tftp_binary and tftp_file callback modules can be used to override the default behavior.

[tftp] Yet another callback module has been added in order to allow customized handling of error, warning and info messages. See the new configuration parameter, logger.

[tftp] Yet another configuration parameter, max_retries, has been added in order to control the number of times a packet can be resent. The default is 5.

[tftp] tftp:info/1 and tftp:change_config/2 can now be applied to all daemons or all servers in one command without bothering about their process identifiers.

External TR HI89527.

Own Id: OTP-7266

[tftp] Blocks with too low block numbers are silently discarded. For example if a server receives block #5 when it expects block #7 it will discard the block without interrupting the file transfer. Too high block numbers does still imply an error. External TR HI96072.

Own Id: OTP-7220

[tftp] The problem with occasional case_clause errors in tftp_engine:common_read/7 has been fixed. External TR HI97362.

Own Id: OTP-7221

Changed calls to file open to concur with the API and not use deprecated syntax.

Own Id: OTP-7172

[tftp] Server lost the first packet when the client timed out

Own Id: OTP-7173

Updated copyright headers and fixed backwards compatibility for an undocumented feature, for now. This feature will later be removed and a new and documented option will take its place.

Own Id: OTP-7144

[httpd] - Error logs now has a pretty and a compact format and access logs can be written on the common log format or the extended common log format.

Own Id: OTP-6661 Aux Id: Seq 7764

[httpc] - Added acceptance of missing reason phrase to the relaxed mode.

Own Id: OTP-7024

[httpc] - A new option has been added to enable the client to act as lower version clients, by default the client is an HTTP/1.1 client.

Own Id: OTP-7043

[httpd] - Deprecated function httpd:start/1 did not accept all inputs that it had done previously. This should now work again.

Own Id: OTP-7040

[httpd] - Changed validity check on bind_address so that it uses inet:getaddr instead of inet:gethostbyaddr as the former puts a too hard restriction on the bind_address.

Own Id: OTP-7041 Aux Id: seq10829

[httpc] - Internal process now does try-catch and terminates normally in case of HTTP parse errors. Semantical the client works just as before returning an error message to the client, even if the error massage has been enhanced, but there is no supervisor report in the shell of a internal process crashing. (Which was the expected behavior and not a fault.)

Own Id: OTP-7042

[httpd, httpc] - Deprecated base64 decode/encode functions have been removed. Inets uses base64 in STDLIB instead.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6485

[httpd] - It is now possible to restrict the length of acceptable URI:s in the HTTP server.

Own Id: OTP-6572

[httpc] - Profiles are now supported i.e. the options available in set_options/1 can be set locally for a certain profile and do not have to affect all HTTP-requests issued in the Erlang node. Calls to the HTTP client API functions not using the profile argument will use the default profile.

Own Id: OTP-6690

A new uniform Inets interface provides a flexible way to start/stop Inets services and get information about running services. See inets(3). This also means that inflexibilities in the HTTP server has been removed and more default values has been added.

Own Id: OTP-6705

[tftp] Logged errors have been changed to be logged warnings.

Own Id: OTP-6916 Aux Id: seq10737

[httpc] - The client will now return the proper value when receiving a HTTP 204 code instead of hanging.

Own Id: OTP-6982

The Inets application now has to be explicitly started and stopped i.e. it will not automatically be started as a temporary application as it did before. Although a practical feature when testing things in the shell it is not desirable that people take advantage of this and not start the Inets application in a correct way in their products. Added functions to the Inets API that call application:start/stop.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6993