Inets Release Notes

Inets 6.0

Fixed Bugs and Malfunctions

Fix race condition in httpc. If the socket is closed by the peer do not try to close it again.

Own Id: OTP-11845

Avoid process leak by gracefully terminating httpc request handler process when send operation fails.

Own Id: OTP-12362

Reject messages with a Content-Length less than 0

Own Id: OTP-12739 Aux Id: seq12860

Let gen_tcp:controlling_process/2 and inet_sctp:connect/[45] propagate prim_inet:setopt/3 errors instead of having them generate badmatch exceptions.

Own Id: OTP-12798

Improvements and New Features

Remove Server Side Include support from inets, as this is an old technic that has security issues and was not well tested.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12156

New value in server_tokens config for limiting banner grabbing attempts.

By setting {server_tokens, none} in ServiceConfig for inets:start(httpd, ServiceConfig), the "Server:" header will not be set in messages from the server.

Own Id: OTP-12661 Aux Id: seq12840

To enable the HTTP server to run in a virtualized environment, where there can be more that one server that has the same ip-address and port, we add a new option profile.

Own Id: OTP-12674

httpc: Fix implementation of gracefull shudown to work as intended for keep alive connections not using pipelining.

Own Id: OTP-12803

Correct handling of proxy options when using persistent connections.

Own Id: OTP-12822

Inets 5.10.3

Fixed Bugs and Malfunctions

Fix some spelling mistakes in documentation

Own Id: OTP-12152

Improvements and New Features

httpd: Seperate timeout for TLS/SSL handshake from keepalive timeout

Own Id: OTP-12013

Warning: this is experimental and may disappear or change without previous warning.

Experimental support for running Quickcheck and PropEr tests from common_test suites is added to common_test. See the reference manual for the new module ct_property_testing.

Experimental property tests are added under lib/{inet,ssh}/test/property_test. They can be run directly or from the commont_test suites inet/ftp_property_test_SUITE.erl and ssh/test/ssh_property_test_SUITE.erl.

See the code in the test directories and the man page for details.

(Thanks to Tuncer Ayaz for a patch adding Triq)

Own Id: OTP-12119

Inets 5.10

Fixed Bugs and Malfunctions

Fixed a spelling mistake in httpc doc (Thanks to Wasif Riaz Malik)

Own Id: OTP-11538

Application upgrade (appup) files are corrected for the following applications:

asn1, common_test, compiler, crypto, debugger, dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, inets, observer, odbc, os_mon, otp_mibs, parsetools, percept, public_key, reltool, runtime_tools, ssh, syntax_tools, test_server, tools, typer, webtool, wx, xmerl

A new test utility for testing appup files is added to test_server. This is now used by most applications in OTP.

(Thanks to Tobias Schlager)

Own Id: OTP-11744

ftp now sanitize file name, user name and passwords from <CR> and <LF> tags (Thanks to Sergei Golovan)

Own Id: OTP-11750

Corrected error handling in the HTTP client, making it behave more graceful.

Thanks to Kirilll Zaborsky

Own Id: OTP-11794

Support identity transfer-encoding in httpc.

Thanks to Anthony Ramine

Own Id: OTP-11802

Ignore empty Set-Cookie headers to increase interoperability with servers that violate the RFC.

Thanks to Kirilll Zaborsky

Own Id: OTP-11803

Improvements and New Features

The commit 6189bc07 "inets: httpc improve pipelining" has been reverted, as it turned out to break things rather than improve pipelining utilization. It is instead up to the user to configure httpc and use it wisely to be able to get the most out of pipelining.

Own Id: OTP-11756

Handle all response codes in httpd_util:message/3

Own Id: OTP-11838

Inets 5.9.7

Fixed Bugs and Malfunctions

Fix httpd config option 'script_timeout' and fixed httpd config option 'keep_alive_timeout'. Thanks to Johannes Weissl.

Own Id: OTP-11276

Make httpc:request_cancel/[1,2] asynchronous. Previously these functions tried to guarantee request answer would not reach the client, which only worked for some of the use cases. Now these functions are totally asynchronous which makes it the clients responsibility to disregard possible answers to canceled requests.

Also pipelining implementation has been changed to improve the utilization factor. Further investigation of possible enhancements in this area are planned for later.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11312

[httpd] Add handling of new response for mod_head (otherwise causing case_clause crash). Also updated logging: Removed logging for keep-alive connections timeout (this is a normal occurrence and not an error) and some access-log body size corrections.

Own Id: OTP-11328

Improvements and New Features

The ftp client now supports ftp over tls (ftps).

Own Id: OTP-11037

Inets 5.9.6

Improvements and New Features

httpc: Allow content body in DELETE requests. Thanks to James Wheare.

Own Id: OTP-11190

Add missing brackets to report formatting on ftp_progress process exit. Thanks to Artur Wilniewczyc.

Own Id: OTP-11202

Fix some errors in the inets documentation. Thanks to Johannes Weissl.

Own Id: OTP-11210

Fix various typos in httpd, inets. Thanks to Tomohiko Aono.

Own Id: OTP-11226

Fix httpd config option 'erl_script_nocache'. Thanks to Johannes Weissl.

Own Id: OTP-11260

Inets 5.9.5

Fixed Bugs and Malfunctions

Reverted incorrect commit that broke cookie handling when using httpc-profiles.

Own Id: OTP-10956

Improvements and New Features

Fix http_request:http_headers/1 to send content-length when length is zero. Thanks to CA Meijer.

Own Id: OTP-10934

Integrate elliptic curve contribution from Andreas Schultz

In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto.

This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API.

Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments.

Own Id: OTP-11009

Fix {stream, {self, once}} in httpc to work as expected. Thanks to Masatake Daimon

Own Id: OTP-11122

Inets 5.9.4

Improvements and New Features

httpd: The modules option now defaults to the documented value.

Own Id: OTP-10844

httpc: Fixed persistent connection implementation that was broken by a patch to R13. The patch made persisten connections behaved the same way as pipelining.

Own Id: OTP-10845

httpd: Simplified configuration of ssl in httpd, this also enables all ssl options to be configured. The old and limited way is no longer documented but will be supported for backwards comatibility for some time.

Own Id: OTP-10846

Handle correctly the "No files found or file unavailable" error code. Thanks to Serge Aleynikov

Own Id: OTP-10886

Inets 5.9.1

Improvements and New Features

Better handling of errorI(s) during update of the session database.

Also added and updated some debugging functions which_sessions/[0,1] and info/0.

Own Id: OTP-10093

Aux Id: Seq 12062

Removed R14B compatible version of (inets-service and tftp) behaviour definition.

Own Id: OTP-10095

[httpc] Documentation of KeepAlive and Pipeline timeout options have been improved.

Own Id: OTP-10114

Fixed Bugs and Malfunctions

[httpc] Cancel request does not work due to incorrect handler table creation (wrong keypos).

Vyacheslav Vorobyov

Own Id: OTP-10092

Incompatibilities

Inets 5.9

Improvements and New Features

[httpd] Make the server header configurable with new config option server_tokens. The value of the server header, which was previously hard-coded (at compile time), is now possible to manipulate through the means of the server_tokens config option.

Own Id: OTP-9805

Improve inets support for inets as an included application.

inets_app calls supervisor:start_link/3 directly rather than calling the root supervisor function inets_sup:start_link/0. This precludes using included_applications to start inets without having a wrapper function.

Jay Nelson

Own Id: OTP-9960

[httpc] Add function for retrieving current options, get_options/[1,2].

Own Id: OTP-9979

Utility module http_uri now officially supported.

Also, the parse function has been extended with more scheme support and a way to provide your own scheme info.

Own Id: OTP-9983

Aux Id: Seq 12022

Fixed Bugs and Malfunctions

Inets 5.8.1

Improvements and New Features

Fixed Bugs and Malfunctions

[ftp] Fails to open IPv6 connection due to badly formatted IPv6 address in EPRT command. The address part of the command incorrectly contained decimal elements instead of hexadecimal.

Own Id: OTP-9827

Aux Id: Seq 11970

[httpc] Bad Keep Alive Mode. When selecting a session, the "state" of the session (specifically if the server has responded) was not taken into account.

Own Id: OTP-9847

[httpc] The client incorrectly streams 404 responses. The documentation specifies that only 200 and 206 responses shall be streamed.

Shane Evens

Own Id: OTP-9860

Inets 5.8

Improvements and New Features

[ftpc] Add a config option to specify a data connect timeout. That is how long the ftp client will wait for the server to connect to the data socket. If this timeout occurs, an error will be returned to the caller and the ftp client process will be terminated.

Own Id: OTP-9545

[httpc] Wrong Host header in IPv6 HTTP requests. When a URI with a IPv6 host is parsed, the brackets that encapsulates the address part is removed. This value is then supplied as the host header. This can cause problems with some servers. A workaround for this is to use headers_as_is and provide the host header with the requst call. To solve this a new option has been added, ipv6_host_with_brackets. This option specifies if the host value of the host header shall include the brackets or not. By default, it does not (as before).

Own Id: OTP-9628

Fixed Bugs and Malfunctions

[httpd] Fix logging of content length in mod_log.

Garrett Smith

Own Id: OTP-9715

[httpd] Sometimes entries in the transfer log was written with the message size as list of numbers. This list was actually the size as a string, e.g. "123", written with the control sequence ~w. This has now been corrected so that any string is converted to an integer (if possible).

Own Id: OTP-9733

Fixed various problems detected by Dialyzer.

Own Id: OTP-9736

Incompatibilities

[httpc] Deprecated interface module http has been removed. It has (long) been replaced by http client interface module httpc.

Own Id: OTP-9359

[httpc|httpd] The old ssl implementation (based on OpenSSL), has been deprecated. The config option that specified usage of this version of the ssl app, ossl, has been removed.

Own Id: OTP-9522

Inets 5.7.1

Improvements and New Features

Fixed Bugs and Malfunctions

[httpc] Parsing of a cookie expire date should be more forgiving. That is, if the parsing fails, the date should be ignored. Also added support for (yet another) date format: "Tue Jan 01 08:00:01 2036 GMT".

Own Id: OTP-9433

[httpc] Rewrote cookie parsing. Among other things solving cookie processing from www.expedia.com.

Own Id: OTP-9434

[httpd] Fix httpd directory traversal on Windows. Directory traversal was possible on Windows where backward slash is used as directory separator.

András Veres-Szentkirályi.

Own Id: OTP-9561

Inets 5.7

Improvements and New Features

[httpc|httpd] Added support for IPv6 with ssl.

Own Id: OTP-5566

Fixed Bugs and Malfunctions

[httpc] Remove unnecessary usage of iolist_to_binary when processing body (for PUT and POST).

Filipe David Manana

Own Id: OTP-9317

[ftp] FTP client doesn't work with IPv6 host.

Attila Rajmund Nohl

Own Id: OTP-9342 Aux Id: seq11853

[httpd] Peer/sockname resolv doesn't work with IPv6 addrs in HTTP.

Attila Rajmund Nohl.

Own Id: OTP-9343

[httpc] Clients started stand-alone not properly handled. Also it was not documented how to use them, that is that once started, they are represented by a pid() and not by their profile().

Own Id: OTP-9365

Inets 5.6

Improvements and New Features

[httpc] Add support for upload body streaming (PUT and POST).

For more info, see the definition of the Body argument of the request/[4,5] function.

Filipe David Manana

Own Id: OTP-9094

[ftp] Added (type) spec for all exported functions.

Own Id: OTP-9114 Aux Id: seq11799

[httpd] mod_esi:deliver/2 made to accept binary data.

Bernard Duggan

Own Id: OTP-9123

[httpd] Prevent XSS in error pages. Prevent user controlled input from being interpreted as HTML in error pages by encoding the reserved HTML characters.

Michael Santos

Own Id: OTP-9124

[httpd] Improved error messages.

Ricardo Catalinas Jiménez

Own Id: OTP-9157

[httpd] Extended support for file descriptors. In order to be able to bind to a privileged port without running the erlang VM as root, the support for using file descriptors has been improved. It is now possible to add the file descriptor to the config (option fd) when calling the inets:start(httpd, ...) function.

Attila Rajmund Nohl

Own Id: OTP-9202

Aux Id: seq11819

The default ssl kind has now been changed to essl.

ossl will work for as long as the ssl application supports it.

See the httpd socket_type communication property or the httpc request/[4,5] function for more info.

Own Id: OTP-9230

*** POTENTIAL INCOMPATIBILITY ***

Fixed Bugs and Malfunctions

[httpd] Wrong security property names used in documentation.

security_data_file used instead of data_file.

security_max_retries used instead of max_retries.

security_block_time used instead of block_time.

security_fail_expire_time used instead of fail_expire_time.

security_auth_timeout used instead of auth_timeout.

Garrett Smith

Own Id: OTP-9131

[httpd] Fix timeout message generated by mod_esi. When a mod_esi request times out, the code to send a timeout response was incorrect and generated an internal server error as well as an invalid response line.

Bernard Duggan

Own Id: OTP-9158

[httpc] httpc manager crashes. When a request results in a retry, the request id will be "reused" in the previous implementation a race condition could occur causing the manager to crash.

This is now avoided by using proc_lib:init_ack and gen_server:enter_loop to allow mor advanced initialization of httpc_handlers without blocking the httpc_manger and eliminating extra processes that can cause race conditions.

Own Id: OTP-9246

[httpc] Issuing a request (httpc:request) to an host with the ssl option {ip, {127,0,0,1}} results in an handler crash. The reason was that the connect call resulted in an exit with reason badarg (this was the same for both ssl and gen_tcp).

Exits was not catched. This has now been improved.

Own Id: OTP-9289

Aux Id: seq11845

Inets 5.5.1

Improvements and New Features

Miscellaneous inet6 related problems.

Own Id: OTP-8927

Updated http-server to make sure URLs in error-messages are URL-encoded. Added support in http-client to use URL-encoding. Also added the missing include directory for the inets application.

Own Id: OTP-8940

Aux Id: seq11735

Fixed Bugs and Malfunctions

Fix format_man_pages so it handles all man sections and remove warnings/errors in various man pages.

Own Id: OTP-8600

[httpc] Pipelined and queued requests not processed when connection closed remotelly.

Own Id: OTP-8906

Inets 5.5

Fixed Bugs and Malfunctions

[httpc] If a request times out (not connect timeout), the handler process exited (normal) but neglected to inform the manager process. For this reason, the manager did not clean up the request table., resulting in a memory leak. Also the manager did not create a monitor for the handler, so in an unforseen handler crash, this could also create a memory leak.

Own Id: OTP-8739

The service tftp was spelled wrong in documentation and in some parts of the code. It should be tftp.

Own Id: OTP-8741 Aux Id: seq11635

[httpc] Replaced the old http client api module (http) with the new, httpc in the users guide.

Own Id: OTP-8742

Improvements and New Features

Eliminated warnings for auto-imported BIF clashes.

Own Id: OTP-8840

Inets 5.4

Improvements and New Features

[httpc|httpd] - Now allow the use of the "new" ssl, by using the essl tag instead.

See the http_option option in the request/[4,5] or the socket-type section of the Communication properties chapter for more info,

Own Id: OTP-7907

Deprecated functions designated to be removed in R14 has been removed. Also, some new functions has been marked as deprecated (the old http client api module).

Own Id: OTP-8564

*** POTENTIAL INCOMPATIBILITY ***

[httpd] - Improved mod_alias. Now able to do better URL rewrites.

See URL aliasing properties and the CGI properties section(s) for more info,

Own Id: OTP-8573

Fixed Bugs and Malfunctions

Inets 5.3.1

Improvements and New Features

Fixed Bugs and Malfunctions

[httpc] - Badly formated error reason for errors occuring during initial connect to a server. Also, the possible error reasons was not properly documented.

Own Id: OTP-8508

Aux Id: seq11407

[httpd] - Issues with ESI erl_script_timeout.

The erl_script_timeout config option is ducumented as a number of seconds. But when parsing the config, in the new format (not a config file), it was handled as if in number of milliseconds.

When the erl-script-timeout time was exceeded, the server incorrectly marked the answer as sent, thereby leaving client hanging (with an incomplete answer). This has been changed, so that now the socket will be closed.

Own Id: OTP-8509

Inets 5.3

Improvements and New Features

[httpc] - Allow users to pass socket options to the transport module when making requests.

See the socket_opts option in the request/4 or set_options/[1,2] for more info,

Own Id: OTP-8352

[httpc] Fix bug crafting Host header when port is not 80.

The host header should include the port number as well as the host name when making a request to a server listening on a port other than the HTTP default of 80. Currently, only the host name is included. This is important to make the http client more compliant with the HTTP specification.

Own Id: OTP-8371

Kelly McLaughlin

[httpc|httpd] http_chunk data handling/passing improvement.

This is a modification to the http_chunk module to forward any full chunk received, regardless of whether the size field for the following chunk has been received yet. This allows http_chunk to be used in situations where a long term HTTP connection is used to send periodic status updates as individual chunks. Previously a given chunk would not be forwarded to the client process until the size for the next chunk had been read which rendered the module difficult to use for the scenario described.

Bernard Duggan

Own Id: OTP-8351

Include the inets test suite in the release of the application.

Own Id: OTP-8349

[httpc] - It is now possible to configure the client to deliver an async reply to more receivers then the calling process.

See the receiver option for more info,

Own Id: OTP-8106

[httpd] - Methods "PUT" and "DELETE" now allowed.

huntermorris@gmail.com

Own Id: OTP-8103

[httpc] Several more or less critical fixes:

Initial call between the httpc manager and request handler was synchronous.

When the manager starts a new request handler, this is no longer a synchronous operation. Previously, the new request handler made the connection to the server and issuing of the first request (the reason for starting it) in the gen_server init function. If the connection for some reason "took some time", the manager hanged, leaving all other activities by that manager also hanging.

As a side-effect of these changes, some modules was also renamed, and a new api module, httpc, has been introduced (the old module http is not removed, but is now just wrapper for httpc).

Own Id: OTP-8016

*** POTENTIAL INCOMPATIBILITY ***

Fixed Bugs and Malfunctions

[httpd] The server did not fully support the documented module callback api. Specifically, the load function should be able to return the atom ok, but this was not accepted.

Own Id: OTP-8359

Fixing various documentation-related bugs (bad quotes).

Own Id: OTP-8327

Fixing minor Dialyzer and copyright problem(s).

Own Id: OTP-8315

[httpc] - Added basic sanity check of option value combinations.

adam.kocoloski@gmail.com

Own Id: OTP-8056

Inets 5.2

Improvements and New Features

[ftpc] - Start of the FTP client has been changed in the following way:

It is now also possible to start a standalone FTP client process using the re-introduced ftp:open function.

This is an alternative to starting the client using the inets service framework.

The old ftp:open/1, undocumented, function, caused the client to be hooken into the inets service supervision framework. This is no longer the case.

*** POTENTIAL INCOMPATIBILITY ***

Previously, the FTP client attempted to use IPv6, unless otherwise instructed (the ip_v6_disabled flag), and only used IPv4 if this did not work. This has now been changed.

A new option, ipfamily, has been introduced, with the default value inet (IPv4).

See ftp:open for more info.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8258

The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.

Own Id: OTP-8249

Fixed Bugs and Malfunctions

[httpc] - Streaming to file did not work.

dizzyd@gmail.com

Own Id: OTP-8204

[ftpc] - The ls/2 function (LIST command) and the nlist/2 function (NLST command) with wildcards did not work properly.

These functions is documented as working on directories, but this is actually not according the standard. The LIST and NLST commands are specified to operate on a directory or other group of files, or a file.

Previously, an attempt was made to check if the listing returned by the server was actually an error message. This was done by changing remote directory (cd) into the (assumed) "directory". This may work if Pathname was actually a directory, but as this is not always the case, this test does not work. Instead, we now return the actual server result and leave the interpretation to the caller.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8247

Aux Id: seq11407

[httpc] - Fixes various bugs in timeout and keep-alive queue handling.

When a queued request times, out the error mssage is sent the owner of the active request.

Requests in the keep-alive queue is forgotten when handler terminates.

Timeout out requests are retried.

Jean-Sébastien Pédron

Own Id: OTP-8248

[httpd] - Unnecessarily strict matching when handling closing sockets.

Own Id: OTP-8280

Inets 5.1.2

Improvements and New Features

[httpc] - Added http option connect_timeout for http client request. The connect_timeout option is used for the initial request, when the client connects to the server. Default value is that of the timeout option.

See the request/[4,5] function for more info.

Own Id: OTP-7298

Fixed Bugs and Malfunctions

[httpd] - Failed to create listen socket with invalid option combo. The http-server failed to create its listen socket when the bind-address was an IPv4-address (a tuple of size 4) and the ipfamily option was inet6fb4.

Own Id: OTP-8118

Aux Id: seq11321

[httpd] - Removed documentation for non-existing function (httpd_util:header/2,3,4).

Own Id: OTP-8101

Inets 5.1.1

Improvements and New Features

[httpd] - When starting inets (the web-server) and supplying a descriptor on the command line (example: erl -httpd_8888 <descriptor>) it is now possible to specify which ip-family to use: inet | inet6 | inet6fb4.

Example: erl -httpd_8888 10|inet6

When starting the web-server either using a file with property list (the proplist_file) or a an property list, using the ipfamily option: {ipfamily, inet | inet6 | inet6fb4}.

Finally, when starting the web-server using the classical apache-style config file, the BindAddress directive has been augmented to allow the specification of the IpFamily: BindAddress blirk.ericsson.se|inet

Default is inet6fb4 which emulates the behaviour of the previous version.

See the Communication properties section for more info.

Own Id: OTP-8069

Aux Id: seq11086

Fixed Bugs and Malfunctions

[httpc] - Reception of unexpected data causes handler crash.

Own Id: OTP-8052

Inets 5.1

Improvements and New Features

[httpc] Added support for web services using only basic auth, with a token as the user part and no password part.

twoggle@gmail.com

Own Id: OTP-7998

[httpc] - Bind HTTP client to IP-addr. It is now possible to specify an alternate ip-address and port to be used when the client connects to the server.

As a side-effect of this, the option ipv6 has been removed and replaced by the ipfamily option.

See http:set_options/[1,2] for more info.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8004

Fixed Bugs and Malfunctions

Updated guard tests (i.e. is_list(L) instead of list(L) and possibly andalso/orelse instead of ","/";").

Own Id: OTP-7994

[httpc] - Remove use of the deprecated regexp module.

Own Id: OTP-8001

[httpc] - The option max_keep_alive_length was not handled properly.

Own Id: OTP-8005