OTP-9094: [httpc] Add support for upload body streaming (PUT and POST).

	  Filipe David Manana

OTP-9114: [ftp] Added (type) spec for all exported functions.

OTP-9123: mod_esi:deliver/2 made to accept binary data.
	  Bernard Duggan

OTP-9124: [httpd] Prevent XSS in error pages.
	  Michael Santos

OTP-9131: [httpd] Wrong security property names used in documentation.
	  Garrett Smith

OTP-9157: [httpd] Improved error messages.
	  Ricardo Catalinas Jim�nez

OTP-9158: [httpd] Fix timeout message generated by mod_esi.
	  Bernard Duggan

OTP-9202: [httpd] Extended support for file descriptors.
	  Attila Rajmund Nohl

OTP-9230: The default ssl kind has now been changed to essl.

OTP-9246: [httpc] httpc manager crash because of a handler retry
	  race condition.

Merge branch 'bmk/inets/inet56_integration' into dev

6 files changed, 235 insertions, 72 deletions

diff --git a/lib/inets/doc/src/httpc.xml b/lib/inets/doc/src/httpc.xml
index 8f68087871..f6b6827e93 100644
--- a/lib/inets/doc/src/httpc.xml
+++ b/lib/inets/doc/src/httpc.xml
@@ -76,38 +76,37 @@ socket_opt() = See the Options used by gen_tcp(3) and
     <p>For more information about HTTP see rfc 2616</p>
 
     <code type="none"><![CDATA[
-method()        = head | get | put | post | trace | options | delete
-request()       = {url(), headers()} | 
-                  {url(), headers(), content_type(), body()}
-url()           = string() - Syntax according to the URI definition in rfc 2396, ex: "http://www.erlang.org"
-status_line()   = {http_version(), status_code(), reason_phrase()}
-http_version()  = string() ex: "HTTP/1.1"
-status_code()   = integer()
-reason_phrase() = string()
-content_type()  = string()
-headers()       = [header()]
-header()        = {field(), value()}
-field()         = string() 
-value()         = string() 
-body()          = string() | binary() 
-filename()      = string() 
+method()         = head | get | put | post | trace | options | delete
+request()        = {url(), headers()} | 
+                   {url(), headers(), content_type(), body()}
+url()            = string() - Syntax according to the URI definition in rfc 2396, ex: "http://www.erlang.org"
+status_line()    = {http_version(), status_code(), reason_phrase()}
+http_version()   = string() ex: "HTTP/1.1"
+status_code()    = integer()
+reason_phrase()  = string()
+content_type()   = string()
+headers()        = [header()]
+header()         = {field(), value()}
+field()          = string() 
+value()          = string() 
+body()           = string() | 
+                   binary() |
+                   {fun(accumulator()) -> body_processing_result(), 
+                    accumulator()} |
+                   {chunkify, 
+                    fun(accumulator()) -> body_processing_result(), 
+                    accumulator()}
+body_processing_result() = eof | {ok, iolist(), accumulator()}
+accumulator()    = term()
+filename()       = string() 
     ]]></code>
 
   </section>
   
   <section>
     <title>SSL DATA TYPES </title>
-    <p>Some type definitions relevant when using https,
-      for details <seealso marker="ssl:ssl">ssl(3)</seealso>: </p>
-    <code type="none"><![CDATA[
-ssl_options() = {verify,     code()}   | 
-                {depth,      depth()}  | 
-                {certfile,   path()}   | 
-                {keyfile,    path()}   | 
-                {password,   string()} | 
-                {cacertfile, path()}   | 
-                {ciphers,    string()} 
-    ]]></code>
+    <p>See <seealso marker="ssl:ssl">ssl(3)</seealso> for information
+    about ssl options (<c>ssloptions()</c>). </p>
   </section>
 
   <section>
@@ -142,8 +141,9 @@ ssl_options() = {verify,     code()}   |
       <fsummary>Sends a get HTTP-request</fsummary>
       <type>
         <v>Url = url() </v> 
-	<v>Result = {status_line(), headers(), body()} | 
-                    {status_code(), body()} | request_id() </v>
+	<v>Result = {status_line(), headers(), Body} | 
+                    {status_code(), Body} | request_id() </v>
+	<v>Body = string() | binary()</v>
 	<v>Profile = profile()</v>
 	<v>Reason = term() </v>
       </type>
@@ -166,9 +166,9 @@ ssl_options() = {verify,     code()}   |
         <v>http_options() = [http_option()]</v>
         <v>http_option() = {timeout,         timeout()} | 
                            {connect_timeout, timeout()} | 
-                           {ssl,             ssl_options()} | 
-                           {ossl,            ssl_options()} | 
-                           {essl,            ssl_options()} | 
+                           {ssl,             ssloptions()} | 
+                           {ossl,            ssloptions()} | 
+                           {essl,            ssloptions()} | 
                            {autoredirect,    boolean()} | 
                            {proxy_auth, {userstring(), passwordstring()}} | 
                            {version,         http_version()} | 
@@ -191,8 +191,9 @@ ssl_options() = {verify,     code()}   |
         <v>Function = atom() </v>
         <v>Args = list() </v>
 	<v>body_format() = string | binary </v>
-        <v>Result = {status_line(), headers(), body()} | 
-                    {status_code(), body()} | request_id() </v>
+        <v>Result = {status_line(), headers(), Body} | 
+                    {status_code(), Body} | request_id() </v>
+        <v>Body = string() | binary()</v>
         <v>Profile = profile() </v>
         <v>Reason = {connect_failed, term()} | 
                     {send_failed, term()} | term() </v>
@@ -226,7 +227,7 @@ ssl_options() = {verify,     code()}   |
           <tag><c><![CDATA[ssl]]></c></tag>
           <item>
             <p>This is the default ssl config option, currently defaults to 
-              <c>ossl</c>, see below. </p>
+              <c>essl</c>, see below. </p>
             <p>Defaults to <c>[]</c>. </p>
 	  </item>
 
diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml
index 6470b6fac7..edacb73b65 100644
--- a/lib/inets/doc/src/httpd.xml
+++ b/lib/inets/doc/src/httpd.xml
@@ -4,7 +4,7 @@
 <erlref>
   <header>
     <copyright>
-      <year>1997</year><year>2010</year>
+      <year>1997</year><year>2011</year>
       <holder>Ericsson AB. All Rights Reserved.</holder>
     </copyright>
     <legalnotice>
@@ -154,7 +154,7 @@
           <c>ossl</c> specifically uses the OpenSSL based (old) SSL. 
           <c>essl</c> specifically uses the Erlang based (new) SSL. 
           When using <c>ssl</c> it <em>currently</em> defaults to 
-          <c>ossl</c>. </p>
+          <c>essl</c>. </p>
 	<p>Defaults to <c>ip_comm</c>. </p> 
       </item>
 
@@ -794,14 +794,14 @@ bytes
     <marker id="sdir_prop"></marker>
     <p> Here follows the valid properties for security directories</p>
     <taglist>
-      <tag>{security_data_file, path()}</tag>
+      <tag>{data_file, path()}</tag>
       
       <item>
 	Name of the security data file.  The filename can either
 	absolute or relative to the server_root. This file is used to
 	store persistent data for the mod_security module. </item>
 	  
-      <tag>{security_max_retries, integer()}</tag>
+      <tag>{max_retries, integer()}</tag>
 	  
       <item> Specifies the maximum number of tries to authenticate a
 	user has before the user is blocked out. If a user
@@ -811,13 +811,13 @@ bytes
 	server will return 401 (Unauthorized), for security
 	reasons. Defaults to 3 may also be set to infinity.</item>
 	
-      <tag>{security_block_time, integer()}</tag>
+      <tag>{block_time, integer()}</tag>
       
       <item> Specifies the number of minutes a user is blocked. After
 	this amount of time, he automatically regains access.
 	Defaults to 60</item>
       
-      <tag>{security_fail_expire_time, integer()}</tag>
+      <tag>{fail_expire_time, integer()}</tag>
       
       <item>
 	Specifies the number of minutes a failed user authentication
@@ -825,7 +825,7 @@ bytes
 	time, his previous failed authentications are
 	forgotten. Defaults to 30</item>
       
-      <tag>{security_auth_timeout, integer()}</tag>
+      <tag>{auth_timeout, integer()}</tag>
 
       <item>
 	Specifies the number of seconds a successful user
diff --git a/lib/inets/doc/src/inets.xml b/lib/inets/doc/src/inets.xml
index c367d7fa77..a2bf42320f 100644
--- a/lib/inets/doc/src/inets.xml
+++ b/lib/inets/doc/src/inets.xml
@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="latin1" ?>
+<?xml version="1.0" encoding="iso-8859-1" ?>
 <!DOCTYPE erlref SYSTEM "erlref.dtd">
 
 <erlref>
@@ -32,8 +32,10 @@
   <modulesummary>The inets services API</modulesummary>
   <description>
     <p>This module provides the most basic API to the 
-      clients and servers, that are part of the Inets application,
-      such as start and stop. </p>
+    clients and servers, that are part of the Inets application,
+    such as start and stop. </p>
+
+    <marker id="common_data_types"></marker>
   </description>
 
   <section>
@@ -42,7 +44,10 @@
       this module: </p>
     <p><c> service() = ftpc | tftp | httpc | httpd</c></p>
     <p><c> property() = atom() </c></p>
+    <marker id="functions"></marker>
+    <marker id="services"></marker>
   </section>
+
   <funcs>
     <func>
       <name>services() -> [{Service, Pid}]</name>
@@ -54,11 +59,13 @@
       <desc>
         <p>Returns a list of currently running services.</p>
         <note>
-          <p>Services started as <c>stand_alone</c> will not
-            be listed.</p>
+	  <p>Services started as <c>stand_alone</c> will not be listed.</p>
         </note>
+
+	<marker id="services_info"></marker>
       </desc>
     </func>
+
     <func>
       <name>services_info() -> [{Service, Pid, Info}]</name>
       <fsummary>Returns a list of currently running services where
@@ -73,11 +80,13 @@
       </type>
       <desc>
         <p>Returns a list of currently running services where each
-          service is described by a [{Option, Value}] list. The
-          information given in the list is specific for each service
-          and it is probable that each service will have its own info
-          function that gives you even more details about the
-          service.</p>
+	service is described by a [{Option, Value}] list. The
+	information given in the list is specific for each service
+	and it is probable that each service will have its own info
+	function that gives you even more details about the
+	service.</p>
+
+	<marker id="service_names"></marker>
       </desc>
     </func>
     
@@ -89,6 +98,8 @@
       </type>
       <desc>
         <p>Returns a list of available service names.</p>
+
+	<marker id="start"></marker>
       </desc>
     </func>
     
@@ -101,18 +112,24 @@
       </type>
       <desc>
         <p>Starts the Inets application. Default type
-          is temporary. See also
-          <seealso marker="kernel:application">application(3)</seealso></p>
+	is temporary. See also
+	<seealso marker="kernel:application">application(3)</seealso>. </p>
+
+	<marker id="stop"></marker>
       </desc>
     </func>
+
     <func>
       <name>stop() -> ok </name>
       <fsummary>Stops the inets application.</fsummary>
       <desc>
         <p>Stops the inets application. See also
-          <seealso marker="kernel:application">application(3)</seealso></p>
+	<seealso marker="kernel:application">application(3)</seealso>. </p>
+	
+	<marker id="start2"></marker>
       </desc>
     </func>
+
     <func>
       <name>start(Service, ServiceConfig) -> {ok, Pid} | {error, Reason}</name>
       <name>start(Service, ServiceConfig, How) -> {ok, Pid} | {error, Reason}</name>
@@ -144,8 +161,11 @@
             some sense the calling process has now become the top
             supervisor.</p>
         </note>
+	
+	<marker id="stop2"></marker>
       </desc>
     </func>
+
     <func>
       <name>stop(Service, Reference) -> ok | {error, Reason} </name>
       <fsummary>Stops a started service of the inets application or takes
@@ -157,9 +177,11 @@
       </type>
       <desc>
         <p>Stops a started service of the inets application or takes
-          down a "stand_alone-service" gracefully. When the
-	  <c>stand_alone</c> option is used in start,
-	  only the pid is a valid argument to stop.</p>
+	down a "stand_alone-service" gracefully. When the
+	<c>stand_alone</c> option is used in start,
+	only the pid is a valid argument to stop.</p>
+	
+	<marker id="see_also"></marker>
       </desc>
     </func>
   </funcs>
diff --git a/lib/inets/doc/src/mod_esi.xml b/lib/inets/doc/src/mod_esi.xml
index e81308a502..9674cd9a88 100644
--- a/lib/inets/doc/src/mod_esi.xml
+++ b/lib/inets/doc/src/mod_esi.xml
@@ -1,10 +1,10 @@
-<?xml version="1.0" encoding="latin1" ?>
+<?xml version="1.0" encoding="iso-8859-1" ?>
 <!DOCTYPE erlref SYSTEM "erlref.dtd">
 
 <erlref>
   <header>
     <copyright>
-      <year>1997</year><year>2010</year>
+      <year>1997</year><year>2011</year>
       <holder>Ericsson AB. All Rights Reserved.</holder>
     </copyright>
     <legalnotice>
@@ -32,16 +32,19 @@
   <modulesummary>Erlang Server Interface </modulesummary>
   <description>
     <p>This module defines the API - Erlang Server Interface (ESI).
-      Which is a more efficient way of writing erlang scripts
-      for your Inets web server than writing them as common CGI scripts.</p>
+    Which is a more efficient way of writing erlang scripts
+    for your Inets web server than writing them as common CGI scripts.</p>
+
+    <marker id="deliver"></marker>
   </description>
+
   <funcs>
     <func>
       <name>deliver(SessionID, Data) -> ok | {error, Reason}</name>
       <fsummary>Sends Data back to client.</fsummary>
       <type>
         <v>SessionID = term()</v>
-        <v>Data = string() | io_list()</v>
+        <v>Data = string() | io_list() | binary()</v>
         <v>Reason = term()</v>
       </type>
       <desc>
@@ -51,13 +54,15 @@
 	parts of the content to the user.</p>
 	<p>Sends data from a Erl Scheme script back to the client.</p>
 	
-	<note><p>Note
-          that if any HTTP-header fields should be added by the
-          script they must be in the first call to deliver/2  and the
-          data in the call must be a string. Do not
-          assume anything about the data type of SessionID, the
-          SessionID must be the value given as input to the esi
-          call back function that you implemented.</p></note>
+	<note>
+	  <p>Note that if any HTTP-header fields should be added by the
+	  script they must be in the first call to deliver/2  and the
+	  data in the call must be a string. Calls after the headers
+	  are complete may contain binary data to reduce copying
+	  overhead.  Do not assume anything about the data type of 
+	  SessionID, the SessionID must be the value given as input to 
+	  the esi call back function that you implemented.</p>
+	</note>
       </desc>
     </func>
   </funcs>
diff --git a/lib/inets/doc/src/mod_security.xml b/lib/inets/doc/src/mod_security.xml
index 2a871d29d8..a3c91dca5b 100644
--- a/lib/inets/doc/src/mod_security.xml
+++ b/lib/inets/doc/src/mod_security.xml
@@ -1,10 +1,10 @@
-<?xml version="1.0" encoding="latin1" ?>
+<?xml version="1.0" encoding="iso-8859-1" ?>
 <!DOCTYPE erlref SYSTEM "erlref.dtd">
 
 <erlref>
   <header>
     <copyright>
-      <year>1998</year><year>2010</year>
+      <year>1998</year><year>2011</year>
       <holder>Ericsson AB. All Rights Reserved.</holder>
     </copyright>
     <legalnotice>
diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml
index 5da9d98002..0926df8581 100644
--- a/lib/inets/doc/src/notes.xml
+++ b/lib/inets/doc/src/notes.xml
@@ -32,6 +32,140 @@
     <file>notes.xml</file>
   </header>
   
+  <section><title>Inets 5.6</title>
+
+    <section><title>Improvements and New Features</title>
+<!--
+    <p>-</p>
+-->
+      <list>
+        <item>
+          <p>[httpc] Add support for upload body streaming (PUT and POST).</p>
+	  <p>For more info, 
+	  see the definition of the <c>Body</c> argument of the 
+	  <seealso marker="httpc#request2">request/4,5</seealso> 
+	  function. </p>
+	  <p>Filipe David Manana</p>
+	  <p>Own Id: OTP-9094</p>
+        </item>
+
+        <item>
+          <p>[ftp] Added (type) spec for all exported functions.</p>
+          <p>Own Id: OTP-9114 Aux Id: seq11799</p>
+        </item>
+
+        <item>
+          <p>[httpd] 
+	  <seealso marker="mod_esi#deliver">mod_esi:deliver/2</seealso> 
+	  made to accept binary data. </p>
+          <p>Bernard Duggan</p>
+          <p>Own Id: OTP-9123</p>
+        </item>
+
+        <item>
+          <p>[httpd] Prevent XSS in error pages.
+	  Prevent user controlled input from being interpreted 
+	  as HTML in error pages by encoding the reserved HTML 
+	  characters. </p>
+	  <p>Michael Santos</p>
+          <p>Own Id: OTP-9124</p>
+        </item>
+
+        <item>
+          <p>[httpd] Improved error messages. </p>
+	  <p>Ricardo Catalinas Jim�nez</p>
+          <p>Own Id: OTP-9157</p>
+        </item>
+
+        <item>
+          <p>[httpd] Extended support for file descriptors. 
+	  In order to be able to bind to a privileged port 
+	  without running the erlang VM as root, the support 
+	  for using file descriptors has been improved. 
+	  It is now possible to add the file descriptor to the config 
+	  (option fd) when calling the 
+	  <seealso marker="inets#start2">inets:start(httpd, ...)</seealso> 
+	  function. </p>
+	  <p>Attila Rajmund Nohl</p>
+          <p>Own Id: OTP-9202</p>
+          <p>Aux Id: seq11819</p>
+        </item>
+
+        <item>
+          <p>The default ssl kind has now been changed to <c>essl</c>. </p>
+	  <p><c>ossl</c> will work for as long as the ssl application 
+	  supports it. </p>
+	  <p>See the httpd 
+	  <seealso marker="httpd#comm_prop">socket_type</seealso> 
+	  communication property or the httpc 
+	  <seealso marker="httpc#request2">request/4,5</seealso> function
+	  for more info. </p>
+	  <p>Own Id: OTP-9230</p>
+	  <p>*** POTENTIAL INCOMPATIBILITY ***</p>
+        </item>
+
+      </list>
+    </section>
+
+    <section><title>Fixed Bugs and Malfunctions</title>
+<!--
+    <p>-</p>
+-->
+
+      <list>
+        <item>
+          <p>[httpd] Wrong 
+	  <seealso marker="httpd#sec_prop">security property</seealso> 
+	  names used in documentation. </p>
+	  <p><c>security_data_file</c> used instead of <c>data_file</c>. </p>
+	  <p><c>security_max_retries</c> used instead of <c>max_retries</c>. </p>
+	  <p><c>security_block_time</c> used instead of <c>block_time</c>. </p>
+	  <p><c>security_fail_expire_time</c> used instead of <c>fail_expire_time</c>. </p>
+	  <p><c>security_auth_timeout</c> used instead of <c>auth_timeout</c>. </p>
+	  <p>Garrett Smith</p>
+	  <p>Own Id: OTP-9131</p>
+        </item>
+
+        <item>
+          <p>[httpd] Fix timeout message generated by mod_esi.
+	  When a mod_esi request times out, the code to send a 
+	  timeout response was incorrect and generated an 
+	  internal server error as well as an invalid response 
+	  line. </p>
+	  <p>Bernard Duggan</p>
+	  <p>Own Id: OTP-9158</p>
+        </item>
+
+        <item>
+          <p>[httpc] httpc manager crashes. 
+	  When a request results in a retry, the request id will be "reused" 
+	  in the previous implementation a race condition could occur causing 
+	  the manager to crash. </p>
+	  <p>This is now avoided by using proc_lib:init_ack and 
+	  gen_server:enter_loop to allow mor advanced initialization of 
+	  httpc_handlers without blocking the httpc_manger and eliminating 
+	  extra processes that can cause race conditions. </p>
+          <p>Own Id: OTP-9246</p>
+        </item>
+
+        <item>
+          <p>[httpc] Issuing a request (<c>httpc:request</c>) to an 
+	  host with the ssl option 
+	  <c>{ip, {127,0,0,1}}</c> results in an handler crash. 
+	  The reason was that the connect call resulted in an exit with 
+	  reason <c>badarg</c> 
+	  (this was the same for both <c>ssl</c> and <c>gen_tcp</c>). </p>
+	  <p>Exits was not catched. This has now been improved. </p>
+          <p>Own Id: OTP-9289</p>
+          <p>Aux Id: seq11845</p>
+        </item>
+
+      </list>
+    </section>
+
+  </section> <!-- 5.6 -->
+
+
   <section><title>Inets 5.5.2</title>
 
     <section><title>Improvements and New Features</title>
@@ -80,7 +214,8 @@
 	  are URL-encoded. Added support in http-client to use
 	  URL-encoding. Also added the missing include directory
 	  for the inets application.</p>
-          <p>Own Id: OTP-8940 Aux Id: seq11735 </p>
+          <p>Own Id: OTP-8940</p>
+          <p>Aux Id: seq11735</p>
         </item>
       </list>
     </section>